Information processing apparatus, information processing method, and program

Abstract

It is determined whether each processing target existing in the real world recognizes the virtual content added to the real world.SOLUTION: An information processing apparatus according to the present disclosure includes an output unit and a determination unit. The output unit causes the virtual content outputting device to output the virtual content in a state of being superimposed on the real world. The determination unit determines whether the processing target recognizes the virtual content on the basis of a reaction to the output of the virtual content by the processing target existing in the real world.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] A technology related to this disclosure is disclosed in Patent Document 1. Patent Document 1 discloses a technology for displaying a virtual object superimposed on the real world. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2014 / 171200 Summary of the Invention [Problem to be solved by the invention]

[0004] Technologies for providing various contents to processing targets existing in the real world are being considered. One example is the use of technologies such as AR (Augmented Reality) and MR (Mixed Reality), which add virtual content (digital information) to the real world. By using these technologies, for example, by placing virtual content linked to a certain location in the real world, it is possible to provide the virtual content to processing targets located at that location.

[0005] However, not all processing targets in the real world are in a state where they can recognize virtual content, and there may be a mixture of processing targets that can recognize virtual content and processing targets that cannot recognize virtual content. Therefore, the inventors have discovered a new problem of determining whether each processing target in the real world recognizes virtual content added to the real world.

[0006] An example of a purpose of this disclosure is to provide a technology for determining whether or not each processing target present in the real world recognizes virtual content added to the real world. [Means for solving the problem]

[0007] According to one aspect of this disclosure, an output means for causing a virtual content output device to output virtual content by superimposing it on the real world; a determination means for determining whether the processing target present in the real world recognizes the virtual content based on a reaction of the processing target to an output of the virtual content; An information processing device having the above configuration is provided.

[0008] According to one aspect of this disclosure, One or more computers The virtual content output device outputs virtual content superimposed on the real world, There is provided an information processing method for determining whether a processing target existing in the real world recognizes the virtual content based on a reaction of the processing target to an output of the virtual content.

[0009] According to one aspect of this disclosure, Computer, an output means for causing a virtual content output device to output virtual content superimposed on the real world; a determination means for determining whether a processing target existing in the real world recognizes the virtual content based on a reaction of the processing target to an output of the virtual content; A program is provided to function as a [Effects of the Invention]

[0010] According to one example of this disclosure, a technique is realized for determining whether or not each processing target existing in the real world recognizes virtual content added to the real world. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an example of a functional block diagram of an information processing device. [Figure 2] FIG. 2 is a flowchart showing an example of the flow of processing by the information processing device. [Figure 3] FIG. 3 is a diagram illustrating an example of a hardware configuration of an information processing device. [Figure 4] FIG. 4 is a diagram illustrating an example of output of virtual content by the information processing device. [Figure 5] FIG. 5 is a diagram showing another example of output of virtual content by the information processing device. [Figure 6] FIG. 6 is a diagram showing another example of output of virtual content by the information processing device. [Figure 7] FIG. 7 is a flowchart showing another example of the flow of processing by the information processing device. [Figure 8] FIG. 8 is a flowchart showing another example of the flow of processing by the information processing device. [Figure 9] FIG. 9 is a flowchart showing another example of the flow of processing by the information processing device. [Figure 10] FIG. 10 is a diagram illustrating another example of a functional block diagram of an information processing device. [Figure 11] FIG. 11 is a flowchart showing another example of the flow of processing by the information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In this disclosure, the drawings relate to one or more embodiments. In all drawings, similar components are designated by similar reference numerals, and descriptions thereof will be omitted as appropriate.

[0013] <<First embodiment>>

[0014] Fig. 1 is a functional block diagram showing an overview of an information processing device 10. Fig. 2 is a flowchart showing an example of the flow of processing executed by the information processing device 10.

[0015] 1, the information processing device 10 includes an output unit 11 and a determination unit 12. These functional units execute the process shown in the flowchart of FIG.

[0016] In S10, the output unit 11 causes the virtual content output device to output the virtual content by superimposing it on the real world. In S11, the determination unit 12 determines whether the processing target existing in the real world recognizes the virtual content based on the "reaction to the output of the virtual content in S10" by the processing target.

[0017] The reaction of the processing target to the output of virtual content may differ depending on whether or not the processing target recognizes the virtual content. If the processing target recognizes the virtual content, it is considered that the processing target will have some kind of reaction to the output of the virtual content. On the other hand, if the processing target does not recognize the virtual content, it is considered that the processing target will not have any reaction to the output of the virtual content.

[0018] The information processing device 10 can determine whether the processing target recognizes the virtual content based on the difference in the processing target's reaction to the output of such virtual content.

[0019] Incidentally, one example of a process for determining whether a processing target recognizes virtual content is a process for determining, through image analysis, whether the processing target is wearing a virtual content output device (such as an AR device or an MR device). In this example, if the processing target is wearing a virtual content output device, it is determined that the processing target recognizes the virtual content. On the other hand, if the processing target is not wearing a virtual content output device, it is determined that the processing target does not recognize the virtual content. However, this example has several problems.

[0020] First, there are cases where it is not easy to determine by image analysis whether a virtual content output device is being worn, such as in the case of a contact lens type device.

[0021] Furthermore, even if the processing target has a virtual content output device attached, the attached virtual content output device may not be in a state where it can output the virtual content. An output-enabled state is, for example, a state where the power is on and the function for outputting virtual content is on. Naturally, if the attached virtual content output device is not in a state where it can output the virtual content, the processing target does not recognize the virtual content. Even if it is possible to determine by image analysis whether a virtual content output device is attached, it is not easy to determine by image analysis whether the attached virtual content output device is in a state where it can output the virtual content.

[0022] Thus, there is a problem with the process of "determining whether the processing target recognizes virtual content" by "determining whether the processing target is equipped with a virtual content output device through image analysis."

[0023] In contrast, the above problem can be alleviated by the information processing device 10, which determines whether a processing target recognizes virtual content based on the processing target's reaction to the output of virtual content. That is, the information processing device 10 determines whether a processing target recognizes virtual content based on the "reaction of the processing target," which can be relatively easily identified by image analysis. Such an information processing device 10 does not need to determine whether the processing target is wearing a device, such as a contact lens-type device, whose wearing is difficult to determine from its appearance. Furthermore, the information processing device 10 does not need to "determine whether the virtual content output device worn by the processing target is in a state where it can output virtual content," which is difficult to determine by image analysis.

[0024] In this way, the information processing device 10 realizes a technique for determining whether or not a processing target existing in the real world recognizes virtual content added to the real world.

[0025] <<Second embodiment>> <Summary> The information processing device 10 of the second embodiment is a specific implementation of the configuration of the information processing device 10 of the first embodiment, which will be described in detail below.

[0026] <Hardware configuration> First, an example of the hardware configuration of the information processing device 10 will be described. Each functional unit of the information processing device 10 is realized by any combination of hardware and software. Those skilled in the art will understand that there are various variations in the realization method and device. The software includes programs that are pre-loaded when the device is shipped, and programs downloaded from recording media such as CDs (Compact Discs) or servers on the Internet.

[0027] FIG. 3 is a block diagram illustrating an example of the hardware configuration of an information processing device 10. As shown in FIG. 3, the information processing device 10 has a processor 1A, a memory 2A, an input / output interface 3A, a peripheral circuit 4A, and a bus 5A. The peripheral circuit 4A includes various modules. The information processing device 10 does not necessarily have to have the peripheral circuit 4A. Note that the information processing device 10 may be composed of multiple devices that are physically and / or logically separated. In this case, each of the multiple devices can have the above hardware configuration.

[0028] The bus 5A is a data transmission path for the processor 1A, memory 2A, peripheral circuit 4A, and input / output interface 3A to mutually transmit and receive data. The processor 1A is, for example, a central processing unit (CPU) or a graphics processing unit (GPU). The memory 2A is, for example, a random access memory (RAM) or a read-only memory (ROM). The input / output interface 3A includes interfaces for acquiring information from input devices, external devices, external servers, external sensors, cameras, etc., and interfaces for outputting information to output devices, external devices, external servers, etc. The input / output interface 3A also includes an interface for connecting to a communication network such as the Internet. Examples of input devices include a keyboard, mouse, microphone, physical buttons, and touch panel. Examples of output devices include a display, projector, speaker, printer, and mailer. The processor 1A can issue commands to each module and perform calculations based on the results of those calculations.

[0029] <Functional configuration> Next, a detailed description will be given of the functional configuration of the information processing device 10. Fig. 1 is an example of a functional block diagram of the information processing device 10. As shown in the figure, the information processing device 10 has an output unit 11 and a determination unit 12. Each functional unit will be described in detail below.

[0030] The output unit 11 causes the virtual content output device to output the virtual content by superimposing it on the real world.

[0031] A "virtual content output device" is a device that has the function of outputting virtual content superimposed on the real world, such as an AR device or an MR device. The virtual content output device can output virtual content via various output devices. The virtual content output device can include at least one output device selected from the group consisting of a display, a projection device, a speaker, an odor output device, a fan, a vibrator, and a light. A processing object existing in the real world can use the virtual content output device.

[0032] Examples of devices for outputting virtual content include, but are not limited to, glass-type devices, contact lens-type devices, head-mounted displays, other wearable devices, smartphones, tablet devices, personal computers, game consoles, etc.

[0033] The information processing device 10 is communicably connected to a virtual content output device. The output unit 11 transmits information about the virtual content (image data, audio data, information indicating the content of the smell, vibration, light, etc. to be output) to the virtual content output device, and causes the virtual content output device to output the virtual content.

[0034] A "processing target" is a target that can experience (view, etc.) virtual content using a virtual content output device. A processing target is a target of processing that determines whether it recognizes virtual content. Processing targets exist in the real world. Processing targets include, but are not limited to, people, animals, robots, etc. Among processing targets, there may be processing targets that experience (view, etc.) virtual content using a virtual content output device and processing targets that do not use a virtual content output device (i.e., do not experience (view, etc.) virtual content).

[0035] "Virtual content" is content that does not exist in the real world, but is output superimposed on the real world via a virtual content output device. A processing subject who experiences (views, etc.) virtual content via a virtual content output device can experience the sensation of the virtual content existing in the real world. Furthermore, a processing subject who experiences (views, etc.) virtual content tends to have some kind of reaction to the output of the virtual content.

[0036] The virtual content includes at least one of an image, a sound, a smell, a wind, a vibration, and a light.

[0037] The "virtual content of the image" may include at least one of an obstacle, content that attracts the attention of the processing target, and guidance information that requests a predetermined action. The examples shown here are merely illustrative and are not limited to these.

[0038] An "obstacle" is, for example, something that hinders the action (passage, etc.) of the subject of processing, and examples include, but are not limited to, man-made objects such as buildings and vehicles, natural objects such as trees and rocks, people, animals, fictional creatures, and fictional objects.

[0039] "Content (image) that catches the attention of the processing subject" may be something that stands out from surrounding objects or something that the processing subject is interested in, but is not limited to these. For example, content (image) that catches the attention of the processing subject may be a character, an avatar, a person with a distinctive color, or an object with a distinctive color (including artificial objects, natural objects, animals, fictional creatures, fictional objects, etc.). Other content (image) that catches the attention of the processing subject may be a very large person or a very large object (including artificial objects, natural objects, animals, fictional creatures, fictional objects, etc.). Other content (image) that catches the attention of the processing subject may be a person making a predetermined movement (such as waving their hands vigorously) or an object making a predetermined movement (including artificial objects, natural objects, animals, fictional creatures, fictional objects, etc.). Note that the examples shown here are merely illustrative and are not limiting.

[0040] The "guidance information (image) requesting a predetermined action" includes, for example, a message such as "Please jump." In addition to or instead of a message, the guidance information (image) may include an image showing an instructor, such as a person, character, or avatar, performing a predetermined action. The "predetermined action" is preferably one that can be relatively easily identified by image analysis and that is not performed by a processing subject who is not viewing the guidance information (i.e., an action that the processing subject does not frequently perform in daily life). Examples of the predetermined action include, but are not limited to, jumping, raising a hand, waving, and lifting one leg.

[0041] The "audio virtual content" may include at least one of audio that attracts the attention of a processing target and guidance information that requests a predetermined action. The examples shown here are merely examples and are not limited to these.

[0042] The "sound that attracts the attention of the processing subject" is, for example, a sound that is not frequently heard in daily life, a sound that the processing subject is interested in, etc., but is not limited to these. For example, the sound that attracts the attention of the processing subject may be the sound of thunder, the sound of rain, the sound of wind, a warning sound, a siren, the sound of braking of a moving object, a scream, etc. Note that the examples given here are merely examples and are not limited to these.

[0043] The "guidance information (audio) requesting a predetermined action" includes, for example, a message such as "Please jump." The "predetermined action" is preferably one that can be relatively easily identified by image analysis and that a processing subject who is not viewing the guidance information would not perform (i.e., an action that the processing subject does not frequently perform in daily life). Examples of the predetermined action include, but are not limited to, jumping, raising a hand, waving a hand, and raising one leg.

[0044] The "virtual odor content" may include an odor that attracts the attention of the processing target. Examples of odors that attract the attention of the processing target include, but are not limited to, odors that are not frequently smelled in daily life or odors that attract people's interest. For example, the odor that attracts the attention of the processing target may be a food odor, a pungent odor, or the like. Note that the examples shown here are merely examples and are not limited to these.

[0045] The "virtual wind content" may include wind that draws the attention of the processing subject, such as, but not limited to, wind of a strength that is not frequently encountered in daily life.

[0046] The "vibration virtual content" may include a vibration that can attract the attention of a processing subject, such as, but not limited to, a vibration of a predetermined strength.

[0047] The "virtual light content" may include light that attracts the attention of the processing subject, such as, but not limited to, light or lightning that is not frequently encountered in daily life.

[0048] Next, an example of processing in which the output unit 11 causes a virtual content output device to output virtual content will be described.

[0049] First, the output unit 11 can determine the output mode of the virtual content. The output unit 11 can determine at least one of the following. Timing for outputting virtual content overlaid on the real world Position where virtual content is output overlaid on the real world - Virtual content to be superimposed on the real world

[0050] The output unit 11 can make the above determination in accordance with predetermined rules. For example, the output unit 11 may randomly determine the timing, position, content, etc. Alternatively, the output unit 11 may arrange and output virtual content of predetermined content at a predetermined position at a predetermined time interval.

[0051] Additionally, the output unit 11 may determine the timing, position, content, etc. based on the analysis results of an image captured of the real world.

[0052] For example, the output unit 11 may analyze the image to detect a processing target from within the image. The output unit 11 may also analyze the image to estimate attributes of the processing target detected from the image. Examples of the attributes of the processing target include, but are not limited to, human characteristics (gender, age, nationality, etc.), animal characteristics (type, gender, age, etc.), and robot characteristics (manufacturer, type, model number, etc.).

[0053] The output unit 11 may then output the virtual content by placing it at a position that appears in the image when the analysis result of the image satisfies a predetermined condition. The predetermined condition may be, for example, "a predetermined number or more of processing objects are detected in the image." Alternatively, the predetermined condition may be, for example, "a predetermined number or more of processing objects with a predetermined attribute (e.g., female) are detected." Note that the predetermined conditions exemplified here are merely examples and are not limited to these. By appropriately setting the predetermined condition, the virtual content can be output at a position and timing when the presence situation of the processing objects in the real world is a desired situation, and it can be determined whether the processing objects present in that location recognize the virtual content.

[0054] In an example in which the timing, position, content, etc. are determined based on the analysis results of an image captured in the real world, the information processing device 10 may have a function of acquiring an image captured in the real world. For example, the information processing device 10 may be communicably connected to an external device and acquire images generated by a surveillance camera installed in the real world from the external device. The external device may be a surveillance camera or a storage device that accumulates images generated by the surveillance camera. The information processing device 10 may acquire images generated by the surveillance camera through real-time processing.

[0055] A surveillance camera can generate (take) at least one of still images and moving images. A surveillance camera may detect visible light and create an image, or may detect other electromagnetic waves such as infrared light and create an image. A surveillance camera may be installed at a predetermined location and continuously capture images from that location. Alternatively, a surveillance camera may be mounted on a mobile object and capture images from various locations. In this case, at least one of the mobile object and the surveillance camera has a current position identification function and a function to generate information indicating the image capture position. The output unit 11 can identify the capture position of each image, for example, based on this information. Alternatively, the output unit 11 may identify the capture position of each image by detecting landmarks from within the image.

[0056] A mobile body equipped with a surveillance camera may have the ability to move on land, in the air, or in or on water. The mobile body may have the ability to move autonomously or move under external control.

[0057] "Acquisition" includes at least one of the following: a device going to retrieve data or information stored in another device or storage medium (active acquisition), and a device inputting data or information output from another device (passive acquisition). Examples of active acquisition include making a request to another device and receiving a response, and accessing and reading information from another device or storage medium. An example of passive acquisition is receiving information that is distributed (or transmitted, push notification, etc.). Furthermore, acquisition may involve selecting and acquiring data or information from received data or information, or selecting and receiving distributed data or information.

[0058] Additionally, the output unit 11 may determine the timing, position, content, etc., taking safety into consideration. Details of this processing will be described in the following embodiments.

[0059] After determining the output mode of the virtual content, the output unit 11 causes the virtual content output device to output the virtual content in the determined output mode. For example, the output unit 11 places the virtual content with the determined content at the determined timing, linked to the determined position (designated position) in real space. Then, the output unit 11 transmits the virtual content, linked to and placed at the designated position, to a virtual content output device located near the designated position. The virtual content output device outputs the received virtual content by superimposing it on the real world.

[0060] For example, when a camera is pointed at a specified position, the virtual content output device can superimpose an image of the virtual content at a predetermined position on the image generated by the camera and display it. Furthermore, when the distance to the specified position is within a threshold, the virtual content output device can output at least one of the virtual content types selected from sound, smell, wind, vibration, and light.

[0061] The determination unit 12 determines whether a processing target existing in the real world recognizes virtual content based on the "reaction to the output of virtual content" by the processing target. The determination unit 12 can determine whether each processing target recognizes virtual content based on the "reaction to the output of virtual content" by each of multiple processing targets existing in the real world.

[0062] The determination unit 12 detects the "reaction to the output of virtual content" by the processing target based on at least one of images generated by a surveillance camera present in the real world and data (sensing data) generated by other sensors present in the real world.

[0063] The surveillance cameras present in the real world and the images generated by the surveillance cameras are as described above.

[0064] The sensor may be carried by each processing object and sense the data of each processing object. Alternatively, the sensor may be installed at a predetermined location and sense the data of an unspecified number of processing objects present at that location. Alternatively, the sensor may be mounted on a mobile object and sense the data of an unspecified number of processing objects present at various locations. In this case, at least one of the mobile object and the sensor has a current location identification function and a function to generate information indicating the sensing position of the sensed data. The output unit 11 can identify the sensing position of each data based on that information.

[0065] The mobile body equipped with the sensor may have the function of moving on land, in the air, or in or on water. The mobile body may have the function of moving autonomously or may have the function of moving under external control.

[0066] Examples of sensors include, but are not limited to, radar, LiDAR (Light Detection and Ranging), temperature sensors, weight sensors, human presence sensors, attitude detection sensors, speed sensors, and acceleration sensors.

[0067] The determination unit 12 can detect whether each processing target has made a predetermined reaction based on an image generated by a surveillance camera or data generated by a sensor. For example, the determination unit 12 detects processing targets from the image generated by a surveillance camera or data generated by a sensor. Then, the determination unit 12 detects whether each of the detected processing targets has made a predetermined reaction.

[0068] A "predetermined reaction" is a reaction that a processing target may have in response to the output of virtual content, after experiencing (viewing, etc.) the virtual content. Reactions to be detected are determined in advance. Then, the determination unit 12 detects that the processing target has performed one of these reactions.

[0069] Note that if the content of the virtual content differs, the content of the reaction that the processing target can perform may also differ. Therefore, if there are multiple contents of virtual content that the output unit 11 can output, the reaction to be detected may be determined for each content of virtual content.

[0070] For example, when an obstacle (virtual content B) is present in front of the processing target H as shown in FIG. 4, the processing target H is likely to perform an action such as stopping in front of the obstacle or avoiding the obstacle.

[0071] Furthermore, as shown in Figure 5, if there is content (virtual content B) that attracts attention around the processing target H, it is thought that the processing target H will perform actions such as turning its gaze toward the content, turning its face toward the content, or turning its body toward the content.

[0072] Furthermore, as shown in FIG. 6, when guidance information (virtual content B) requesting a predetermined action is present in front of the processing target H, it is considered that the processing target H will perform the predetermined action.

[0073] As in these examples, the content of the predetermined reaction varies, but the predetermined reaction may include, for example, at least one of the following: Note that the examples shown here are merely examples and are not limited to these. - Stopping in front of virtual content that is an obstacle -Avoiding virtual content that is an obstacle (changing direction without crashing into an obstacle, etc.) The action of directing one's gaze towards virtual content that attracts the processing target's attention - Turning your face towards the virtual content that is the content that attracts your attention - The action of turning one's body towards the virtual content that is attracting the processing subject's attention - A specific action requested by guidance information (virtual content) that requires a specific action - Predetermined reactions to sounds, smells, wind, vibrations, or light that attract the subject's attention (startling, looking around, looking up, etc.)

[0074] The determination unit 12 can analyze images generated by a surveillance camera or data generated by a sensor to detect a processing target that has made a predetermined reaction. The determination unit 12 can then determine that a processing target that has made a predetermined reaction recognizes virtual content. The determination unit 12 can acquire information indicating the content, position, timing, etc. of the output virtual content from the output unit 11. Based on this information, the determination unit 12 can then identify the content of the virtual content, the timing, and the position at which the output virtual content is being output. The determination unit 12 can then use the identification result to detect the processing target's reaction to the output of the virtual content as described above. Note that the detection of the above-described reaction can be achieved using widely known technology.

[0075] As a modified example, the determination unit 12 may determine that a processing object that has performed a predetermined reaction to the output of virtual content "a predetermined number of times or more" recognizes the virtual content. A processing object that does not recognize the virtual content may happen to perform an action that is the same as the predetermined reaction described above. In this case, there is a risk that the processing object that does not recognize the virtual content may be erroneously determined to recognize the virtual content based on the action that happened to be the same as the predetermined reaction performed by the processing object. By determining that a processing object that has performed a predetermined reaction to the output of virtual content "a predetermined number of times or more" recognizes the virtual content as such, as in this modified example, the occurrence of such erroneous determinations can be reduced.

[0076] In this modification, the output unit 11 causes the virtual content output device to output virtual content multiple times, superimposed on the real world. The determination unit 12 can determine whether each processing target recognizes the virtual content based on the processing target's reaction to the multiple outputs of the virtual content. The output unit 11 may output the same virtual content or different virtual content multiple times.

[0077] Next, an example of the processing flow of the information processing device 10 will be described using the flowchart in Fig. 2. Note that the purpose here is to explain the processing flow. Since the details of each process have been described above, the explanation here will be omitted as appropriate.

[0078] In S10, the information processing device 10 causes the virtual content output device to output virtual content superimposed on the real world. For example, the information processing device 10 can determine the output mode (timing, position, content, etc.) of the virtual content. After determining the output mode of the virtual content, the information processing device 10 can cause the virtual content output device to output the virtual content in the determined output mode.

[0079] In S11, the information processing device 10 determines whether the processing target existing in the real world recognizes the virtual content based on the “reaction of the processing target to the output of the virtual content in S10” by the processing target. For example, the information processing device 10 can determine whether each of the multiple processing targets existing in the real world recognizes the virtual content based on the reaction of each of the multiple processing targets.

[0080] <Action and effect> According to the information processing device 10 of the second embodiment, it is possible to achieve the same effects as those of the information processing device 10 of the first embodiment.

[0081] Incidentally, the information processing device 10 determines whether the processing target recognizes the virtual content based on the processing target's reaction to the output of the virtual content. For this reason, it is necessary to output virtual content that can elicit an appropriate reaction from the processing target.

[0082] Therefore, the information processing device 10 can superimpose virtual content, which is at least one image of an "obstacle," "content that attracts the attention of the processing subject," and "guidance information that requests a predetermined action," on the real world and display it on the virtual content output device. When such distinctive virtual content is output, it is possible to elicit reactions such as "stopping in front of the obstacle," "avoiding the obstacle," "directing one's gaze, face, body, etc. toward the content that attracts the processing subject's attention," and "performing the requested predetermined action" with a high probability.

[0083] Furthermore, the information processing device 10 can superimpose virtual content, which is at least one of "audio that attracts the attention of the processing target" and "guidance information that requests a predetermined action," on the real world and output the virtual content output device as sound. When such distinctive virtual content is output, it is possible to elicit reactions such as "performing the requested predetermined action" or "performing a predetermined reaction (being surprised, looking around, looking up at the sky, etc.)" with a high probability.

[0084] Furthermore, the information processing device 10 can superimpose virtual content, which is "at least one of smell, wind, vibration, and light that attracts the attention of the processing subject," on the real world and output it to the virtual content output device. When such distinctive virtual content is output, it is possible to elicit "a predetermined reaction (such as being surprised, looking around, or looking up at the sky)" with a high probability.

[0085] The information processing device 10 capable of outputting such distinctive virtual content can elicit a predetermined reaction from a processing target who experiences (views, etc.) the virtual content with a high probability, and can accurately determine whether the processing target recognizes the virtual content.

[0086] Furthermore, the information processing device 10 determines whether the processing target recognizes the virtual content based on the detection result of "the processing target's reaction, which can be detected relatively easily by image analysis or sensing data analysis," as exemplified above. Such information processing device 10 can accurately determine whether the processing target recognizes the virtual content.

[0087] <<Third embodiment>> In the third embodiment, the information processing device 10 outputs virtual content to a virtual world that is merged with the real world, causing a virtual content output device to output the virtual content superimposed on the real world. Then, the information processing device 10 determines whether the processing target recognizes the virtual world based on the determination result of whether the processing target recognizes the virtual content. This will be described in detail below.

[0088] 1 is an example of a functional block diagram of an information processing device 10. As shown in the figure, the information processing device 10 has an output unit 11 and a determination unit 12. Each functional unit will be described in detail below.

[0089] The output unit 11 differs from that of the second embodiment in the method of outputting virtual content by superimposing it on the real world. The other configurations of the output unit 11 can be the same as those of the first and second embodiments.

[0090] The output unit 11 outputs virtual content to a virtual world that is merged with the real world. That is, the output unit 11 can place obstacles, content that attracts the attention of the processing subject, etc., in the virtual world. The output unit 11 can also output at least one virtual content of sound, smell, wind, vibration, and light to the virtual world. The processing subject experiencing a world where the real world and the virtual world are merged can experience (view, etc.) the virtual content output to the virtual world. The output by the output unit 11 is realized using widely known technology.

[0091] The determination unit 12 determines whether the processing target recognizes the virtual content using the same method as in the first and second embodiments. Then, based on the result of the determination, the determination unit 12 determines whether the processing target recognizes the virtual world.

[0092] The determination unit 12 can determine that a processing object determined to be aware of virtual content is aware of the virtual world, and can determine that a processing object determined not to be aware of virtual content is not aware of the virtual world.

[0093] Next, an example of the processing flow of the information processing device 10 will be described using the flowchart in Fig. 7. Note that the purpose here is to explain the processing flow. Since the details of each process have been described above, the explanation here will be omitted as appropriate.

[0094] In S20, the information processing device 10 outputs virtual content to a virtual world that is merged with the real world, causing a virtual content output device to output the virtual content superimposed on the real world. For example, the information processing device 10 can determine the output mode (timing, position, content, etc.) of the virtual content. After determining the output mode of the virtual content, the information processing device 10 can output the virtual content to the virtual world that is merged with the real world in the determined output mode.

[0095] In S21, the information processing device 10 determines whether the processing target existing in the real world recognizes the virtual content based on the "reaction of the processing target to the output of the virtual content in S20" by the processing target. For example, the information processing device 10 can determine whether each of the multiple processing targets existing in the real world recognizes the virtual content based on the reaction of each of the multiple processing targets.

[0096] In S22, the information processing device 10 determines whether the processing target is aware of the virtual world based on the determination result of S21. For example, the information processing device 10 can determine that a processing target determined to be aware of the virtual content is aware of the virtual world. Furthermore, the information processing device 10 can determine that a processing target determined not to be aware of the virtual content is not aware of the virtual world.

[0097] Other configurations of the information processing device 10 of the third embodiment can be similar to the configurations of the information processing device 10 of the first and second embodiments.

[0098] According to the information processing device 10 of the third embodiment, it is possible to achieve the same effects as those of the information processing device 10 of the first and second embodiments. In addition, the information processing device 10 can determine whether the processing target is aware of a virtual world that is merged with the real world.

[0099] <<Fourth embodiment>> In the fourth embodiment, the information processing device 10 outputs virtual content to a virtual world that is merged with the real world, causing a virtual content output device to output the virtual content superimposed on the real world. There are multiple virtual worlds that are merged with the real world. The information processing device 10 determines which of the multiple virtual worlds the processing target recognizes based on the determination result of whether the processing target recognizes the virtual content. This will be described in detail below.

[0100] 1 is an example of a functional block diagram of an information processing device 10. As shown in the figure, the information processing device 10 has an output unit 11 and a determination unit 12. Each functional unit will be described in detail below.

[0101] The output unit 11 outputs virtual content superimposed on the real world using the same method as in the third embodiment. That is, the output unit 11 outputs virtual content to a virtual world that is merged with the real world. The output unit 11 can place obstacles, content that attracts the attention of the processing subject, etc. in the virtual world. The output unit 11 can also output at least one virtual content of sound, smell, wind, vibration, and light to the virtual world. The processing subject experiencing a world where the real world and the virtual world are merged can experience (view, etc.) the virtual content output to the virtual world. The output by the output unit 11 is realized using widely known technology.

[0102] In the fourth embodiment, there are multiple virtual worlds that are integrated into the real world. A user can select a desired virtual world and experience (view, view, etc.) digital information that exists in that virtual world by superimposing it on the real world. For example, one virtual world may be aimed at women, and digital information aimed at women may exist there. Another virtual world may be aimed at children, and digital information aimed at children may exist there.

[0103] A server that provides digital information about the virtual world can transmit the digital information about the virtual world selected by each user to each user's virtual content output device, and each user's virtual content output device can receive the digital information about the virtual world selected by each user from the server and output it by superimposing it on the real world.

[0104] Alternatively, a server providing digital information about virtual worlds may transmit digital information about multiple virtual worlds to each user's virtual content output device, and each user's virtual content output device may extract digital information about a virtual world selected by the user from the digital information about the multiple virtual worlds received from the server, and output the extracted information by superimposing it on the real world.

[0105] The output unit 11 outputs virtual content to each of the multiple virtual worlds by changing at least one of the output timing, position, and content of the virtual content. As a result, the virtual content output devices of the processing targets experiencing different virtual worlds output virtual content that differs from one another in at least one of the output timing, position, and content. The processing targets experiencing different virtual worlds then react to the output of virtual content that differs from one another in at least one of the output timing, position, and content.

[0106] Other configurations of the output unit 11 can be the same as those in the first to third embodiments.

[0107] The determination unit 12 determines whether the processing target recognizes the virtual content using a method similar to that of the first to third embodiments. The determination unit 12 can determine whether the processing target recognizes the virtual content output to each of the multiple virtual worlds based on the processing target's reaction to the output of the virtual content to each of the multiple virtual worlds. Then, based on the result of the determination, the determination unit 12 identifies the virtual world that the processing target recognizes from among the multiple virtual worlds.

[0108] The determination unit 12 can determine that a processing object determined to recognize virtual content output to a first virtual world recognizes the first virtual world. The determination unit 12 can also determine that a processing object determined to recognize virtual content output to an Nth virtual world (N is an integer equal to or greater than 2) different from the first world recognizes the Nth virtual world. The determination unit 12 can also determine that a processing object determined not to recognize virtual content output to any virtual world does not recognize a virtual world.

[0109] Next, an example of the processing flow of the information processing device 10 will be described using the flowchart in Fig. 8. Note that the purpose here is to explain the processing flow. Since the details of each process have been described above, the explanation here will be omitted as appropriate.

[0110] In S30, the information processing device 10 outputs virtual content to a virtual world that is merged with the real world, causing the virtual content output device to output the virtual content superimposed on the real world. The information processing device 10 outputs the virtual content to each of the multiple virtual worlds by changing at least one of the output timing of the virtual content, the position of the virtual content, and the content of the virtual content.

[0111] For example, the information processing device 10 can determine the output mode (timing, position, content, etc.) of the virtual content in each of the multiple virtual worlds. After determining the output mode of the virtual content in each virtual world, the information processing device 10 can output the virtual content to each of the multiple virtual worlds that are merged with the real world in the determined output mode.

[0112] In S31, the information processing device 10 determines whether the processing target existing in the real world recognizes the virtual content based on the "reaction to the output of the virtual content in S30" by the processing target. The information processing device 10 can determine whether the processing target recognizes the virtual content output to each of the multiple virtual worlds based on the reaction of the processing target to the output of the virtual content to each of the multiple virtual worlds. The information processing device 10 can determine whether each of the multiple processing targets existing in the real world recognizes the virtual content based on the reaction by each of the multiple processing targets.

[0113] In S32, the information processing device 10 identifies the virtual worlds recognized by the processing target based on the result of the determination in S31. For example, the information processing device 10 can determine that a processing target determined to recognize virtual content output to a first virtual world recognizes the first virtual world. Furthermore, the information processing device 10 can determine that a processing target determined to recognize virtual content output to an Nth virtual world (N is an integer equal to or greater than 2) different from the first world recognizes the Nth virtual world. Then, the determination unit 12 can determine that a processing target determined not to recognize virtual content output to any virtual world does not recognize a virtual world.

[0114] Other configurations of the information processing device 10 of the fourth embodiment can be similar to the configurations of the information processing device 10 of the first to third embodiments.

[0115] According to the information processing device 10 of the fourth embodiment, it is possible to realize the same effects as those of the information processing device 10 of the first to third embodiments. Furthermore, the information processing device 10 can determine which virtual world, among a plurality of virtual worlds merged with the real world, the processing target recognizes.

[0116] <<Fifth Embodiment>> In the fifth embodiment, the information processing device 10 determines, taking safety into consideration, at least one of the timing at which the virtual content output device outputs the virtual content and the position at which the virtual content is displayed superimposed on the real world. This will be described in detail below.

[0117] 1 is an example of a functional block diagram of an information processing device 10. As shown in the figure, the information processing device 10 has an output unit 11 and a determination unit 12. Each functional unit will be described in detail below.

[0118] The output unit 11 determines, based on the environment in the real world, at least one of the timing at which the virtual content output device outputs the virtual content and the position at which the virtual content is to be displayed superimposed on the real world.

[0119] First, the output unit 11 can identify an environment in the real world. For example, information indicating what exists at what location in the real world may be stored in advance in the information processing device 10. Then, the output unit 11 may identify the environment in the real world based on that information. For example, a virtual space that reproduces the environment in the real world may be generated using technology such as a digital twin. Then, the output unit 11 may identify the environment in the real world based on this virtual space that reproduces the environment in the real world.

[0120] Through this processing, the output unit 11 can identify the positions of objects in the real world environment whose positions do not change much, for example. The objects here include at least one of artificial objects and natural objects, and may be, for example, stairs, crosswalks, ditches, rivers, ponds, etc. Note that the examples shown here are merely examples and are not limited to these.

[0121] Alternatively, the output unit 11 may identify the environment in the real world based on at least one of an image generated by a surveillance camera in the real world and data (sensing data) generated by other sensors in the real world. Through this processing, the output unit 11 can identify the position of an object whose position moves frequently as the environment in the real world. Furthermore, through this processing, the output unit 11 can identify the state of an object whose state changes frequently as the environment in the real world. The object here includes at least one of a person, an artificial object, and a natural object, and may be, for example, a person, a car, an animal, a traffic light, etc. Examples of the state of an object include the state of a traffic light (red or green), whether a car is moving, etc. Note that the example shown here is merely an example and is not limited to these.

[0122] In addition, the output unit 11 may further identify the position of the above-mentioned object whose position does not change much based on at least one of an image generated by a surveillance camera existing in the real world and data (sensing data) generated by other sensors existing in the real world.

[0123] After identifying the environment in the real world, the output unit 11 determines, based on the identification result, at least one of the timing at which the virtual content output device outputs the virtual content and the position at which the virtual content is to be displayed superimposed on the real world. Then, the output unit 11 causes the virtual content output device to output the virtual content at the determined timing and position.

[0124] The output unit 11 can determine at least one of the timing and position at which to output the virtual content in accordance with predetermined rules. The rules are defined to ensure safety, i.e., to prevent accidents or the like from occurring due to the output of the virtual content. The contents of the rules vary, but may include at least one of the following, for example: - Virtual content will not be output in crowded areas where the density of people exceeds a predetermined threshold. -Do not output virtual content in dangerous areas within a specified distance from predetermined hazards (rivers, stairs, etc.) -Do not output virtual content in dangerous areas within a specified distance from hazardous objects (red lights, moving cars, etc.) that are in a predetermined dangerous state. -Do not place virtual content during pre-determined busy times -Do not output virtual content when it is difficult to detect the reaction of the target -Do not output virtual content at times when the processing target may not notice the virtual content.

[0125] The "timing at which it is difficult to detect the reaction of the processing target" refers to a timing at which it is difficult to detect the reaction of the processing target. The difficult-to-detect state is defined in advance. The output unit 11 monitors whether the difficult-to-detect state has occurred using image analysis or the like, and outputs the virtual content at a timing at which the difficult-to-detect state has not occurred.

[0126] Examples of difficult-to-detect states include, but are not limited to, "the size of the processing target in the image generated by the surveillance camera is below a threshold" and "a predetermined percentage or more of the processing target's body is hidden in the image generated by the surveillance camera." The "timing at which the processing target may not notice the virtual content" refers to a timing at which the processing target is in a difficult-to-recognize state where it does not notice the virtual content. The difficult-to-recognize state is defined in advance. The output unit 11 monitors whether the processing target is in a difficult-to-recognize state by image analysis or the like, and outputs the virtual content at a timing at which the processing target is not in a difficult-to-recognize state.

[0127] The recognition difficulty state is, for example, while performing a predetermined task such as "while having a conversation" or "while playing sports", but is not limited to these.

[0128] Other configurations of the output unit 11 can be the same as those in the first to fourth embodiments. The configuration of the determination unit 12 can be the same as those in the first to fourth embodiments.

[0129] Next, an example of the processing flow of the information processing device 10 will be described using the flowchart in Fig. 9. Note that the purpose here is to explain the processing flow. Since the details of each process have been described above, the explanation here will be omitted as appropriate.

[0130] In S40, the information processing device 10 identifies the environment in the real world.

[0131] In S41, the information processing device 10 determines, based on the result of the identification in S40, at least one of the timing at which the virtual content output device outputs the virtual content and the position at which the virtual content is to be displayed superimposed on the real world.

[0132] In S42, the information processing device 10 causes the virtual content output device to output the virtual content superimposed on the real world based on the determination made in S41.

[0133] In S43, the information processing device 10 determines whether the processing target existing in the real world recognizes the virtual content, based on the reaction of the processing target to the output of the virtual content.

[0134] The information processing device 10 may determine whether the processing target is aware of a virtual world that is integrated with the real world, based on the determination result of S43. The information processing device 10 may also determine which of a plurality of virtual worlds that are integrated with the real world the processing target is aware of, based on the determination result of S43.

[0135] Other configurations of the information processing device 10 of the fifth embodiment can be similar to the configurations of the information processing device 10 of the first to fourth embodiments.

[0136] The information processing device 10 of the fifth embodiment can achieve the same effects as the information processing devices 10 of the first to fourth embodiments. Furthermore, the information processing device 10 can determine, based on the real-world environment and taking safety into consideration, at least one of the timing for outputting virtual content from a virtual content output device and the position of the virtual content to be displayed superimposed on the real world. This can prevent inconveniences such as unexpected accidents occurring in the real world due to the output of virtual content.

[0137] <<Sixth embodiment>> In the sixth embodiment, the information processing device 10 can perform a predetermined process based on the determination result by the determination unit 12. This will be described in detail below.

[0138] 10 is an example of a functional block diagram of the information processing device 10. As shown in the figure, the information processing device 10 has an output unit 11, a determination unit 12, and a processing unit 13. Each functional unit will be described in detail below.

[0139] The functions of the output unit 11 and the determination unit 12 can be the same as those in the first to fifth embodiments.

[0140] The processing unit 13 executes a predetermined process based on at least one of the determination result of the determination unit 12 and the statistical result of the determination result.

[0141] An example of the "predetermined processing" is providing information to a processing target present in the observation area. The processing unit 13 can change at least one of the method of providing information, the content of the information to be provided, and the destination of the information, based on at least one of the judgment result of the judgment unit 12 and the statistical result of the judgment result.

[0142] The "observation area" may be indoors or outdoors. The observation area may be, for example, inside a room, inside a facility, inside an event venue, a station platform, an airport waiting room, or other locations. The observation area may be photographed by a surveillance camera. The monitoring area may also be sensed by the various sensors described above.

[0143] The "determination result of the determination unit 12" includes at least one of the following. -Whether each processing object in the observation area recognizes the virtual content -Whether each processing object in the observation area recognizes the virtual world -Which of the multiple virtual worlds does each processing object in the observation area recognize?

[0144] The "statistical result of the judgment result" is generated by collecting statistics of the judgment results of the judgment unit 12 as described above.

[0145] For example, the processing unit 13 may count the number of processing targets that exist within the observation area and recognize virtual content, based on the determination result of the determination unit 12. Alternatively, the processing unit 13 may count the number of processing targets that exist within the observation area and do not recognize virtual content, based on the determination result of the determination unit 12. Alternatively, the processing unit 13 may calculate the proportion of processing targets that exist within the observation area and recognize virtual content, based on the determination result of the determination unit 12. Alternatively, the processing unit 13 may calculate the proportion of processing targets that do not recognize virtual content, based on the determination result of the determination unit 12.

[0146] Additionally, the processing unit 13 may count the number of processing targets that exist within the observation area and that recognize the virtual world, based on the determination result of the determination unit 12. The processing unit 13 may also count the number of processing targets that exist within the observation area and that do not recognize the virtual world, based on the determination result of the determination unit 12. The processing unit 13 may also calculate the proportion of processing targets that exist within the observation area and that recognize the virtual world, based on the determination result of the determination unit 12. The processing unit 13 may also calculate the proportion of processing targets that exist within the observation area and that do not recognize the virtual world, based on the determination result of the determination unit 12.

[0147] Additionally, the processing unit 13 may count, for each virtual world, the number of processing targets that exist within the observation area and that recognize each virtual world, based on the determination result of the determination unit 12. Furthermore, the processing unit 13 may count, for each virtual world, the number of processing targets that exist within the observation area and that do not recognize each virtual world, based on the determination result of the determination unit 12. Furthermore, the processing unit 13 may calculate, for each virtual world, the proportion of processing targets that exist within the observation area and that recognize each virtual world, based on the determination result of the determination unit 12. Furthermore, the processing unit 13 may calculate, for each virtual world, the proportion of processing targets that do not recognize each virtual world, based on the determination result of the determination unit 12.

[0148] Additionally, the processing unit 13 may count the number of processing targets with a predetermined attribute that are present in the observation area and recognize virtual content, based on the determination result of the determination unit 12 and the attributes of each processing target. Furthermore, the processing unit 13 may count the number of processing targets with a predetermined attribute that are present in the observation area and do not recognize virtual content, based on the determination result of the determination unit 12 and the attributes of each processing target. Furthermore, the processing unit 13 may calculate the proportion of processing targets with a predetermined attribute that recognize virtual content among the processing targets present in the observation area, based on the determination result of the determination unit 12 and the attributes of each processing target. Furthermore, the processing unit 13 may calculate the proportion of processing targets with a predetermined attribute that do not recognize virtual content among the processing targets present in the observation area, based on the determination result of the determination unit 12 and the attributes of each processing target.

[0149] The "attributes of the processing target" are estimated by analyzing images from a surveillance camera, for example. The attributes of the processing target include, but are not limited to, human characteristics (gender, age, nationality, etc.), animal characteristics (species, gender, age, etc.), and robot characteristics (manufacturer, type, model number, etc.).

[0150] Examples of "processing target with a predetermined attribute" include, but are not limited to, "female" and "female in her 20s or 30s."

[0151] Additionally, the processing unit 13 may count the number of processing objects with a predetermined attribute that are present in the observation area and that recognize the virtual world, based on the determination result of the determination unit 12 and the attributes of each processing object. Furthermore, the processing unit 13 may count the number of processing objects with a predetermined attribute that are present in the observation area and that do not recognize the virtual world, based on the determination result of the determination unit 12 and the attributes of each processing object. Furthermore, the processing unit 13 may calculate the proportion of processing objects with a predetermined attribute that recognize the virtual world, among the processing objects present in the observation area, based on the determination result of the determination unit 12 and the attributes of each processing object. Furthermore, the processing unit 13 may calculate the proportion of processing objects with a predetermined attribute that do not recognize the virtual world, among the processing objects present in the observation area, based on the determination result of the determination unit 12 and the attributes of each processing object.

[0152] Alternatively, the processing unit 13 may count, for each virtual world, the number of processing targets with a predetermined attribute that exist within the observation area and recognize each virtual world, based on the determination result of the determination unit 12 and the attributes of each processing target. Alternatively, the processing unit 13 may count, for each virtual world, the number of processing targets with a predetermined attribute that exist within the observation area and are not aware of each virtual world, based on the determination result of the determination unit 12 and the attributes of each processing target. Alternatively, the processing unit 13 may calculate, for each virtual world, the proportion of processing targets with a predetermined attribute that exist within the observation area and recognize each virtual world, based on the determination result of the determination unit 12 and the attributes of each processing target. Alternatively, the processing unit 13 may calculate, for each virtual world, the proportion of processing targets with a predetermined attribute that do not recognize each virtual world, based on the determination result of the determination unit 12 and the attributes of each processing target.

[0153] Next, a process of changing the information providing method based on at least one of the determination result of the determination unit 12 and the statistical result of the determination result will be described.

[0154] The processing unit 13 can determine the method of providing information based on the above-mentioned count value and ratio, and can provide information to the processing target present in the observation area using the determined method of providing information.

[0155] For example, if a processing target that recognizes virtual content or a virtual world is present in the observation area, the processing unit 13 may decide to provide information via a virtual content output device.

[0156] Additionally, the processing unit 13 may decide to provide information via a virtual content output device when there are a predetermined number or more processing targets in the observation area that recognize virtual content or a virtual world.

[0157] Additionally, the processing unit 13 may decide to provide information via a virtual content output device when the proportion of processing objects in the observation area that recognize virtual content or a virtual world is equal to or greater than a predetermined reference value.

[0158] In addition, if a processing subject that is not aware of virtual content or the virtual world is present in the observation area, the processing unit 13 may decide to provide information via a speaker or display device (display or projection device) installed in the observation area.

[0159] In addition, the processing unit 13 may decide to provide information via speakers or display devices (displays or projection devices) installed in the observation area if there are a predetermined number or more processing objects in the observation area that are not aware of virtual content or the virtual world.

[0160] In addition, the processing unit 13 may decide to provide information via speakers or display devices (displays or projection devices) installed in the observation area if the proportion of processing objects in the observation area that do not recognize the virtual content or virtual world is equal to or greater than a predetermined reference value.

[0161] The content of the information provided in these cases varies. For example, the information provided in these cases may be emergency information related to earthquakes, weather, etc. In addition, the information provided in these cases may be advertising or notice information related to the observation area. Note that the content of the information exemplified here is merely an example and is not limited to these.

[0162] Next, a process of changing the content of information to be provided based on at least one of the determination result of the determination unit 12 and the statistical result of the determination result will be described.

[0163] The processing unit 13 can determine the content of the information to be provided based on the above-mentioned count value and ratio, and can provide the determined information content to the processing target present in the observation area.

[0164] For example, a plurality of groups of processing targets are generated in advance based on the attributes of the processing targets described above. In one example, the first group is "females in their teens," the second group is "females in their twenties or thirties," the third group is "females in their forties or fifties," and the fourth group is "others." The processing unit 13 may then identify the group (attribute) with the largest count value or percentage. The processing unit 13 may then determine to provide information for the identified group (attribute). The processing unit 13 can output the information determined in this manner via a virtual content output device.

[0165] Next, a process of changing the output destination of information based on at least one of the determination result of the determination unit 12 and the statistical result of the determination result will be described.

[0166] For example, the processing unit 13 can determine, as the output destination of information, a virtual world in which a recognized processing target exists among the plurality of virtual worlds. Additionally, for example, the processing unit 13 can determine, as the output destination of information, a virtual world in which a reference number or more of processing targets recognize among the plurality of virtual worlds.

[0167] Additionally, the processing unit 13 may estimate the attributes (age, gender, etc.) of the processing target determined to recognize the virtual content by image analysis or the like, and analyze (calculate statistical results) what kind of attributes the processing target has and what kind of virtual content or virtual world is using. The processing unit 13 may then provide the analysis results to virtual world operators and advertising companies for marketing purposes.

[0168] Next, an example of the processing flow of the information processing device 10 will be described using the flowchart in Fig. 11. Note that the purpose here is to explain the processing flow. Details of each process have been described above, so explanations here will be omitted as appropriate.

[0169] In S50, the information processing device 10 causes the virtual content output device to output the virtual content by superimposing it on the real world.

[0170] In S51, the information processing device 10 determines whether the processing target in the real world recognizes the virtual content based on the reaction of the processing target to the output of the virtual content. Note that the information processing device 10 may also determine whether the processing target recognizes a virtual world. Alternatively, the information processing device 10 may determine which of multiple virtual worlds the processing target recognizes.

[0171] In S52, the information processing device 10 executes a predetermined process based on the determination result of S51. For example, the information processing device 10 may provide information to a processing target present in the observation area as the predetermined process. Then, based on the determination result of S51, the information processing device 10 may change at least one of the method of providing information, the content of the information to be provided, and the output destination of the information.

[0172] The information processing device 10 may perform statistical processing on the determination result of S51 to generate the above-mentioned "statistical result of the determination result." Then, the information processing device 10 may change at least one of the method of providing information, the content of the information to be provided, and the output destination of the information, based on at least one of the determination result of S51 and the statistical result of the determination result.

[0173] Other configurations of the information processing device 10 of the sixth embodiment can be similar to the configurations of the information processing device 10 of the first to fifth embodiments.

[0174] The information processing device 10 of the sixth embodiment can achieve the same effects as the information processing devices 10 of the first to fifth embodiments. Furthermore, the information processing device 10 can provide information to a processing target present in an observation area in an appropriate manner determined based on the determination result by the determination unit 12. Such an information processing device 10 can provide information to a processing target in a reliable manner. Furthermore, such an information processing device 10 can provide information that is likely to interest the majority of processing targets present in the observation area.

[0175] <<Modifications>> Below, a description will be given of modified examples that can be applied to the information processing device 10 of the first to sixth embodiments. Even in this modified example, it is possible to achieve the same effects as the information processing device 10 of the first to sixth embodiments.

[0176] In this example, the devices used to output virtual content are limited to those that provide a more immersive experience, such as glasses, contact lenses, and head-mounted displays. By using devices that provide a more immersive experience, the target is more likely to react to the output of virtual content. As a result, it becomes possible to accurately determine whether the target recognizes the virtual content.

[0177] Although this disclosure has been described above with reference to the embodiments, this disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of this disclosure within the scope of this disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0178] In addition, in the flowcharts used in the above description, multiple steps (processes) are described in order. However, the order of the steps performed in each embodiment is not limited to the order described. In each embodiment, the order of the steps shown in the drawings can be changed as long as it does not cause any problems in terms of the content.

[0179] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. 1. An output means for causing a virtual content output device to output virtual content superimposed on the real world; a determination means for determining whether the processing target present in the real world recognizes the virtual content based on a reaction of the processing target to an output of the virtual content; An information processing device having the above. 2. The output means An information processing device as described in 1, which superimposes and displays the virtual content, which is at least one image of an obstacle, content that attracts the attention of the processing target, and guidance information that requests a specified action, on the real world. 3. The output means An information processing device as described in 1 or 2, which superimposes the virtual content, which is at least one of a sound that attracts the attention of the processing target and a sound of guidance information that requests a predetermined action, on the real world and outputs the sound. 4. The determination means stopping in front of the obstacle; an operation of avoiding the obstacle; an action of directing one's gaze to content that attracts the processing target's attention; an action of turning one's face toward content that attracts the processing target's attention; A movement of turning one's body toward content that attracts the target's attention; and the predetermined action, 3. The information processing device according to 2, which detects at least one of the above as the reaction by the processing target. 5. The determination means a reaction to the audio that attracts the attention of the processing subject; and the predetermined action, 4. The information processing device according to claim 3, wherein at least one of the above is detected as the reaction by the processing target. 6. The output means outputting the virtual content to a virtual world that is merged with the real world; The determination means 6. The information processing device according to any one of 1 to 5, which determines whether the processing target is aware of the virtual world based on the result of the determination. 7. The output means outputting the virtual content to each of a plurality of virtual worlds merged with the real world, by changing at least one of an output timing of the virtual content, a position of the virtual content, and a content of the virtual content; The determination means 7. An information processing device according to any one of 1 to 6, which identifies the virtual world recognized by the processing target from among the plurality of virtual worlds based on the result of the determination. 8. The output means An information processing device described in any one of 1 to 7, which determines at least one of the timing at which the virtual content is output from the virtual content output device and the position at which the virtual content is displayed superimposed on the real world based on the environment in the real world. 9. One or more computers: The virtual content output device outputs virtual content superimposed on the real world, An information processing method for determining whether a processing target existing in the real world recognizes the virtual content based on a reaction of the processing target to an output of the virtual content. 10. Computer an output means for causing a virtual content output device to output virtual content superimposed on the real world; a determination means for determining whether a processing target existing in the real world recognizes the virtual content based on a reaction of the processing target to an output of the virtual content; A program that functions as a

[0180] Some or all of Supplements 2 to 8 that are dependent on the information processing device of Supplement 1 described above may also be dependent on the information processing method of Supplement 9 and the program of Supplement 10 in the same dependent relationship as Supplement 1 and Supplements 2 to 8. Furthermore, within the scope of each of the above-mentioned embodiments, some or all of the configurations described as Supplements can be realized in various hardware, software, various recording means for recording software, or systems. [Explanation of symbols]

[0181] 10. Information processing equipment 11 Output section 12 Judgment section 13 Processing section 1A processor 2A Memory 3A input / output I / F 4A peripheral circuit 5A Bus

Claims

1. an output means for causing a virtual content output device to output virtual content by superimposing it on the real world; a determination means for determining whether the processing target present in the real world recognizes the virtual content based on a reaction of the processing target to an output of the virtual content; An information processing device having the above.

2. The output means The information processing device according to claim 1 , wherein the virtual content, which is at least one image of an obstacle, content that attracts the processing target's attention, and guidance information that requests a predetermined action, is superimposed on the real world and displayed.

3. The output means The information processing device according to claim 1 , wherein the virtual content, which is at least one of a sound that attracts the attention of the processing target and a sound of guidance information that requests a predetermined action, is output as a sound superimposed on the real world.

4. The determination means stopping in front of the obstacle; an operation of avoiding the obstacle; an action of directing one's gaze toward content that attracts the processing target's attention; an action of turning one's face toward content that attracts the processing target's attention; A movement of turning one's body toward content that attracts the target's attention; and the predetermined action, The information processing device according to claim 2 , wherein at least one of the following is detected as the reaction by the processing target.

5. The determination means a reaction to the audio that attracts the attention of the processing subject; and the predetermined action, The information processing apparatus according to claim 3 , wherein at least one of the following is detected as the reaction by the processing object:

6. The output means outputting the virtual content to a virtual world that is merged with the real world; The determination means The information processing apparatus according to claim 1 , further comprising: determining whether the processing target is aware of the virtual world based on a result of the determination.

7. The output means outputting the virtual content to each of a plurality of virtual worlds merged with the real world, while changing at least one of an output timing of the virtual content, a position of the virtual content, and a content of the virtual content; The determination means The information processing apparatus according to claim 1 , wherein the virtual world recognized by the processing target is identified from among the plurality of virtual worlds based on the result of the determination.

8. The output means The information processing device according to claim 1 , wherein at least one of a timing for causing the virtual content output device to output the virtual content and a position for displaying the virtual content superimposed on the real world is determined based on the environment in the real world.

9. One or more computers The virtual content output device outputs virtual content superimposed on the real world, An information processing method for determining whether a processing target existing in the real world recognizes the virtual content based on a reaction of the processing target to an output of the virtual content.

10. Computer, an output means for causing a virtual content output device to output virtual content superimposed on the real world; a determination means for determining whether a processing target existing in the real world recognizes the virtual content based on a reaction of the processing target to an output of the virtual content; A program that functions as a

Images