Information processing apparatus, information processing system, display unit, information processing method, and program

JP2023157275A5Active Publication Date: 2024-11-28CANON KK
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2022067075
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2022-04-14
Publication Date
2024-11-28
Estimated Expiration
2042-04-14

AI Technical Summary

Technical Problem

Existing technologies fail to accurately notify a user of the possibility of contact while viewing an XR image on another user's device, as they may not differentiate between the user's and other users' potential dangers.

Method used

An information processing system that acquires images and determines the orientation of both users' display devices, assessing the possibility of contact, and controls the display devices to provide distinct notifications based on the risk level, using different warning items for the user and other viewers.

Benefits of technology

Accurately informs users of potential contact risks, enhancing safety by providing differentiated warnings based on the user's and other viewers' proximity and orientation, thereby reducing the likelihood of physical interactions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

To provide a technique that, when a specific user sees an image according to another user, can notify the specific user of the possibility of contact with a person and objects.SOLUTION: An information processing apparatus has: acquisition means that acquires a first image based on an attitude of a first display unit used by a first user; determination means that determines the possibility of contact of a second user; display control means that controls a second display unit used by the second user to display the first image; and control means that, when the possibility of contact of the second user is determined to be higher than a threshold, controls the second display unit to notify the second user of the possibility of contact of the second user in a form of notification different from a case where the second user is notified of the possibility of contact of the first user.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing system, a display device, an information processing method, and a program.

Background Art

[0002] In recent years, XR (Extended Reality) technology that fuses the virtual world and the real world in real time to create a new space (experience) has been known. As devices for presenting an image using XR technology to a user, there are head-mounted devices (HMDs: Head Mount Displays), tablets, or smartphones.

[0003] In Patent Document 1, a technique of a game device that enables a user who is viewing an XR image (experiencing XR) as part of a game to recognize a dangerous situation is described. First, when the user approaches (or reaches) a specific area, the game device performs a display that prompts the user to stop moving. Then, when the user enters a specific area, the game device performs a process (at least any one of a process of changing the displayed image, a process of changing the progress mode of the game, and a process of ending the game) that makes the user hesitate to view the XR image.

[0004] Patent Document 2 describes a technique of transmitting an XR image viewed by a specific user to the devices of other users and presenting the XR image to the other users. According to this, a plurality of users can view the XR image viewed by a specific user and conduct discussions or give instructions.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Patent Document 3

[0006] However, Patent Document 2 states that when a user is viewing an XR image on another user's display device, there is a possibility (risk) of them coming into contact with a person or object by moving their body. In this case, using the technology of Patent Document 1, a warning indicating the possibility of contact can be displayed on the XR image, making the user hesitant to continue the experience. On the other hand, even when using the technology of Patent Document 1, a user may see an XR image that includes a warning for other users, so there is a possibility that the user themselves may not be able to correctly understand whether or not they are in danger.

[0007] Therefore, the present invention aims to provide a technology that can appropriately notify a specific user of the possibility of contact when that specific user views an image that is appropriate for another user. [Means for solving the problem]

[0008] One aspect of the present invention is, Acquisition means for acquiring a first image based on the orientation of a first display device used by a first user, A determination means for determining the possibility of contact by a second user, Display control means for controlling a second display device used by the second user to display the first image, If it is determined that the likelihood of contact by the second user is higher than a threshold, control means control the second display device to notify the second user of the possibility of contact by the second user in a form of notification different from that used when the first user is notified of the possibility of contact by the second user. This is an information processing device characterized by having [a certain feature].

[0009] One aspect of the present invention is, A display device, A display means for displaying a first image based on the orientation of a first display device used by a first user, Control means to control the display means to notify the second user of the possibility of contact by the second user, in a form of notification different from that used when the first user is notified of the possibility of contact by the second user, if the possibility of contact by the second user is higher than a threshold, This is a device characterized by having [a certain feature].

[0010] One aspect of the present invention is, An acquisition step of acquiring a first image based on the orientation of a first display device used by a first user, A determination step to determine the possibility of contact with a second user, A display control step that controls a second display device used by the second user to display the first image, If it is determined that the likelihood of the second user's contact is higher than a threshold, a control step is made to control the second display device to notify the second user of the possibility of contact by the second user in a form of notification different from that used when the first user's contact is notified to the second user; This is an information processing method characterized by having [a certain feature]. [Effects of the Invention]

[0011] According to the present invention, when a specific user views an image that is appropriate for another user, that specific user can be appropriately notified of the possibility of contact. [Brief explanation of the drawing]

[0012] [Figure 1] This is a diagram showing the configuration of the information processing system according to Embodiment 1. [Figure 2] This is a hardware configuration diagram of the first and second devices according to Embodiment 1. [Figure 3] This is a flowchart of the display control process according to Embodiment 1. [Figure 4] It is a diagram for explaining the display control process according to Embodiment 1. [Figure 5] It is a diagram for explaining the warning according to Embodiment 1. [Figure 6] It is a diagram for explaining the warning according to Embodiment 1. [Figure 7] It is a configuration diagram of the information processing system according to Embodiment 2. [Figure 8] It is a diagram for explaining the display control process according to Embodiment 2. [Figure 9] It is a flowchart of the display control process according to Embodiment 2. [Figure 10] It is a diagram for explaining the warning according to Embodiment 2. [Figure 11] It is a flowchart of the display control process according to Modification 2. [Figure 12] It is a diagram for explaining the warning according to Modification 2.

Mode for Carrying Out the Invention

[0013] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the configurations described in the following embodiments are representative examples, and the scope of the present invention is not necessarily limited to those specific configurations. Also, in the following, the distance between two objects may be the length between the center positions of the two objects, or may be the length of the interval between the two objects. That is, it may be the length between the center positions of the two objects, or it may be the length of the interval between the two objects.

[0014] <Embodiment 1> FIG. 1 is a configuration diagram of an information processing system 1 according to Embodiment 1. The information processing system 1 according to Embodiment 1 includes a first device 100, a second device 200, a display device 120 used by a first user, and a display device 220 used by a second user. Since the display devices 120 and 220 are controlled by the first device 100 and the second device 200, the first device 100 and the second device 200 may be regarded as a control system together.

[0015] The first device 100 and the second device 200 can communicate with each other. The first device 100 and the display device 120 can communicate with each other. The second device 200 and the display device 220 can communicate with each other. Communication between each component may be conducted via wired or wireless methods.

[0016] Here, when multiple users are experiencing XR, the information processing system 1 can choose whether or not to share the display image seen by one user with other users. The user who shares such a display image is referred to as the "multi-viewpoint sharing user." One way to select the "multi-viewpoint sharing user" is to assign an ID or name to each of the multiple users and select the "multi-viewpoint sharing user" in response to an operation by any of the users. Another way to select the "multi-viewpoint sharing user" is to use a real-world image, as described in Patent Document 3. In Embodiment 1, the "multi-viewpoint sharing user" is assumed to be the first user who uses (wears) the display device 120.

[0017] (Regarding the internal configuration of the first device 100) Referring to Figure 1, the internal configuration of the first device 100 will be described. The first device 100 is a control device (information processing device; electronic device) that controls the display device 120. The first device 100 includes an image acquisition unit 101, a measurement unit 102, a data acquisition unit 103, a warning determination unit 104, a data holding unit 106, an image generation unit 107, and a shared image transmission unit 110.

[0018] The image acquisition unit 101 acquires a real-world image from the imaging unit 121 of the display device 120. The real-world image is an image captured by the imaging unit 121 of the area (space) in front of the display device 120. The image acquisition unit 101 outputs the real-world image to the measurement unit 102 and the image generation unit 107.

[0019] The measurement unit 102 performs image processing on the real image to extract characteristic information (feature information) such as points and lines in the real image. Based on the feature information, the measurement unit 102 measures the position and orientation (position and orientation) of the display device 120. Then, the measurement unit 102 determines the position and orientation of the first user from the position and orientation of the display device 120. The measurement unit 102 outputs the information of the first user's position and orientation to the warning determination unit 104 and the image generation unit 107.

[0020] In Embodiment 1, the measurement unit 102 measures the position and orientation of the display device 120 based on the image processing results. However, the measurement unit 102 may also mechanically measure the position and orientation of the display device 120. The measurement unit 102 may measure the position and orientation using infrared light or ultrasound, or it may measure the position and orientation using a magnetic sensor. Furthermore, the measurement unit 102 may measure the position and orientation of the display device 120 using a depth sensor. These sensors may be present in the display device 120, or they may be present in the first device 100 if the display device 120 and the first device 100 are located in similar positions.

[0021] The data acquisition unit 103 acquires information on the position, orientation, and shape of real objects (objects or people placed in real space). Here, real objects are objects (objects other than people) or people that are determined to have the potential to come into contact (collide) with the first user. Furthermore, real objects include users other than the first user (such as a second user). The data acquisition unit 103, for example, The measurement unit 202 of the second device 200 may acquire information on the position and orientation of the second user, or it may acquire information on the position and orientation of the second user from values ​​measured by sensors on the display device 120. The data acquisition unit 103 may also assume that the shape of the second user is, for example, a predetermined shape or a cylindrical shape. The data acquisition unit 103 outputs information on the position, orientation and shape of the real object as real object data to the warning determination unit 104.

[0022] Furthermore, the data acquisition unit 103 may acquire information on the position, orientation, and shape of real objects based on the results of scanning real space in advance using, for example, a known 3D reconstruction technique. Alternatively, the data acquisition unit 103 may acquire multiple real images captured from multiple positions (viewpoints) and acquire information on the position, orientation, and shape of real objects by measuring the depth of the real objects based on the parallax between the multiple real images.

[0023] The user may pre-register information about the position, orientation, and shape of a real object. In this case, the user may set a 3D model (for example, a cube or cylinder model) that includes the space containing the real object. The data acquisition unit 103 may then acquire the position where the 3D model set by the user is placed as the position of the real object, and acquire the shape of the 3D model as the shape of the real object. Alternatively, the user may set the plane (2D space) in which the real object is placed as the position of the real object.

[0024] The warning determination unit 104 determines the possibility (risk) of contact (collision) based on the position and orientation of the first user and real-world object data. The warning determination unit 104 then determines whether or not to warn (notify) that there is a high possibility of contact. The warning determination unit 104 outputs the result of its determination of whether or not to warn about the high possibility of contact (first determination result) to the image generation unit 107. In Embodiment 1, the warning determination unit 104 determines whether or not to warn about the high possibility of contact (the possibility of contact) depending on whether or not the distance between the first user and the real-world object is within the threshold Th1. Here, the warning determination unit 104 can calculate the distance between the first user and the real-world object by, for example, calculating the coordinate difference between the position of the first user and the position of the real-world object indicated by the real-world object data. The warning determination unit 104 may also measure the distance between the first user and the real-world object using the distance sensor of the display device 120. Furthermore, if the second user is a real object, the warning determination unit 104 may receive radio waves emitted by the display device 220 and measure the distance between the second user and the first user according to the strength of the received radio waves. The warning determination unit 104 may also use other determination methods if it can detect the possibility of the user coming into contact with a real object.

[0025] For example, as described in Patent Document 3, the warning determination unit 104 estimates the area where the first user is located based on the position and orientation of the first user. The warning determination unit 104 may then determine whether or not to issue a warning indicating a high probability of contact, depending on whether the distance between the area where the first user is located and the real object is within a threshold Th1.

[0026] The warning determination unit 104 may further determine whether to warn of a high probability of contact, not only based on the distance between the first user and the real object, but also depending on whether the first user is looking at the display device 120. In other words, the warning determination unit 104 may determine not to warn of a high probability of contact if the first user is not looking at the display device 120. In this case, the image generation unit 107 controls the display device 120 so as not to warn (notify) of a high probability of contact by the first user (by notifying the user of the high probability of contact). If the display device 120 is a handheld device such as a tablet, there is a high need to warn of a high probability of contact if the first user is immersed in the XR experience, as there is a high probability that the first user is not checking their surroundings. On the other hand, if the first user is not immersed in the XR experience, there is a low need to warn of a high probability of contact because there is a high probability that the first user is checking their surroundings. In other words, if the first user is not looking at the display device 120 In this case, there is little need to warn of the high probability of contact. Furthermore, whether or not the first user is looking at the display device 120 can be determined by the internal camera of the display device 120 acquiring an image and performing gaze detection.

[0027] In other words, the warning determination unit 104 may determine that there is a high probability of contact if the distance between the first user and the real object is within the threshold Th1 and the first user is looking at the display device 120. On the other hand, in other cases, the warning determination unit 104 may determine that there is no high probability of contact and therefore no warning should be issued.

[0028] The data storage unit 106 stores data from the virtual space. The data storage unit 106 outputs the data from the virtual space to the image generation unit 107. The data from the virtual space includes data related to virtual objects that make up the virtual space (information about the shape and position / orientation of virtual objects), or data related to light sources that emit light in the virtual space.

[0029] The image generation unit 107 generates an XR image (mixed reality image) based on the position and orientation of the first user (or display device 120), virtual space data, and a real-world image. Specifically, the image generation unit 107 places virtual objects represented by the virtual space data relative to the real-world space represented by the real-world image, based on the orientation (or position and orientation) of the display device 120. As a result, the image generation unit 107 generates an XR image that appears as if virtual objects are placed in real space. Furthermore, if the warning determination unit 104 determines that there is a high probability of contact and warrants a warning (according to the first determination result), the image generation unit 107 superimposes a first warning item (warning mark) onto the XR image to generate a display image. If the image generation unit 107 determines that there is no high probability of contact and warrants a warning, it treats the XR image as is and uses it as the display image. The image generation unit 107 outputs the display image to the display unit 122 (display device 120). As a result, the image generation unit 107 controls the display on the display unit 122. The image generation unit 107 further outputs the displayed image to the shared image transmission unit 110.

[0030] The shared image transmission unit 110 transmits the displayed image to the second device 200 as an "other-viewpoint image". At this time, the shared image transmission unit 110 may also transmit to the second device 200 an image in which the image quality or frame rate of the displayed image has been changed, or an image which is a cropped portion of the displayed image, as an "other-viewpoint image".

[0031] (Regarding the internal configuration of the second device 200) Referring to Figure 1, the internal configuration of the second device 200 will be described. The second device 200 is a control device (electronic device; information processing device) that controls the display device 220. The second device 200 includes an image acquisition unit 201, a measurement unit 202, a data acquisition unit 203, a warning determination unit 204, a shared image acquisition unit 205, and a display control unit 206.

[0032] The image acquisition unit 201 acquires a real-world image captured by the imaging unit 221 of the real space (the space in front of the display device 220). The image acquisition unit 201 outputs the acquired real-world image to the measurement unit 202.

[0033] The measurement unit 202 measures the position and orientation of the display device 220, similar to the measurement unit 102. Then, the measurement unit 202 determines the position and orientation of the second user from the position and orientation of the display device 220. The measurement unit 202 outputs the information of the second user's position and orientation to the warning determination unit 204.

[0034] The data acquisition unit 203 acquires information on the position, orientation, and shape of real objects, similar to the data acquisition unit 103. Here, real objects include users other than the second user (the first user). The data acquisition unit 203 outputs the information on the position, orientation, and shape of real objects as real object data to the warning determination unit 204.

[0035] The warning determination unit 204, like the warning determination unit 104, determines the possibility of contact (collision) with the second user based on the second user's position and orientation and real-world object data. The warning determination unit 204, like the warning determination unit 104, determines the possibility of contact with the second user depending on the distance between the second user and the real-world object, or whether the second user is looking at the display device 120. The warning determination unit 204 then determines whether or not to warn (notify) that there is a high possibility of contact (collision) with the second user. The warning determination unit 204 outputs the result of its determination of whether or not to warn about the high possibility of contact (second determination result) to the display control unit 206.

[0036] The shared image acquisition unit 205 requests the first device 100, which is associated with the first user who is the "multi-viewpoint sharing user," to transmit the "multi-viewpoint image" (XR image). Once the shared image acquisition unit 205 acquires the "multi-viewpoint image," it outputs the acquired "multi-viewpoint image" to the display control unit 206.

[0037] The display control unit 206 is a control unit that controls the display of the display unit 222. Based on the second judgment result from the warning judgment unit 204 and the "multi-viewpoint image", the display control unit 206 controls the display image to be displayed on the display unit 222. The control of the display image is achieved by executing the process according to the flowchart in Figure 3B, which will be described later. The display control unit 206 outputs the display image to the display unit 222 (display device 220).

[0038] (Regarding the internal configuration of the display device) Referring to Figure 1, the internal configuration of the display device 120 will be described. Since the display device 120 and the display device 220 have similar configurations, the description of the internal configuration of the display device 220 will be omitted. The display device 120 includes an imaging unit 121 and a display unit 122.

[0039] The imaging unit 121 acquires an image by imaging the real space and outputs the image as a real image to the first device 100. The display unit 122 displays the display image acquired from the first device 100.

[0040] Although the display devices 120 and 220 in Embodiment 1 are described as HMDs (Head-Mounted Displays), they are not limited to these. The display device may be any device (for example, a smartphone, digital camera, or tablet terminal) that includes an imaging unit and a display unit. Furthermore, the HMD will be described as a video see-through type HMD (an HMD that displays an XR image generated based on an image captured by the imaging unit on the display unit). However, the display devices 120 and 220 may also be optical see-through type HMDs (HMDs that superimpose an image of virtual space onto a display medium that allows observation through real space).

[0041] Furthermore, the display device 120 may include the first device 100. The display device 220 may include the second device 200. In addition, the display device 120 may include both the first device 100 and the second device 200. The display device 220 may include both the first device 100 and the second device 200.

[0042] Figure 2 is a hardware configuration diagram of the first device 100 and the second device 200 to realize the configurations shown in Figure 1. The CPU 11 comprehensively controls each device (each hardware configuration) connected via the bus 10. The CPU 11 reads and executes processing step information and programs stored in read-only memory (ROM 12). ROM 12 stores information such as the operating system (OS), programs related to Embodiment 1, and device drivers. The CPU 11 expands (temporarily stores) the information stored in ROM 12 into random access memory (RAM 13) and executes it. This is possible. In addition, the input interface 14 (input I / F) acquires input signals from an external device in a format that can be processed by the first device 100 and the second device 200. In addition, the output interface 15 (output I / F) outputs output signals to an external device in a format that can be processed by the external device.

[0043] (Regarding display control processing) Referring to the flowcharts in Figures 3A and 3B, the processes for controlling the displays of the first device 100 and the second device 200 (display control processes) will be explained. Figure 3A shows the processes performed by the first device 100. Figure 3B shows the processes performed by the second device 200.

[0044] In describing the display control process, we assume, as shown in Figure 4, that the first user 402 is wearing the display device 120 on their head, and the second user 404 is wearing the display device 220 on their head. The first user 402 is viewing the XR image 406 displayed on the display device 120. Here, the XR image 406 is an image of a virtual space that includes real objects 400 and virtual objects 401. The second user 404 is viewing the XR image 407 displayed on the display device 220. The content of the XR image 407 is the same as that of the XR image 406, except for warning items (display items). Note that the XR image 407 may be an image with different image quality or frame rate than the XR image 406, or it may be an image of a part of the range of the XR image 406.

[0045] First, the process performed by the first device 100 will be explained by referring to the flowchart in Figure 3A. The process in the flowchart in Figure 3A starts each time the image acquisition unit 101 acquires a real image corresponding to one frame.

[0046] In step S300, the measurement unit 102 acquires a real image from the image acquisition unit 101. Then, based on the real image, the measurement unit 102 measures (estimates) the position and orientation of the first user 402.

[0047] In step S301, the image generation unit 107 generates an XR image based on the orientation of the display device 120. Specifically, the image generation unit 107 generates an XR image in which virtual objects are placed in the real space shown by the real image, at positions based on the orientation (positional orientation) of the display device 120.

[0048] In step S302, the warning determination unit 104 determines whether the probability of contact between the first user 402 and the real object 400 is higher than the threshold Th. Specifically, the warning determination unit 104 determines whether the distance between the first user 402 and the real object 400 is within the threshold Th1 (performing the first warning determination). If it is determined that the distance between the first user 402 and the real object 400 is within the threshold Th1, it is determined that the probability of contact between the first user 402 and the real object 400 is higher than the threshold Th, and the process proceeds to step S303. If it is determined that the distance between the first user 402 and the real object 400 is greater than the threshold Th1, the process proceeds to step S304.

[0049] In step S303, the image generation unit 107 generates a display image by superimposing a first warning item on the XR image (an XR image based on the orientation of the display device 120) to warn that there is a high probability of contact by the first user 402. Figure 5A shows an example of an XR image after processing in step S303, in which a triangular first warning item 501 is superimposed on the XR image 500.

[0050] In step S304, the shared image transmission unit 110 transmits the display image generated by the image generation unit 107 to the second device 200 as a "multi-viewpoint image". At this time, the first user 4 If the distance between 02 and the real object 400 is within the threshold Th1 (YES in step S302), the displayed image is an XR image based on the orientation of the display device 120 with the first warning item superimposed. On the other hand, if the distance between the first user 402 and the real object 400 is greater than the threshold Th1 (NO in step S302), the displayed image is an XR image based on the orientation of the display device 120 (an XR image without the first warning item superimposed).

[0051] In step S305, the image generation unit 107 transmits a display image to the display device 120 (first display device). As a result, the image generation unit 107 controls the display unit 122 to display the display image.

[0052] Next, the process performed by the second device 200 will be explained with reference to the flowchart in Figure 3B. The process in the flowchart of Figure 3B starts each time the image acquisition unit 201 acquires a real image corresponding to one frame.

[0053] In step S310, the shared image acquisition unit 205 determines whether or not it has acquired a "multi-viewpoint image" from the first device 100. If it is determined that a "multi-viewpoint image" has been acquired, the process proceeds to step S311. If it is determined that a "multi-viewpoint image" has not been acquired, the process in step S310 is repeated. If it is determined that a "multi-viewpoint image" has not been acquired, the shared image acquisition unit 205 may further request the shared image transmission unit 110 to transmit the "multi-viewpoint image".

[0054] In step S311, the measurement unit 202 acquires a real image from the image acquisition unit 201. Based on the real image, the measurement unit 202 measures the position and orientation of the second user 404.

[0055] In step S312, the warning determination unit 204 determines whether the probability of contact between the second user 404 and the real object 400 is higher than the threshold Th. Specifically, the warning determination unit 204 determines whether the distance between the second user 404 and the real object 400 is within the threshold Th2 (a predetermined distance) (performing a second warning determination). If it is determined that the distance between the second user 404 and the real object 400 is within the threshold Th2, it is determined that the probability of contact between the second user 404 and the real object 400 is higher than the threshold Th, and the process proceeds to step S313. If it is determined that the distance between the second user 404 and the real object 400 is greater than the threshold Th2, it is determined that the probability of contact between the second user 404 and the real object 400 is less than or equal to the threshold Th, and the process proceeds to step S314.

[0056] Furthermore, in step S312, the warning determination unit 204 only needs to be able to determine whether the probability of contact between the second user 404 and the real object 400 is higher than the threshold Th. For this reason, the above-mentioned "distance between the second user 404 and the real object 400" may be reinterpreted as "the value obtained by dividing the distance between the second user 404 and the real object 400 by the speed at which the second user 404 and the real object 400 approach each other (approach speed)." This is because the faster the approach speed, the higher the probability of contact. In other words, by reinterpreting it in this way, the warning determination unit 204 can determine the probability of contact with even greater accuracy. This reinterpretation can also be applied to determining the probability of contact between the first user 402 and the real object 400 (step S302).

[0057] In step S313, the display control unit 206 superimposes a second warning item onto the "other-viewpoint image" acquired by the shared image acquisition unit 205 to warn that there is a high probability of contact by a second user. Here, the second warning item is a different warning item from the first warning item. The second warning item is, for example, a warning item that is more conspicuous than the first warning item.

[0058] Furthermore, if the probability of contact by the first user 402 is higher than a predetermined value, the shared image acquisition unit The "multi-viewpoint image" acquired by 205 is an XR image containing the first warning item 501, as shown in Figure 5A. In such cases, the display control unit 206 superimposes the second warning item 502 onto the XR image 500 so as to cover the entire first warning item 501, as shown in Figure 5B. This allows the second user 404 to recognize only the second warning item 502 in the XR image displayed on the display device 220.

[0059] In step S314, the display control unit 206 transmits the "other-viewpoint image" to the display device 220 (second display device) as the display image. The display control unit 206 then controls the display unit 222 to display the display image.

[0060] According to Embodiment 1, the information processing system 1 displays a first warning item for warning a "multi-viewpoint sharing user" and a second warning item for warning that specific user differently on a display device viewed by a specific user. This allows a user to understand who is being warned when viewing an XR image based on a display device held by another user.

[0061] In Embodiment 1, the information processing system 1 changed the icons for the first warning item and the second warning item, as shown in Figures 5A and 5B. However, any form of warning item (display item) can be used as long as it allows the user to understand whether they may be contacted themselves or a "multi-perspective sharing user". For example, the first warning item and the second warning item may be message windows containing different wording. One of the first and second warning items may be an icon and the other a message window.

[0062] Furthermore, the display control unit 206 may differentiate the display form (notification form) of the warning item depending on whether the probability of the second user coming into contact with an object (an object other than a person) is higher than the threshold Th, or whether the probability of the second user coming into contact with a person is higher than the threshold Th. Specifically, contact between people may have a greater impact on the damage caused by contact. For this reason, for example, the display control unit 206 may make the second warning item larger and brighter in color when the probability of the second user coming into contact with a person is higher than the threshold Th than when the probability of the second user coming into contact with an object is higher than the threshold Th. Note that the display form of the warning item may also differ depending on whether the probability of the second user coming into contact with a specific user is higher than the threshold Th, or when the probability of the second user coming into contact with someone other than the specific user is higher than the threshold Th. Here, the specific user may be the first user, or a user using a display device for viewing XR images (a display device capable of acquiring "multi-viewpoint images" from the first device).

[0063] (Variation 1) In Embodiment 1, even if the "other-viewpoint image" transmitted from the shared image transmission unit 110 contains the first warning item, the display control unit 206 superimposes a second warning item, different from the first warning item, onto the "other-viewpoint image." This allows the second user to understand whether there is a possibility of contact with the first user, who is the "other-viewpoint sharing user," or with themselves (the second user). However, if the "other-viewpoint image" transmitted from the shared image transmission unit 110 contains the first warning item, the display control unit 206 may overwrite the first warning item to indicate that the possibility of contact lies with the "other-viewpoint sharing user." In this case, the display control unit 206 may change the display format of the first warning item, or superimpose another warning item onto the first warning item.

[0064] For example, the shared image transmission unit 110 transmits the XR image and the first judgment result linked together. Then, the shared image acquisition unit 205 acquires the XR image and the first judgment result. Based on the first determination result, the display control unit 206 determines whether or not to warn that there is a high probability of contact. If the display control unit 206 determines that there is a high probability of contact, it displays a warning item 600 to the "other-view sharing user" in text, as shown in Figure 6, indicating that there is a high probability of contact.

[0065] As a result, when users view XR images based on displays held by other users, they can more accurately determine whether they might come into contact with themselves or with the "other-view sharing user."

[0066] <Embodiment 2> Embodiment 1 describes an information processing system 1 that warns of a high probability of contact when a second user is viewing an XR image based on a display device held by the first user (having an XR experience with the XR image). Embodiment 2 describes an information processing system 2 that can switch whether the second user views an XR image based on a display device held by the second user or an XR image based on a display device held by the first user.

[0067] Figure 7 is a diagram showing the configuration of the information processing system 2 according to Embodiment 2. In Figure 7, only the configurations that differ from Embodiment 1 (Figure 1) will be described in detail, and descriptions of other configurations will be omitted.

[0068] The information processing system 2 includes a display device 120, a first device 100, and a display device 700. The first device 100 and the display device 700 are connected to each other so that they can communicate with one another.

[0069] Referring to Figure 7, the internal configuration of the display device 700 will be described. The display device 700 has an imaging unit 221, a display unit 222, and a second device 710. In Embodiment 2, the display device 700 is assumed to be a tablet terminal. The display device 700 may be any device that has an imaging unit 221, a display unit 222, and an information processing unit (second device 710). The display device 700 may be, for example, a mobile terminal such as a smartphone, or a standalone HMD.

[0070] Figures 8A and 8B illustrate examples of the second user 404 viewing an XR image based on the display device 700 (a display device held by the second user 404) and the second user 404 viewing an XR image based on the display device 120 held by the first user 402. In Figures 8A and 8B, the same components as in Figure 4 are given the same numbers, and their descriptions are omitted.

[0071] Figure 8A shows an example where a second user 404 is viewing an XR image based on the display device 700. The XR image 802 displayed on the display device 700 is an XR image in which an image of a virtual object 401 is superimposed on a real image acquired by the imaging unit 221 of the display device 700.

[0072] Figure 8B shows an example where a second user 404 is viewing an XR image based on a display device 120 held by the first user 402. The XR image 803 displayed on the display device 700 is an XR image in which an image of a virtual object 401 is superimposed on a real image acquired by the imaging unit 121 of the display device 120.

[0073] The data storage unit 711 stores data in the virtual space, similar to the data storage unit 106. The data storage unit 711 outputs the data in the virtual space to the image generation unit 712.

[0074] The image generation unit 712, like the image generation unit 107, generates an XR image based on the real image captured by the imaging unit 221 in real space. Below, the XR image generated by the image generation unit 712 The image is called the "self-viewpoint image." The image generation unit 712 outputs the "self-viewpoint image" to the display control unit 714.

[0075] The mode selection unit 713 selects the display mode of the display device 700. Specifically, the mode selection unit 713 selects either a display mode that displays the "own-viewpoint image" generated by the image generation unit 712, or a display mode that displays the "other-viewpoint image" (XR image) generated by the first device 100. Hereafter, the display mode that displays the "own-viewpoint image" generated by the image generation unit 712 will be referred to as the "own-viewpoint mode." The display mode that displays the "other-viewpoint image" generated by the first device 100 will be referred to as the "other-viewpoint mode." The mode selection unit 713 outputs information about the selected display mode to the display control unit 714.

[0076] If the mode selection unit 713 selects the multi-viewpoint mode, it then selects the "multi-viewpoint sharing user". In this case, the only users that can be selected as the "multi-viewpoint sharing user" are those associated with the information processing device (display device) in which the self-viewpoint mode is selected. In this case, the mode selection unit 713 outputs the information of the "multi-viewpoint sharing user" to the display control unit 714.

[0077] The display control unit 714 controls the display image according to the display mode information obtained from the mode selection unit 713. The display control unit 714 requests the image generation unit 712 or the shared image acquisition unit 205 to acquire either a "self-viewpoint image" or a "other-viewpoint image" according to the display mode of the display device 700. Then, the display control unit 714 generates a display image based on the acquired image. The generation of the display image is achieved by executing the process according to the flowchart described later (see Figure 9B). The display control unit 714 outputs the display image to the display unit 222.

[0078] Referring to the flowcharts in Figures 9A and 9B, the processes by which the first device 100 and the second device 710 control the display (display control processes) will be explained. Figure 9A shows the processes performed by the first device 100. Figure 9B shows the processes performed by the second device 710. In the following, the same steps as in Figures 3A and 3B will not be explained.

[0079] First, the process performed by the first device 100 will be explained by referring to the flowchart in Figure 9A. The process in the flowchart of Figure 9A starts each time the image acquisition unit 101 acquires a real image corresponding to one frame.

[0080] In step S900, the shared image transmission unit 110 determines the display mode of the display device 700 based on the notification of the display mode from the shared image acquisition unit 205. If it is determined that the display mode of the display device 700 is the user's own viewpoint mode, the process proceeds to step S305. If it is determined that the display mode of the display device 700 is the other-viewpoint mode, the process proceeds to step S304.

[0081] Next, the process performed by the second device 710 will be explained with reference to the flowchart in Figure 9B. The process in the flowchart of Figure 9B starts each time the image acquisition unit 201 acquires a real image corresponding to one frame.

[0082] In step S910, the shared image acquisition unit 205 notifies the first device 100 of the display mode (display mode of the display device 700) selected by the mode selection unit 713. If the display mode of the display device 700 is the multi-viewpoint mode, the shared image acquisition unit 205 can be said to be requesting the shared image transmission unit 110 to transmit the "multi-viewpoint image" by notifying the display mode.

[0083] In step S911, the display control unit 714 determines the display mode selected by the mode selection unit 713. If this is the case, proceed to step S912. If it is determined that the mode selection unit 713 has selected the multi-viewpoint mode, proceed to step S310.

[0084] In step S912, the display control unit 714 requests the image generation unit 712 to generate a "self-viewpoint image". In response to the request from the display control unit 714, the image generation unit 712 generates a "self-viewpoint image" (XR image) based on the real-world image acquired by the imaging unit 221.

[0085] In step S913, the measurement unit 202 measures the position and orientation of the second user, similar to step S311.

[0086] In step S914, the warning determination unit 204 determines, similar to step S312, whether the distance between the second user and the real object is within the threshold Th2. If it is determined that the distance between the second user and the real object is within the threshold Th2 (i.e., if it is determined that the probability of contact between the second user and the real object is higher than the threshold Th), the process proceeds to step S915. If it is determined that the distance between the second user and the real object is greater than the threshold Th2, the process proceeds to step S916.

[0087] In step S915, the display control unit 714 superimposes a third warning item onto the "own-viewpoint image" to warn of the high probability of contact by a second user. For example, as shown in Figure 10, the third warning item 1001 is superimposed on the XR image 1000. The third warning item 1001 is a display item of a different form from the first warning item 501 and the second warning item 502, as shown in Figures 5A and 5B.

[0088] In step S916, the display control unit 714 transmits a display image to the display unit 222. Here, if the display mode of the display device 700 is the user's own viewpoint mode, the display image is the "user's own viewpoint image," and if the display mode of the display device 700 is the user's other viewpoint mode, the display image is the "user's other viewpoint image."

[0089] In Embodiment 2, the first warning item, the second warning item, and the third warning item are each different, predetermined forms of warning items. However, for example, the form of the warning items may be arbitrarily determined by the user. For example, the setting unit (not shown) of the first device 100 or the setting unit (not shown) of the second device 710 may set the form of each warning item according to the user's operation.

[0090] The setting unit calls a GUI for setting information about the warning item (the GUI to be displayed as a warning item, and the placement of the warning item). The setting unit then sets the information about the warning item according to the user's operation. The image generation unit 107 superimposes the first warning item onto the XR image based on the GUI and placement information of the first warning item. The setting unit of the second device 710 outputs the GUI and placement information of the second and third warning items to the display control unit 714. If the display mode is the user's viewpoint mode, the display control unit 714 superimposes the third warning item onto the "user's viewpoint image" based on the acquired information. If the display mode is the user's viewpoint mode, the display control unit 714 superimposes the second warning item onto the "user's viewpoint image" based on the acquired information.

[0091] In this way, by making the forms of the first, second, and third warning items different, the user can accurately understand whether they might come into contact with themselves or the "other-view sharing user," as well as the display mode of the display device 700.

[0092] (Modification 2) In variation 2, the information processing system 2 responds to the possibility of the second user coming into contact with a real object. The image displayed on the display unit 222 is switched. When the display mode is the multi-viewpoint mode, the second user cannot see their surroundings in real space, increasing the likelihood of contact with real objects. For this reason, the information processing system 2 may use the "self-viewpoint image" (an XR image based on a real-world image acquired by the imaging unit 221) as the display image if the distance between the second user and the real object is within a threshold (within a predetermined distance).

[0093] Referring to the flowchart shown in Figure 11, the display control process performed by the second device 710 according to the modified example 2 will be described. In the following, the same reference numbers are used for the same steps as in Figure 3B or Figure 9B, and their explanations will be omitted.

[0094] The process in step S1100 begins when it is determined in step S911 that the display mode of the display device 700 is the multi-viewpoint mode. In step S1100, the measurement unit 202 measures the position and orientation of the second user, similar to steps S311 and S913.

[0095] In step S1101, the warning determination unit 204 determines whether the distance between the second user and the real object is within the threshold Th2. If it is determined that the distance between the second user and the real object is within the threshold Th2 (i.e., if it is determined that the probability of contact between the second user and the real object is higher than the threshold Th), the process proceeds to step S1103. If it is determined that the distance between the second user and the real object is greater than the threshold Th2, the process proceeds to step S1102.

[0096] In step S1102, the display control unit 714 determines whether the shared image acquisition unit 205 has acquired a "multi-viewpoint image". If it is determined that a "multi-viewpoint image" has been acquired, the process proceeds to step S313. If it is determined that a "multi-viewpoint image" has not been acquired, the process in step S1102 is repeated.

[0097] In step S1103, the display control unit 714 requests the image generation unit 712 to generate a "self-viewpoint image," similar to step S912. In response to the request from the display control unit 714, the image generation unit 712 generates a "self-viewpoint image" based on the real-world image acquired by the imaging unit 221.

[0098] As described above, when the second user is viewing an XR image based on the display device held by the first user, the information processing system 2 switches to the XR image based on the display device held by the second user if there is a possibility of contact between the second user and a real object. This allows the second user to understand their surroundings and thus determine whether or not there is a possibility of contact with a real object.

[0099] In the modified example 2, the information processing system 2 used only the "self-viewpoint image" as the display image when the display mode of the display device 700 was in multi-viewpoint mode and the distance between the second user and the real object was within the threshold Th2, but it may also use the "other-viewpoint image". That is, when the display mode is in multi-viewpoint mode and the distance between the second user and the real object is within the threshold Th2, the information processing system 2 may superimpose the "other-viewpoint image" and a warning item onto a part of the "self-viewpoint image". For example, as shown in Figure 12, the display control unit 714 may superimpose the warning item 1201 and the "other-viewpoint image" XR image 1202 onto a part of the "self-viewpoint image" XR image 1200. This can be achieved by the display control unit 714 acquiring the "other-viewpoint image" from the shared image acquisition unit 205 and further performing the process of superimposing the "other-viewpoint image" onto the display image.

[0100] (Variation 3) In the modified example 2, the information processing system 2 has the display mode of the display device 700 set to multi-view mode. If the conditions are met and the distance between the second user and the real object is within the threshold Th2, the display image on the display device 700 is switched from "other-viewpoint image" to "own-viewpoint image". However, the information processing system 2 may change the display mode in such cases.

[0101] The mode selection unit 713 switches the display mode to the user's own viewpoint mode if the display mode of the display device 700 is the other viewpoint mode and the distance between the second user and the real object is within the threshold Th2.

[0102] As described above, if there is a high probability of contact with the second user while the second user is viewing the "other-viewpoint image," the image displayed on the display device 700 can be switched to the "own-viewpoint image." This allows the second user to understand whether or not there is a high probability of contact with themselves.

[0103] Furthermore, in each of the embodiments and modifications described above, the information processing system warned of a high probability (risk) of contact by displaying a warning item. Here, in addition to (or instead of) displaying a warning item, the information processing system may also provide warnings by sound or vibration. That is, the display control unit may cause the display device to output sound from its sound output unit, or to vibrate the display device, to warn of a high probability of contact between the user and a real object. The information processing system may also vary the sound and vibration patterns depending on the display mode.

[0104] Furthermore, in each of the embodiments and modifications described above, the XR image is an image in which virtual objects represented by virtual space data are placed in the real space represented by the real image. However, the XR image may also be a VR image, as long as it is an image based on the orientation of the display device. In other words, the XR image may be an image in which virtual objects are placed based on the orientation of the display device, regardless of whether it is a real image.

[0105] Furthermore, in the above, the phrase "If A is greater than or equal to B, proceed to step S1; if A is less than (lower than) B, proceed to step S2" can be rephrased as "If A is greater than (higher than) B, proceed to step S1; if A is less than or equal to B, proceed to step S2." Conversely, the phrase "If A is greater than (higher than) B, proceed to step S1; if A is less than or equal to B, proceed to step S2" can be rephrased as "If A is greater than or equal to B, proceed to step S1; if A is less than (lower than) B, proceed to step S2." Therefore, as long as no contradiction arises, the expression "greater than or equal to A" can be replaced with "A or greater than (higher; longer; more)" or rephrased as "greater than (higher; longer; more)." On the other hand, the expression "less than or equal to A" can be replaced with "A or less than (lower; shorter; fewer)" or rephrased as "less than (lower; shorter; fewer)." And "greater than (higher; longer; more)" is It can be reinterpreted as "A or greater," and "less than A (lower; shorter; fewer)" can be reinterpreted as "A or less."

[0106] Although the present invention has been described in detail above based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms that do not depart from the spirit of the invention are also included in the present invention. Some of the above embodiments may be combined as appropriate.

[0107] Furthermore, each functional unit in each of the above embodiments (each modified example) may or may not be individual hardware. The functions of two or more functional units may be implemented by common hardware. Each of the multiple functions of a single functional unit may be implemented by individual hardware. Two or more functions of a single functional unit may be implemented by common hardware. In addition, each functional unit may or may not be implemented by hardware such as ASIC, FPGA, or DSP. For example, the device may include a processor and a control unit. The device may have a memory (storage medium) on which a program is stored. Furthermore, the functions of at least some of the functional units of the device may be realized by a processor reading a control program from the memory and executing it.

[0108] (Other embodiments) The present invention can also be realized by supplying a program that implements one or more of the functions of the above embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0109] The above-disclosed embodiments include the following configurations, methods, and programs. (Composition 1) Acquisition means for acquiring a first image based on the orientation of a first display device used by a first user, A determination means for determining the possibility of contact by a second user, Display control means for controlling a second display device used by the second user to display the first image, If it is determined that the likelihood of contact by the second user is higher than a threshold, control means control the second display device to notify the second user of the possibility of contact by the second user in a form of notification different from that used when the first user is notified of the possibility of contact by the second user. An information processing device characterized by having the following features. (Configuration 2) The first image includes the first display item if the second user is notified that the first user may have contact with it. If the control means determines that the likelihood of contact by the second user is higher than the threshold, it controls the second display device to display a second display item different from the first display item together with the first image. The information processing device according to configuration 1, characterized by the above. (Composition 3) If the control means determines that the likelihood of contact by the second user is higher than the threshold, and the first image includes the first display item, it controls the second display device to display an image in which the second display item is superimposed on the first image so as to cover the entire first display item. The information processing apparatus according to configuration 2, characterized in that... (Composition 4) The control means controls the second display device to notify the second user of the possibility of contact by emitting sound from the second display device or by vibrating the second display device. The information processing device according to configuration 1, characterized by the above. (Composition 5) The determination means determines the possibility of contact between the object and the second user. An information processing apparatus according to any one of configurations 1 to 4, characterized by the above. (Composition 6) The control means provides different forms of notification to the second user regarding the possibility of contact between the second user and a user using a specific display device, depending on whether the probability of contact between the second user and a user other than the user using the specific display device is higher than the threshold. The information processing apparatus according to configuration 5, characterized by the features described herein. (Composition 7) The determination means determines that 1) if the distance between the object and the second user is shorter than a predetermined distance, the probability of contact by the second user is higher than the threshold, and 2) if the distance between the object and the second user is longer than the predetermined distance, the probability of contact by the second user is lower than the threshold. The information processing apparatus according to configuration 5 or 6, characterized by the above. (Composition 8) The control means controls the second display device so as not to notify the second user of the possibility of contact by the second user if the second user is not looking at the second display device. An information processing device according to any one of configurations 1 to 7, characterized by the above. (Composition 9) The second display device further has a selection means for selecting a display mode, The display control means controls the second display device to display a second image based on the orientation of the second display device when the selection means has selected a specific display mode. An information processing device according to any one of configurations 1 to 8. (Composition 10) The second display device acquires an image by imaging the space in front of the second display device, The information processing device has a generation means for generating a second image based on the captured image. The information processing apparatus according to configuration 9, characterized by the features described therein. (Composition 11) The display control means controls the second display device to switch the displayed image from the first image to the second image if the selection means has not selected the specific display mode and it is determined that the probability of the second user contacting the device is higher than the threshold. The information processing apparatus according to configuration 10, characterized by the above. (Composition 12) If the selection means has not selected the specific display mode and it is determined that the likelihood of the second user contacting the display is higher than the threshold, the display control means controls the second display device to display an image in which the first image is superimposed on a portion of the second image. The information processing apparatus according to configuration 10, characterized by the above. (Composition 13) If the selection means has not selected the specific display mode, and it is determined that the likelihood of contact by the second user is higher than the threshold, the selection means switches the display mode to the specific display mode. The information processing apparatus according to configuration 10, characterized by the above. (Composition 14) If the control means determines that the possibility of contact by the second user is higher than the threshold, it controls the second display device to notify the second user of the possibility of contact by the second user in a different form of notification depending on the display mode selected by the selection means. An information processing apparatus according to any one of the configurations 9 to 12, characterized by the above. (Composition 15) The first display device and the second display device are head-mounted displays. An information processing apparatus according to any one of configurations 1 to 14, characterized by the features described herein. (Composition 16) The information processing apparatus according to any one of configurations 1 to 15, characterized in that the first image is an image with the same content as the image displayed on the first display device. (System 1) An information processing device as described in any one of items 1 to 16, If the probability of contact by the first user is higher than the threshold, the system has a second control means for controlling the first display device to notify the first user of the possibility of contact by the first user. An information processing system characterized by the following: (Second composition 1) A display device, A display means for displaying a first image based on the orientation of a first display device used by a first user, Control means to control the display means to notify the second user of the possibility of contact by the second user, in a form of notification different from that used when the first user is notified of the possibility of contact by the second user, if the possibility of contact by the second user is higher than a threshold, A display device characterized by having the following features. (Method 1) An acquisition step of acquiring a first image based on the orientation of a first display device used by a first user, A determination step to determine the possibility of contact with a second user, A display control step that controls a second display device used by the second user to display the first image, If it is determined that the likelihood of the second user's contact is higher than a threshold, a control step is made to control the second display device to notify the second user of the possibility of contact by the second user in a form of notification different from that used when the first user's contact is notified to the second user; An information processing method characterized by having the following features. (Program 1) A program for causing a computer to function as one of the information processing devices described in any one of the configurations 1 to 16. [Explanation of Symbols]

[0110] 1: Information processing system, 100: First device, 200: Second device, 120: display device, 220: display device, 204: warning determination section, 205: Shared image acquisition unit, 206: Display control unit

Claims

1. an acquisition means for acquiring a first image based on a posture of a first display device used by a first user; A determination means for determining a possibility of a collision of a second user; a control means for controlling a second display device used by the second user to display the first image; having When it is determined that the second user is highly likely to collide, the control means controls the second display device to notify the second user of the possibility of the second user's collision in a manner different from a manner in which the second user is notified of the possibility of the first user's collision.

23. An information processing apparatus comprising:

2. the first image includes a first display item if the second user is notified of a possible collision of the first user; the control means controls the second display device to display a second display item different from the first display item together with the first image when it is determined that the second user has a high possibility of collision.

2. The information processing apparatus according to claim 1,

3. when it is determined that there is a high possibility of a collision of the second user and the first image includes the first display item, the control means controls the second display device to display an image in which the second display item is superimposed on the first image so as to cover the entirety of the first display item.

3. The information processing apparatus according to claim 2.

4. the control means controls the second display device to notify the second user of a possibility of a collision by emitting a sound from the second display device or vibrating the second display device.

2. The information processing apparatus according to claim 1,

5. The determination means determines a possibility of a collision between an object and the second user.

5. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

6. the control means changes the form of a notification for notifying the second user of the possibility of a collision between a case where a collision is likely between a user using a specific display device and the second user and a case where a collision is likely between a user other than the user using the specific display device and the second user.

6. The information processing apparatus according to claim 5,

7. The determination means determines that there is a high possibility of a collision of the second user if a distance between the object and the second user is shorter than a predetermined distance.

6. The information processing apparatus according to claim 5,

8. the control means controls the second display device so as not to notify the second user of a possibility of a collision of the second user when the second user is not looking at the second display device.

5. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

9. a selection means for selecting a display mode of the second display device; the control means controls the second display device to display a second image based on a posture of the second display device when the selection means selects a specific display mode.

5. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

10. the second display device captures an image of a space in front of the second display device, the information processing device includes a generating unit that generates the second image based on the captured image, 10. The information processing apparatus according to claim 9,

11. The control means controls the second display device to switch the image to be displayed from the first image to the second image when the selection means has not selected the specific display mode and it is determined that there is a high possibility of a collision by the second user.

11. The information processing apparatus according to claim 10,

12. When the selection means has not selected the specific display mode and it is determined that there is a high possibility of a collision by the second user, the control means controls the second display device to display an image in which the first image is superimposed on a portion of the second image.

11. The information processing apparatus according to claim 10,

13. the selection means, when the selection means has not selected the specific display mode and it is determined that the possibility of a collision with the second user is high, switches the display mode to the specific display mode.

11. The information processing apparatus according to claim 10,

14. When it is determined that the second user is highly likely to collide, the control means controls the second display device to notify the second user of the possibility of the second user colliding in a manner that differs depending on the display mode selected by the selection means.

10. The information processing apparatus according to claim 9,

15. The first display device and the second display device are head-mounted displays.

5. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

16. 5. The information processing apparatus according to claim 1, wherein the first image is an image having the same content as an image displayed on the first display device.

17. An information processing device according to any one of claims 1 to 4; and a second control means for controlling the first display device to notify the first user of the possibility of a collision of the first user when the possibility of a collision of the first user is high. An information processing system comprising:

18. A display device, comprising: a display means for displaying a first image based on a posture of a first display device used by a first user; a control means for controlling the display means to notify the second user of the possibility of a collision of the second user using the display device in a manner different from that in the case where the second user is notified of the possibility of a collision of the first user; A display device comprising:

19. A display device, comprising: A display means for displaying an image; A selection means for selecting a display mode from a plurality of display modes; having The display device characterized in that the plurality of display modes include a first display mode in which a first image based on a posture of a first display device used by a first user is displayed on the display means, and a second display mode in which a second image based on a posture of the display device used by a second user is displayed on the display means.

20. The display device further comprises a control means for controlling the image displayed on the display means to be switched from the first image to the second image when the selection means selects the first display mode and it is determined that there is a high possibility of a collision with the second user.

20. The display device according to claim 19 .

21. acquiring a first image based on a posture of a first display device used by a first user; determining a collision probability of a second user; controlling a second display device used by the second user to display the first image; When it is determined that the second user is highly likely to collide, controlling the second display device to notify the second user of the possibility of a collision of the second user in a manner different from a manner in which the second user is notified of the possibility of a collision of the first user; 13. An information processing method comprising:

22. A computer is provided with each of the means of the information processing device according to any one of claims 1 to 4. A program to make it function as such.