Information display terminal

Abstract

The information display terminal receives information indicating a first experience area set up for experiencing content executed on other devices located in the same physical space. If it determines that a second experience area selected by the user does not overlap with the first experience area, it compares the size of the required experience area for experiencing the user-selected content with the size of the second experience area. If the size of the second experience area is greater than or equal to the size of the required experience area, the information display terminal sets the second experience area as the area for experiencing the selected content.

Description

Technical Field

[0001] This invention relates to information display terminals. Background Technology

[0002] Users sometimes wear information display terminals such as head-mounted displays (HMDs) to experience content. Here, users can set the area they use when experiencing content. This area (hereinafter sometimes referred to as the "experience area") is a region of real space in which users can be expected to move and / or perform actions, for example, during the experience of content.

[0003] Furthermore, Patent Document 1 describes: "A head-mounted display (HMD) 1 is operated by the gestures of a user 3, and includes a distance image acquisition unit 106 for detecting gesture operations, a location information acquisition unit 103 for acquiring the location information of the HMD 1, and a communication unit 2 for communicating with other HMDs 1'. The control unit 205 sets and displays the effective operating space 600 for the user 3's gesture operations, and exchanges the location information and operating space information of each other with other HMDs 1 through the communication unit 2, adjusting the operating space of its own HMD so that the operating spaces 600 and 600' do not overlap. Alternatively, when the distance image acquisition unit 106 detects a nearby person 4, it adjusts the operating space so that the location of the detected person does not overlap with the operating space 600."

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: International Patent Publication No. 2019 / 038875 Summary of the Invention

[0007] The technical problem that the invention aims to solve

[0008] In HMD and other information display terminals, users not only use gestures for operation, but also, depending on the content being performed, can anticipate various usage patterns, such as operating while seated or moving within the physical space. Because of these usage patterns, the area required for user experience is not fixed. Therefore, in situations where multiple information display terminals exist within a limited physical space, such as a room, the content being performed and other factors must be considered to appropriately define the experience area.

[0009] Furthermore, although Patent Document 1 discloses adjusting the operating space for users to perform gesture operations between other devices, it does not consider adjusting the space according to the content being performed.

[0010] Technical means to solve the problem

[0011] According to the present invention, as an example, an information display terminal is provided. This information display terminal receives information indicating a first experience area set up for experiencing content executed on another device located in the same physical space. If it is determined that a second experience area selected by the user does not overlap with the first experience area, the information display terminal compares the size of the experience requirement area needed to experience the user-selected content with the size of the second experience area. If the size of the second experience area is greater than or equal to the size of the experience requirement area, the information display terminal sets the second experience area as the area for experiencing the selected content.

[0012] Furthermore, according to the present invention, an information processing method executed on an information display terminal and an information processing system having an information display terminal and a server or content distribution device are also provided.

[0013] Invention Effects

[0014] According to the present invention, a technique is provided that allows for the appropriate setting of an experience area for content. Furthermore, technical problems, features, and effects other than those described above will become clear through the following detailed description of embodiments. Attached Figure Description

[0015] Figure 1 This is a diagram illustrating an example of the hardware structure of an HMD.

[0016] Figure 2 This is an external view showing an example of the structure of an HMD.

[0017] Figure 3 This is an external view showing an example of the structure of an information display terminal.

[0018] Figure 4 This is an example diagram illustrating the experience of VR content.

[0019] Figure 5 This diagram illustrates an example of a VR experience area set up by user B within the real-world space where user A is located.

[0020] Figure 6A This is an example diagram illustrating how to set up a VR experience area.

[0021] Figure 6B This is an example of what is displayed on the HMD 1B when setting up a VR experience area.

[0022] Figure 7 This is a flowchart illustrating an example of the VR experience area setting process in the first embodiment.

[0023] Figure 8 It is visually represented in Figure 7 This is an example of a VR experience area set by the HMD during the processing.

[0024] Figure 9A This is an example of a setup screen used to perform processing for obtaining information from other devices.

[0025] Figure 9B This is a flowchart illustrating an example of a process that retrieves information from other devices.

[0026] Figure 10A This is an example of a setup screen used to perform processing that provides information to other devices.

[0027] Figure 10B This is a flowchart illustrating an example of a process that provides information to other devices.

[0028] Figure 11 This is a diagram illustrating an example of information sent and received by an HMD.

[0029] Figure 12 This is a flowchart illustrating an example of the VR experience area setting process in the second embodiment.

[0030] Figure 13 This is an example of a diagram used to visually illustrate changes in the experience area.

[0031] Figure 14 This is a flowchart illustrating an example of the processing in an HMD that receives a change request from another device.

[0032] Figure 15 This is a flowchart illustrating an example of the VR experience area setting process in the third embodiment.

[0033] Figure 16 This is a diagram showing an example of the VR experience area before and after compositing. Detailed Implementation

[0034] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. These embodiments are examples used to illustrate the present invention, and appropriate omissions and simplifications have been made to make the description clearer. The present invention can also be implemented in various other ways. Unless otherwise specified, each constituent element can be single or multiple. To facilitate understanding of the invention, the positions, sizes, shapes, ranges, etc., of the constituent elements shown in the drawings do not always represent actual positions, sizes, shapes, ranges, etc. Therefore, the present invention is not necessarily limited to the positions, sizes, shapes, ranges, etc., disclosed in the drawings. As examples of various types of information, they are sometimes described using terms such as "table," "list," and "queue," but various types of information can also be expressed using data structures other than these. For example, various types of information such as "XX table," "XX list," and "XX queue" can also be referred to as "XX information." When describing identification information, terms such as "identification information," "identifier," "name," "ID," and "number" are used, but they can be interchanged. When multiple constituent elements with the same or identical function exist, different suffixes are sometimes added to the same reference numerals for description. Furthermore, when it is not necessary to distinguish between the multiple constituent elements, the suffixes are sometimes omitted in the description.

[0035] In implementation methods, the term "processing performed by executing a program" is sometimes used. Here, the computer executes the program via a processor (e.g., CPU, GPU), using storage resources (e.g., memory) and interface devices (e.g., communication ports) to perform the processing specified by the program. Therefore, the entity performing the processing by executing a program can also be considered a processor. A processor includes transistors and other circuits and is considered a circuitry or processing circuitry. Similarly, the entity performing the processing by executing a program can also be a controller, device, system, computer, or node that has a processor. The entity performing the processing by executing a program can be any arithmetic unit and may include dedicated circuitry for performing specific processing. Here, dedicated circuitry may be, for example, an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a CPLD (Complex Programmable Logic Device).

[0036] The program can be installed onto a computer from a program source. The program source can be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and storage resources for storing the program to be distributed, and the processor of the program distribution server distributes the program to other computers. Furthermore, in this embodiment, two or more programs can be implemented by one program, or one program can be implemented by two or more programs.

[0037] The implementation describes a technology where multiple users in the same real-world space wear information display terminals and experience virtual reality (VR) content unfolding in a virtual space. In this case, a content experience area is appropriately set for each user. Here, the experience area refers to the area of ​​the real-world space used when experiencing the content, and it is the area where users are allowed to move and / or perform actions while experiencing the content. If a user leaves the experience area, the information display terminal displays a warning to the user indicating that they have left the experience area, requesting them to return to the experience area, thereby providing a safe content experience.

[0038] First, refer to Figure 1 and Figure 2 The following example illustrates the situation where the information display terminal is configured as an HMD worn on the user's head.

[0039] Figure 1 This diagram illustrates an example of the hardware structure of an HMD. The HMD 1 includes a camera 10, a range sensor 11, a display 12, a speaker 13, a microphone 14, a sensor group 15, a control unit 16, an internal bus 17, a communication unit 20, a processor 21, a memory 22, a storage device 23, a basic operation program 24, a VR audiovisual program 25, and a proximity transmission program 26.

[0040] The processor 21 is composed of a CPU, etc., and the memory 22 is composed of RAM, etc. The storage device 23 is composed of non-volatile storage media such as flash memory ROM, storing the basic operation program 24, the VR audiovisual program 25, and the proximity transmission program 26. The basic operation program 24, the VR audiovisual program 25, and the proximity transmission program 26 are loaded into the memory 22 and executed by the processor 21. Furthermore, in Figure 1 In this context, the basic motion program 24, the VR audiovisual program 25, and the proximity transmission program 26 are different programs. However, this is not a limitation; a single program can contain and provide basic motion functions, VR audiovisual functions, proximity transmission functions, etc. Alternatively, different segmented programs can provide the aforementioned functions.

[0041] When a user wears the HMD 1 and uses the VR audiovisual program 25 to watch VR content, the camera 10 captures the real-world space in front of the HMD 1. When multiple cameras are included, the camera 10 can capture a 360-degree view of the HMD 1's surroundings. Furthermore, when the camera 10 has a 360-degree camera, it can capture a 360-degree view of the HMD 1's surroundings.

[0042] Processor 21 can detect real-world objects (e.g., furniture, people) from images captured by camera 10. Distance sensor 11 acquires the distance to the detected real-world objects, and processor 21 can use data from distance sensor 11 to acquire the position information of the detected objects. Furthermore, by using distance sensor 11, a distance image can be generated to spatially determine the distance to surrounding objects. Additionally, processor 21 can also use images captured by camera 10 to acquire object position information. Here, position information can be world coordinate system information based on real-world space, or local coordinate system information based on the position of HMD 1.

[0043] Display 12 shows images of VR content, and speaker 13 outputs sound from the VR content. Microphone 14 captures the user's voice as needed. Sensor group 15 performs various sensing operations, such as sensing the user's movement in the up, down, forward, backward, left, and right directions, and rotational movement accompanying changes in gaze direction.

[0044] Sensor group 15 includes various sensors for sensing, such as line-of-sight sensors, GNSS sensors, accelerometers, gyroscopes, geomagnetic sensors, and sensors using Wi-Fi (registered trademark) signals. Alternatively, sensor group 15 may include sensors of different types than those described above. Furthermore, sensor group 15 may appropriately omit the structures of the aforementioned sensors. Additionally, sensor group 15 may be omitted from HMD 1.

[0045] A gaze sensor may include a left-eye gaze sensor that detects the gaze of the left eye and a right-eye gaze sensor that detects the gaze of the right eye. It can detect the movement and direction of the left and right eyes to capture the user's viewpoint as the gaze target.

[0046] A GNSS sensor is a device that receives signals from GNSS (Global Navigation Satellite System) satellites located overhead, and is used to detect the current position of the HMD 1. This allows the location of the user wearing the HMD 1 to be detected. Alternatively, distance / position detection can be performed based on the signal reception time between devices using FTM (Fine Timing Measurement) based on Wi-Fi signals, or using CSI (Channel State Information) within the Wi-Fi area.

[0047] An accelerometer is a sensor that detects changes in velocity per unit time, i.e., acceleration, and can capture motion, vibration, and impact. Using an accelerometer, the tilt and orientation of the HMD 1 worn by the user can be detected. Additionally, a gyroscope sensor detects the angular velocity of rotation, capturing longitudinal, lateral, and oblique posture. Therefore, both accelerometers and gyroscopes can be used to detect the tilt and orientation of the HMD 1. A geomagnetic sensor detects the Earth's magnetic field and is used to detect the direction the HMD 1 is facing. By using a 3-axis sensor that detects the geomagnetic field in the forward / backward, left / right, and up / down directions, changes in the geomagnetic field caused by the movement of the HMD 1 can be captured, thus detecting the movement of the HMD 1. Therefore, the posture of the user wearing the HMD 1 can be detected.

[0048] Furthermore, considering the need for shooting in low-light conditions, the camera 10 can be configured to include an infrared camera or infrared illumination to improve infrared light sensitivity. Moreover, the ranging sensor 11 can be a structure such as LiDAR that uses lasers to perform three-dimensional measurements of real space, thereby improving detection accuracy.

[0049] Ministry of Communications 20( Figure 1 The communication interface (in the middle) has multiple communication protocols, including LAN communication, Internet communication, and short-range communication, which are used according to the purpose. For example, the processor 21 receives VR content or sends and receives data from a server that can distribute VR content via Internet communication through the VR audiovisual program 25.

[0050] Furthermore, the processor 21 can communicate directly with other information display terminals, for example, via Bluetooth (registered trademark) or LAN, to send and receive data. Alternatively, the processor 21 can also communicate directly with a content distribution device, for example, to receive VR content or send and receive data. Here, the content distribution device is, for example, a personal computer, a home game console, a portable game console, a smartphone, or other computer.

[0051] Furthermore, some or all of the computational processing performed by the processor 21 can be executed by the server. Alternatively, some or all of the computational processing performed by the processor 21 can also be executed by a content delivery device or the like.

[0052] Figure 2 This is an external view showing an example of an HMD structure. In this example, HMD 1 is a fully immersive HMD. Figure 2 In the middle, to and Figure 1 Structures with the same constituent elements are labeled with the same reference numerals in the accompanying drawings, and descriptions are omitted.

[0053] exist Figure 2 In this example, display 12 has multiple display elements for left and right eyes, displaying images for the left and right eyes respectively, allowing the user to view a three-dimensional image. Alternatively, display 12 can also be configured to perform 3D display using other methods. Furthermore, speakers 13a and 13b correspond to... Figure 1 The speaker 13. The HMD 1 has wearing housings 18a and 18b.

[0054] Furthermore, the information display terminal is not limited to HMDs; it can also be portable information terminals such as smartphones. Additionally, such as Figure 3 In the example shown, the information display terminal can also be a structure composed of a head-mounted device 31 and a portable information terminal 32. Figure 3 In this example, the portable information terminal 32 can be inserted into the head-mounted device 31 and used in this state.

[0055] exist Figure 3 In the example, the camera 10, the range sensor 11, and the display 12 are mounted on the portable information terminal 32. This simplifies the structure of the head-mounted device 31. However, it is not limited to this; the camera 10 and the range sensor 11 can also be mounted on the head-mounted device 31. Furthermore, the speaker 13, microphone 14, sensor group 15, and other structures (not shown) can also be mounted on either the portable information terminal 32 or the head-mounted device 31. Additionally, the above structures can be complemented by combining components mounted on the portable information terminal 32 and the head-mounted device 31.

[0056] A slot 31a for a portable information terminal 32 is formed on the side of the head-mounted device 31, allowing the portable information terminal 32, inserted through the slot 31a, to be held in a storage section 31b inside the head-mounted device 31. Here, a terminal 31c for connecting the portable information terminal 32 and the head-mounted device 31 can be formed on the side opposite to the slot 31a. Data transmission and reception between the portable information terminal 32 and the head-mounted device 31 can be performed via this terminal 31c. Additionally, a battery (not shown) can be provided in the head-mounted device 31, and power can be supplied between the portable information terminal 32 and the head-mounted device 31 via this terminal 31c.

[0057] In this example, an opening 31d is formed on the front side of the head-mounted device 31, which allows the camera 10 of the portable information terminal 32 to take pictures and the ranging sensor 11 to measure distances when the portable information terminal 32 is inserted. However, the opening for the camera 10 to take pictures and the opening for the ranging sensor 11 to measure distances can also be formed on the front side of the head-mounted device 31, respectively. Furthermore, when the head-mounted device 31 has a camera 10, the opening for taking pictures can be omitted. Similarly, when the head-mounted device 31 has a ranging sensor 11, the opening for measuring distances can be omitted.

[0058] Alternatively, the control unit 16 (not shown) can be configured using a processor mounted on the portable information terminal 32, or it can be located on the head-mounted device 31. Alternatively, the control unit 16 can be configured such that a portion is constructed using a device mounted on the portable information terminal 32, and the remainder is located on the head-mounted device 31. Furthermore, the head-mounted device 31 can be equipped with a processor, memory, and other devices for data processing, allowing the control unit 16 to perform data processing separately using both the device mounted on the portable information terminal 32 and the device located on the head-mounted device 31.

[0059] Next, refer to Figure 4 and Figure 5 An example is given to illustrate the situation of experiencing VR content using an information display terminal.

[0060] Figure 4 This scenario depicts a player experiencing VR content while a family member or other third party enters the player's space (real-world space S), i.e., the room. The diagram shows user A experiencing VR content, the HMD 1A used by user A, the controller 41A held by user A, the computer device 42 (a desktop computer in this example) capable of distributing VR content, and the router 43. Meanwhile, user B, as a third party, is not wearing an information display terminal and is not currently experiencing VR content.

[0061] Controller 41A can communicate with HMD 1A, for example, allowing user A to perform specified operations on VR content. Additionally, controller 41A can communicate with content distribution device 42, receiving specified operations from VR content via content distribution device 42. Furthermore, the input interface for user A's operations is not limited to controller 41A; it can also be separately located on HMD 1A. For example, buttons or a touch panel can be provided on HMD 1A, allowing user input through button or touch panel operations. Alternatively, user gestures can be detected by camera 10, or eye-tracking sensors can be included in sensor group 15 to detect user gaze. Router 43 serves as a communication relay structure. Using router 43, for example, a wireless LAN environment can be constructed.

[0062] User A's HMD 1A receives VR content distribution from a server via an access point and network, allowing user A to watch VR content. The network is not limited to the Internet but can also be an intranet. Alternatively, HMD 1A can also receive VR content distribution from content distribution device 42. Furthermore, VR content can be pre-downloaded from the server and content distribution device 42 to HMD 1A, and the VR content stored in HMD 1A's memory 22 can be executed.

[0063] P1 in the diagram represents an example of a VR experience area set up by user A who is experiencing VR content. The VR experience area is the area used to experience VR content; it is the area where the user will move and / or perform actions during the VR content experience. The VR experience area is set to avoid real-world objects present in the real space S. Here, real-world objects are objects that should be avoided during the VR content experience, such as shelves, televisions, and routers 43. However, the types of real-world objects are not particularly limited; they can also be sofas, tables, walls, or other real-world objects.

[0064] Furthermore, if user A leaves the VR experience area P1 due to movement during the VR content experience, HMD 1A can, for example, interrupt the audiovisual experience of the VR content to prevent a collision. Additionally, in the area where user A is about to leave the VR experience area P1, HMD 1A can switch to displaying VR images to draw attention (showing messages, displaying the boundaries of the experience area, etc.), or displaying images of the surrounding real space captured by camera 10, or overlaying such images onto the VR content, thereby prompting user A to avoid leaving the experience area.

[0065] Figure 4 The image shows the VR experience area P1, but user B cannot see the VR experience area P1 set by user A in the real space S. Therefore, user B cannot determine whether they are inside or outside the VR experience area P1 and needs to be careful to avoid contact.

[0066] Figure 5 This diagram illustrates an example of a VR experience area set by user B in the real-world space where user A is located. When user B wears HMD 1B on their head to experience VR content, the VR experience area P2 is set. User B can, for example, use controller 41B to set the VR experience area P2. Here, controller 41B can communicate with HMD 1B, for example, to perform specified operations on the VR content. In this example, HMD 1A and HMD 1B have the same structure, but are not limited to this. HMD 1A and HMD 1B can also be information display terminals with different structures.

[0067] When setting up the VR experience area P2, if user B is unaware of the VR experience area P1 set by user A, then as follows: Figure 5 As shown, a problem arises where VR experience area P2 is set to overlap with VR experience area P1, creating an overlapping area (OS). This overlapping area OS is an area where both user A and user B may coexist, thus posing a high risk of collision.

[0068] Here, refer to Figure 6A and Figure 6B Here is an example of how to set up a VR experience area. When user B operates controller 41B, control unit 16 performs control, and corresponding to the operation, augmented reality lines are superimposed on display 12, which displays images captured by camera 10. Figure 6B This diagram illustrates an example of the display on the HMD 1B when setting up a VR experience area. An image I captured by the camera 10 of the HMD 1B is displayed on the monitor 12, and user B operates the controller 41B to overlay display lines R. According to this example, the user can set up the VR experience area while looking at the augmented reality lines.

[0069] In this way, if other users' VR experience areas can be known in advance, users can operate the controller to set their own VR experience area by avoiding the VR experience areas of other users. Figure 6A , Figure 6B In the example, user A's VR experience area P1 is displayed on the monitor 12 of HMD 1B, and user B can access user A's VR experience area P1. Therefore, user B can draw a line R that avoids the VR experience area P1, thus setting their own VR experience area without creating an overlapping area OS. Furthermore, not limited to the use of controller 41B, even when user B uses other input interfaces, displaying user A's VR experience area P1 can suppress the generation of overlapping area OS.

[0070] Hereinafter, in the first to third embodiments, the technology for setting up the VR experience area will be described in detail.

[0071] [First Implementation Method]

[0072] In the first embodiment, the HMD is able to acquire information about the VR experience area set by other devices. Furthermore, the HMD is able to adjust its own VR experience area.

[0073] Figure 7 This is a flowchart illustrating an example of VR experience area setting processing. This processing is executed by processor 21, for example in… Figure 5 The process starts automatically when the HMD 1B is powered on. Alternatively, for example, user B can start the process by observing user A who is using another device and user B inputting a start command into the HMD 1B via the input interface.

[0074] First, HMD 1 acquires information about other devices (information display terminals used by a third party currently experiencing VR content in the real space) (S701). Here, the information acquired by HMD 1 includes, for example, information about the VR experience area of ​​the other device, location information, and information about the VR content being executed. Alternatively, in the first embodiment, location information and VR content information may be omitted from the acquired information, and other information such as information indicating the user's scheduled usage schedule for the other device may be included.

[0075] HMD 1 overlays the VR experience areas of other devices acquired by other devices onto the display 12 along with the camera images acquired by camera 10 (S702). This overlay display is performed by transforming the coordinate system of the VR experience areas of other devices to the coordinate system of HMD 1 based on the acquired information. The coordinate transformation can be calculated based on the acquired coordinate data, or a VPS (Visual Positioning System) can be used to determine the position information based on feature point information from the camera images. Then, HMD 1 sets its own VR experience area, for example, based on the area selection made by the user through controller operation (S703).

[0076] Here, HMD 1 determines whether the VR experience area set in S703 overlaps with the VR experience area of ​​other devices (S704). HMD 1 makes this determination, for example, by comparing the VR experience area of ​​other devices obtained in S701 with the VR experience area set in S703.

[0077] If the system determines that the VR experience area overlaps with that of other devices (S704 - Yes), HMD 1 notifies the user to choose whether to end the VR content experience. If the user chooses not to end (S708 - No), HMD 1 returns to the processing in S703. After the user selects an area, the VR experience area is set again based on the user's selection.

[0078] On the other hand, if it is determined that there is no overlap between the VR experience area and that of other devices (S704 - No), HMD1 notifies the user to select the VR content to experience and sets the selected VR content (S705). In addition, if the VR content has multiple play modes, HMD1 can also allow the user to select the play mode of the VR content to experience.

[0079] Play modes indicate the style of experiencing VR content. VR content play modes include, for example, seated mode (playing while seated in a chair, etc.), standing mode (playing while standing), and room mode (playing within a designated area). In standing mode, the user stands still in a designated position. In room mode, the user moves within the designated experience area.

[0080] Then, HMD 1 compares the size of the VR experience area set in S703 with the size of the VR experience requirement area (S706). Here, the VR experience requirement area is the area where VR content can be experienced. That is, the VR experience requirement area is the area that must be required (required) to experience the VR content selected by the user, or a recommended area. In addition, if a VR content play mode is selected, the VR experience requirement area is the area where the VR content can be experienced in the selected play mode. The information about the VR experience requirement area can be included in part of the VR content, or it can be obtained separately from the VR content from the server or content distribution device, or read from the memory 22. In addition, the information about the VR experience requirement area is not only the area information of the area, but also the shape information, the volume information of the area including height direction information, etc. In S706, the size of the VR experience area set by the user is compared based on this information.

[0081] If the size of the VR experience area is determined to be larger than the size of the VR experience requirement area (S706 - Yes), HMD 1 begins executing the user-selected VR content because a sufficiently large VR experience area has been ensured for experiencing the VR content, and then the process ends. If the VR experience requirement area is a recommended area (not mandatory), if the VR experience area can ensure a pre-set area satisfaction rate (e.g., 80% or more), it can be determined that a sufficiently large VR experience area has been ensured for experiencing the VR content.

[0082] On the other hand, if it is determined that the size of the VR experience area is smaller than the size of the VR experience requirement area (S706 - No), HMD 1 notifies the user to choose whether to stop the VR content experience. If the user chooses not to stop the experience (S709 - No), HMD 1 notifies the user to reset the VR experience area or content (S710, S711). If the user chooses to reset the VR experience area (S710 - Yes), the process returns to S703. Furthermore, during the reset, HMD 1 preferably overlays the VR experience area of ​​other devices onto the display 12. Then, based on the reset VR experience area, HMD 1 executes the determination in S704 again. Additionally, if the user chooses to reset the VR content to be experienced (S711 - Yes), the process returns to S705. Then, based on the reset VR content's VR experience requirement area, HMD 1 executes the determination in S706 again.

[0083] In S708 and S709, if the user chooses to stop the VR content experience, HMD 1 terminates the process. Alternatively, the process can also terminate if the user does not choose to end the experience or reset the area within the specified time. Furthermore, if the usage schedule information is obtained in S701, the user can be notified of the termination of the schedule for other devices before the process is terminated.

[0084] Additionally, if the user selects to reset the content (S711 - Yes) and the process returns to S705, it can be configured to prevent the user from selecting VR content or play modes with a VR experience requirement area larger than the VR experience area set in S703. In this case, the processing in S706 can be omitted. Alternatively, when the user selects content in S705, the size of the currently set VR experience area can be considered from the beginning, preventing the user from selecting unplayable content or only displaying playable content. This prevents repeated content reselection. Furthermore, for unplayable content, by displaying information about the insufficient area of ​​the VR experience area (size of the insufficient area, satisfaction rate of the VR experience requirement area), the required area size for resetting the VR experience area can be understood.

[0085] Even if a VR experience area is set in a way that does not overlap with the VR experience areas of other devices, the set area may still be unsuitable for experiencing VR content. As shown in this example, in addition to determining whether the set VR experience area overlaps with the VR experience areas of other devices, by comparing the VR experience requirement area of ​​the VR content selected by the user with the set VR experience area, the VR content can only be started after determining that the VR experience area is a suitable area for the user to experience VR content.

[0086] Figure 8 It is visually represented in Figure 7 This diagram illustrates an example of a VR experience area set by the HMD during processing. In this example, HMD 1B worn by user B is in the same real-world space S as user A. Based on user B's operation, the HMD 1B sets its VR experience area P2 in a manner that does not overlap with the VR experience area P1 of user A's HMD 1A (S703). At this time, HMD 1B, based on the information about HMD 1A's VR experience area P1 obtained in S701, overlays the line representing VR experience area P1 onto the image of real-world space S captured by camera 10. In this way, by displaying VR experience area P1, user B, for example, when selecting an area through controller operation, can select the area they wish to set as their own VR experience area P2 in a manner that does not overlap with the VR experience area P1 of other users, i.e., user A.

[0087] However, as Figure 8 As shown, sometimes the size of the VR experience area P2 set by user B does not meet the size of the area required for the VR content that user B wants to experience. That is, the size of the VR experience area P2 set in S703 is sometimes smaller than the area required by the VR content selected in S705 (VR experience requirement area P2R).

[0088] Therefore, in S706, it is determined whether the size of the VR experience area P2 is larger than or equal to the size of the VR experience requirement area P2R. If it is determined that the VR experience area is unsuitable for performing VR content, solutions such as resetting the VR experience area and reselecting the VR content to be experienced are adopted. As a result, HMD 1B can adjust the size of the VR experience area to be suitable for experiencing VR content. For example, if the size of the VR experience area P2 is smaller than the VR experience requirement area P2R, by prompting user B to reset in S710, the VR experience area P2 can be expanded within a range that does not overlap with user A's VR experience area P1 to meet the size of the VR experience requirement area P2R.

[0089] Additionally, HMD 1 can display information to confirm the size of the VR experience area when the user sets the VR experience area. Furthermore, when a play mode is set for VR content, HMD 1 can display information to confirm the size of the VR experience area required in the selected play mode. For example, HMD 1 can overlay lines representing the VR experience area onto an image of the real-world space S captured by camera 10. Furthermore, HMD 1 can also display the size and satisfaction rate of the VR experience area using text, numbers, symbols, etc.

[0090] Figure 9A and Figure 9B This is a diagram illustrating an example of a process for acquiring information from other devices. For example, it is... Figure 5 The HMD 1B shown performs the following process when requesting information from the HMD 1A, which is another device.

[0091] like Figure 9A As shown, HMD 1 displays a setting screen 91 on display 12 for information acquisition processing. In this example, setting screen 91 includes multiple screen areas. When the setting 92 for "Automatically acquire information from other devices" is set to ON, HMD 1 automatically acquires information that is set to ON in the setting 93 for "Acquire information content". For example, when acquiring information from other devices again, since the automatic information acquisition setting 92 is set to ON, HMD 1 again automatically acquires information that is set to ON in the setting 93 for "Acquire information content". This simplifies user operation. Furthermore, the automatic information acquisition setting 92 can be initially set to ON.

[0092] On the other hand, when the automatic information acquisition setting 92 is set to OFF, the user manually selects ON / OFF in the information content acquisition setting 93 to set whether to acquire each piece of information, and HMD 1 acquires the information that is set to ON. Here, for example, when acquiring information from another device again, when the automatic information acquisition setting 92 is set to OFF, the user manually sets the information content acquisition setting 93 again, and HMD 1 acquires information based on the re-set information content acquisition 93. That is, when the automatic information acquisition setting 92 is OFF, the user can confirm the information content acquisition setting 93 each time information is requested, and has the opportunity to change the information to be acquired according to the situation. In addition, based on the automatic information acquisition setting 92 and the information content acquisition 93, the setting information indicating whether to automatically acquire information from other devices, the information indicating the content of the information to be acquired, etc., is stored in, for example, memory 22. In addition to selecting ON / OFF for information acquisition through the setting screen, for information that is set to OFF in setting 93, a message about whether to acquire information can also be displayed on the display 12 of HMD 1, allowing the user to make a selection.

[0093] Reference Figure 9B The flowchart illustrates the process. Additionally, this flowchart corresponds to... Figure 7 The processing in S701. HMD1 determines whether the automatic information acquisition setting 92 is ON (S901). If the automatic information acquisition setting 92 is ON (S901 - Yes), HMD1 requests information from other devices based on the information acquisition content setting 93 (S904). If information is received from other devices (S905 - Yes), the information is stored in memory 22 (S906).

[0094] On the other hand, if the automatic information acquisition setting 92 is OFF (S901 - No), HMD 1 notifies the user to choose whether to request information from other devices (S902). Then, if the user selects to request information (S902 - Yes), HMD 1 accepts the setting content of the information acquisition content setting 93, which the user manually selected as ON / OFF (S903). Then, HMD 1 requests information based on the information acquisition content accepted in S903 (S904). The information request can be sent to devices within a specified range via short-range communication, or to devices pre-registered by the user or selected from a list of nearby devices.

[0095] Additionally, the process ends if the user chooses not to request information from other devices (S902 - No), or if no information is received from other devices within a specified time (S905 - No). Although not illustrated, if no information is received from other devices within a specified time, information can be requested again a predetermined number of times.

[0096] Figure 10A and Figure 10B This is a diagram illustrating an example of a process for providing information to other devices. For example, it is... Figure 5 The processing performed by HMD 1A when it receives an information request from HMD 1B, which is another device.

[0097] like Figure 10A As shown, HMD 1 displays a setting screen 96 on display 12 for setting whether to provide information. In this example, setting screen 96 includes multiple screen areas. When the setting 97 for "Automatically send information to other devices" is set to ON, HMD 1 automatically sends the information that is set to ON in the setting 98 for "Send information content". For example, when sending information to other devices again, since the setting 97 for automatically sending information is set to ON, HMD 1 can again automatically send the information that is set to ON in the setting 98 for "Send information content".

[0098] Furthermore, when the automatic message sending setting 97 is set to ON and the "whether to confirm each message sent" setting 99 is set to ON, HMD 1 notifies the user that a message has been sent each time it is sent automatically. This notification may include the content set in the message content setting 98. On the other hand, when the "whether to confirm each message sent" setting 99 is set to OFF, HMD 1 does not notify the user when sending messages automatically. For example, in situations where messages are sent to a limited number of information display terminals within a home, no notification is given even during automatic sending, thus avoiding interference with the user's content experience.

[0099] Furthermore, when the automatic message sending setting 97 is set to ON, it is preferable to initially set the confirmation setting 99 for each message transmission to ON. This initial setting ensures that even if the user mistakenly sets the automatic message sending setting 97 to ON, the user is aware that a message is being sent to another device. Additionally, information indicating the location where HMD 1 is typically used (e.g., home address or latitude and longitude) can be pre-registered. If the device moves a specified distance from that location, a notification will be sent even if the confirmation setting 99 is OFF. Multiple typically used locations can be registered. Furthermore, even if the automatic message sending setting 97 is ON, if the device moves a specified distance from its typically used location, automatic message transmission may not be performed; instead, a notification will prompt the user to choose whether to send the message. Furthermore, when using a line shared by an unspecified number of users, such as a public network, or an unencrypted line, a notification can be sent to prompt the user to choose whether to send the message, or the message can be not sent at all. This prevents the message from being sent to unexpected devices under unusual usage conditions.

[0100] On the other hand, when the automatic message sending setting 97 is OFF, HMD 1 notifies the user that a message request has been received from another device. Then, when the user manually sets the sendable information setting 98 in the message content sending setting to ON, HMD 1 sends the ON-set information. In this way, with the automatic message sending setting 97 OFF, the user can confirm the message content sending setting 98 each time they send a message to another device, and has the opportunity to change the information to be sent according to the situation. Furthermore, the automatic information retrieval setting 92 can be initially set to ON. This prevents the user from unintentionally sending their own information to other devices.

[0101] Reference Figure 10B The flowchart illustrates the process. When HMD 1 receives a message request from another device (S1001 - Yes), it determines whether the automatic message sending setting 97 is ON (S1002). If the automatic message sending setting 97 is ON (S1002 - Yes), HMD 1 sends the message to the requesting device based on the message content setting 98 (S1006). Alternatively, the message can also be sent from HMD 1 to devices within a specified range via short-range communication.

[0102] On the other hand, when the automatic information transmission setting 97 is OFF (S1002 - No), HMD 1 notifies the user that there is an information request from another device. Here, HMD 1 displays on display 12 information indicating the content requested from another device based on the information content acquisition setting 93 (S1003). If the user selects to respond to the information request (S1004 - Yes), HMD 1 accepts the information transmission content transmission setting 98, which the user has manually selected to be ON / OFF (S1005). Then, HMD 1 transmits the information based on the information transmission content transmission setting 98 (S1006).

[0103] Figure 11 This diagram illustrates one example of the types of information that an HMD can send and receive. For example, HMD 1 sends and receives content information 101, player location information 102, and play area information 103. Additionally, other types of information 104 can also be sent and received.

[0104] Content information 101 contains information about the VR content you are booking or currently experiencing. Content information 101 may include, for example, the content name, content category, supported play modes, supported number of players, supported platforms, and content progress information.

[0105] The content name is information indicating the name (e.g., title) of the VR content. The content category is information indicating the category of the VR content. In the case of VR content being a game, the content category could be information indicating game categories such as action, sports, RPG (role-playing game), SLG (simulation game), shooting, racing, etc.

[0106] Supported play modes refer to information about the play modes available for VR content. As mentioned above, VR content play modes include, for example, seated mode, standing mode, and room mode. Additionally, information about the size of the area used in the play mode may also be included. For content that can be experienced by multiple people in the same real-world space, information about the required area size for the number of participants may also be included.

[0107] The supported player count refers to information about the number of people who can experience VR content. This could include information such as the number of people currently experiencing VR content and the number of people who can experience VR content. This includes not only those experiencing content in a network environment but also those experiencing content in a local environment. Each player count can be set together or separately.

[0108] Supported platforms refer to information about platforms that distribute VR content. These platforms can be content distribution devices such as computer devices configured in the physical space, portable computer devices, or servers capable of distributing VR content. If multiple platforms are supported, information about each supported platform can be included.

[0109] Content progress information describes the advancement of VR content being played on the HMD 1 and experienced by the user. This information may include, for example, player level and ranking. Additionally, it may include information about the current level, scene, stage, number of rounds elapsed, current quest, current event, and currently held items. Furthermore, it may include information about the currently playable or movable areas, venues, and maps within the content.

[0110] Player location information 102 is information about the location of HMD 1, which is scheduled to perform or is currently performing VR content. Player location information 102 can be, for example, location information calculated by HMD 1 using GNSS signals received by the GNSS sensors included in sensor group 15. Alternatively, player location information 102 can also be location information calculated by HMD 1 based on feature points in real space to estimate its own position. That is, player location information 102 can also be location information based on V-SLAM (Visual Simultaneous Localization and Mapping). Furthermore, player location information 102 can also be location information obtained by HMD 1 using location information services. Additionally, player location information 102 can also be location information obtained using Wi-Fi CSI, or location information obtained using other known methods.

[0111] Play area information 103 represents the VR experience area set up by the user for experiencing VR content. Furthermore, as an example, play area information 103 can be represented using coordinates corresponding to vertices. For instance, a quadrilateral VR experience area can be set using four coordinate positions, or other polygonal VR experience areas can be set by adding or removing coordinate positions. The coordinate positions can use a local coordinate system with the user as the origin, or a world coordinate system with a specific location within the real-world space S as the origin.

[0112] Other information 104 may include, for example, information about the user's age and physique when experiencing VR content, information about the amount paid for the VR content, priority information regarding the setting of play areas (VR experience area settings) between users, and information about the user's scheduled use of the content. For example, as a priority, it may specify which user's experience area should be prioritized in overlapping areas. In this case, in the overlapping areas, the VR experience area of ​​a user with higher priority is preferentially assigned compared to the VR experience area of ​​a user with lower priority. In addition, the scheduled use may be indicated by the date and time that the user intends to experience the VR content. Age, physique, amount paid, priority, and scheduled use can be preset.

[0113] As described above, according to this embodiment, by adjusting its own VR experience area in conjunction with the VR experience area of ​​other devices and also considering the VR experience requirements of the content, the experience area can be appropriately set even in a real space with limited size, such as a room, where other devices are present.

[0114] Furthermore, in the above description, the processing of VR experience area settings, information acquisition from other devices, and information provision to other devices are performed on information display terminals such as HMDs, but are not limited to this. For example, information display terminals such as HMD 1A and HMD 1B can pre-register in the server and content distribution device whether to request information from other devices, the information content to be acquired, whether to respond to information requests from other devices, and the information content to be sent, so that... Figure 7 , Figure 9B , Figure 10B At least a portion of the processing is performed by the server and content distribution device. For example, when HMD 1A starts up and connects to the server, HMD 1A sends information about the set VR experience area and content. Then, when HMD 1B starts up and connects to the server, the server determines whether HMD 1A and HMD 1B are located in the same physical space. If it determines that HMD 1A and HMD 1B are in the same physical space, the server can begin processing, sending information about HMD 1A's VR experience area to HMD 1B to cause it to execute S702, and the server can execute S704, S706, S901~S906, etc. Alternatively, when HMD 1A and HMD 1B are connected to the content distribution device via short-range communication, the content distribution device can also begin processing directly without determining whether the two information display terminals are in the same space. In this way, by having the server and content distribution device perform part of the processing, the processing load on the information display terminals can be reduced, and the scope of information directly provided to other devices can be decreased.

[0115] According to the description of the first embodiment, as an example, an information display terminal is provided. This information display terminal includes a display, a communication interface, an input interface used by the user for input operations, and a processor. The processor communicates with other devices (located in the same physical space as the information display terminal) via the communication interface, receives information indicating a first experience area set for experiencing content executed on other devices, and determines whether a second experience area selected by the user via the input interface overlaps with the first experience area. When it is determined that the second experience area does not overlap with the first experience area, the processor displays a message on the display to prompt the user to select content they wish to experience using the information display terminal, and compares the size of the experience requirement area required to experience the content selected by the user via the input interface with the size of the second experience area. Then, if the size of the second experience area is greater than or equal to the size of the experience requirement area, the processor controls and sets the second experience area as the area for experiencing the content selected by the user. Furthermore, an information processing method executed on the information display terminal and an information processing system having an information display terminal and a server or content distribution device are also provided.

[0116] [Second Implementation]

[0117] The second embodiment will now be described. Furthermore, for structures and processes identical to those in the first embodiment, descriptions will be omitted or briefly described. In the second embodiment, the HMD adjusts the experience area with other devices.

[0118] Figure 12 This is a flowchart illustrating an example of VR experience area setting processing. Similar to the first embodiment, this processing is executed by processor 21, for example, in... Figure 5 Processing can begin automatically upon startup of HMD 1B. Alternatively, user B can observe user A using another device and input a processing start command into HMD 1B via the input interface, thereby initiating the processing.

[0119] and Figure 7 Similarly, in S701-S703, HMD 1 acquires information about other devices (S1201) and overlays the acquired VR experience areas of other devices onto the camera image acquired by camera 10 (S1202). Then, HMD 1 sets the VR experience area based on the user's area selection (S1203). Here, the information acquired by HMD 1 includes, for example, information about the VR experience areas of other devices (i.e., play area information 103), location information (i.e., player location information 102), and information about the VR content being executed (i.e., content information 101).

[0120] HMD 1 displays information to prompt the user to select the VR content they wish to experience, and accepts the user's selection (S1204). Here, if the VR content has multiple play modes, HMD 1 also allows the user to select the play mode of the VR content and accepts their selection. Additionally, although not illustrated, in S1203... Figure 7 Similarly, S704 can determine whether the set VR experience area overlaps with the VR experience area of ​​other devices, and only proceed to S1204 when it is determined that there is no overlap.

[0121] Next, with Figure 7 Similarly, in S706, HMD 1 compares the size of the VR experience area set in S1203 with the size of the VR experience requirement area corresponding to the VR content and play mode selected by the user in S1204 (S1205). If it is determined that the size of the VR experience area is greater than the size of the VR experience requirement area (S1205-Yes), since a sufficiently large VR experience area is ensured to experience the VR content selected by the user, HMD 1 begins to execute the VR content (S1214). Then the process ends.

[0122] On the other hand, if it is determined that the size of the VR experience area is smaller than the required size of the VR experience area (S1205 - No), HMD 1 notifies the user, prompting them to choose whether to stop the VR content experience (S1206). If the user chooses not to stop the VR content experience (S1206 - No), HMD 1 executes the process in S1207. On the other hand, if the user chooses to stop the VR content experience (S1206 - Yes), HMD 1 ends the process.

[0123] In S1207, HMD 1 notifies the user, prompting them to choose whether to adjust the VR experience area in conjunction with other devices. If the user chooses not to adjust in conjunction with other devices (S1207 - No), HMD 1 accepts the user's input and returns to S1203 to reset the VR experience area and VR content.

[0124] Additionally, during the reset in S1203, HMD 1 can also overlay the VR experience areas of other devices onto the display. Furthermore, HMD 1 can use measurement information of the real space obtained by camera 10 or range sensor 11, and VR experience area information of other devices obtained in S1201, to calculate a VR experience area scheme and display the calculated VR experience area scheme on display 12. Moreover, HMD 1 controls the calculation of the VR experience area scheme so that the sum of the size of the VR experience areas of other devices and the size of the VR experience area scheme does not exceed the size of the real space. Therefore, the VR experience area can be set according to the calculated VR experience area scheme without requiring user area selection. Additionally, during the reset in S1204, it can be set so that the user cannot select VR content or play modes with a VR experience area larger than the VR experience area set in S1203.

[0125] Furthermore, if even after resetting via S1203 and S1204, it is still impossible to set one's own VR experience area in a way that does not overlap with the VR experience area of ​​other devices and is larger than the VR experience requirement area (S1205 - No), then proceed to S1206 and S1207 again. For example, such as Figure 13 As shown, in the same room or the same real space S, there are users A and B. The VR experience area P1 of HMD 1A has been set. When HMD 1B wants to expand the VR experience area P2 to meet the VR experience requirement area P2R, an overlapping area OS will occur. This situation is predictable.

[0126] On the other hand, in S1207, when the user selects to perform adjustments with other devices (S1207-Yes), HMD1 performs regional adjustments involving negotiations with other devices, including the following description.

[0127] First, in the area adjustment between other devices, HMD 1 calculates the area to be changed in order to experience the VR content selected by the user in S1204 (S1208) for its own VR experience area. That is, HMD 1 calculates the area that is the difference between the VR experience requirement area and the VR experience area.

[0128] Additionally, HMD 1 uses the VR content information of other devices obtained in S1201 to calculate the VR experience requirement area for other devices (S1209). Alternatively, instead of calculating the VR experience requirement area for other devices in HMD 1, information about the VR experience requirement area can be obtained as part of the VR content information in S1201. Then, HMD 1 determines whether the area obtained by subtracting the changed area calculated in S1208 from the VR experience area of ​​other devices obtained in S1201 is larger than or equal to the size of the VR experience requirement area of ​​other devices (S1210). That is, HMD 1 determines whether the user of other devices can still experience VR content after reducing the area calculated in S1208.

[0129] If, even after reducing the size of the modified area, the VR experience area remains larger than the required size and users on other devices can continue to experience VR content (S1210 - Yes), HMD 1 sends a request to other devices to reduce the size of the VR experience area to the modified area calculated in S1208 (S1211). For example, such as... Figure 13 As shown, if the VR experience area P1 of HMD 1A is set to be larger than the VR experience requirement area P1R, and it is determined that even if its size is reduced, user A can continue to experience VR content, HMD 1B sends a request to HMD 1A. Additionally, HMD 1 can also send the result calculated in S1209 along with the request.

[0130] Then, HMD 1 receives a judgment result from other devices regarding whether it agrees to the region change (S1212). Furthermore, the judgment result sent from other devices can be the result of manual operation by the user on those devices, or it can be the result of processing by the processor of those other devices without user input. For example, conditions such as whether to agree to the region change and the upper limit of the change can be preset, and the processor can determine whether to agree to the region change request based on the preset conditions. In this way, by controlling the process so that the processor makes the judgment based on the preset conditions without requiring user input, the immersion of the user experiencing VR content can be maintained. Additionally, if a judgment result agreeing to the region change is sent without user input, other devices can notify the user of the region change at an opportune time, such as when the content ends. Other devices can also adjust whether to notify the user of the region change based on the magnitude of the change.

[0131] If the received judgment result indicates that the user of another device agrees to the change (S1212 - Yes), HMD 1 changes its own VR experience area based on the change area calculated in S1208 (S1213). Then, HMD 1 begins executing the VR content set in S1204 (S1214), and then the process ends. On the other hand, if the judgment result sent from another device indicates that the other device does not agree to the change, or if no judgment result is received after a specified time (S1212 - No), HMD 1 returns the process to S1203. Although not illustrated, if no information is received from another device within the specified time, information can be requested again a predetermined number of times.

[0132] In S1210, if it is determined that the VR experience area of ​​another device, after subtracting the changed area, would become smaller than the VR experience area required by the other device (S1210 - No), HMD 1 determines, based on the content information of the other device obtained in S1201, whether the VR content currently being executed on the other device supports multiple play modes (S1215). Then, if multiple play modes exist (S1215 - Yes), HMD 1 notifies the user, prompting them to choose whether to request the other device to change the play mode (S1216).

[0133] Furthermore, in S1215, HMD 1 preferably determines not only whether multiple play modes exist, but also whether, among these multiple play modes, there exists a play mode in which, even if the VR experience area of ​​other devices is reduced by the changed area portion calculated in S1208, the user of the other devices can still experience VR content. In this case, in S1215, similar to S1210, it is determined whether the size of the reduced area after reducing the changed area portion from the VR experience area of ​​other devices is greater than or equal to the size of the VR experience requirement area corresponding to the play mode.

[0134] If the user selects to request another device to change the play mode (S1216 - Yes), HMD 1 sends a play mode change request to the other device (S1217). Then, as described above, based on the judgment result received from the other device, HMD 1 changes the VR experience area and begins executing VR content (S1213~S1214), or returns the process to S1203. On the other hand, if the user selects not to request from other devices (S1216 - No), HMD 1 returns the process to S1203.

[0135] In S1215, if it is determined that the VR content being executed by another device does not support multiple play modes, or if it is determined that if the VR experience area is reduced or the area is changed, there will be no play mode that can experience VR content (S1215-No), HMD 1 will notify the user and prompt the user to choose whether to request another device to change the VR content (S1218).

[0136] If the user selects to request other devices to change the content (S1218 - Yes), HMD 1 sends a request to other devices to change the VR content currently being executed (S1219). As described above, based on the judgment result received from other devices, HMD 1 changes the VR experience area and starts executing VR content (S1213~S1214), or returns the process to S1203. On the other hand, if the user selects not to request other devices (S1218 - No), HMD 1 returns the process to S1203.

[0137] Furthermore, similar to the determination of whether to change the region, the judgment result sent from other devices regarding whether to change the play mode or VR content can be the result of manual operation by the user on the other device, or it can be the result processed by the processor of the other device without user input. For example, conditions such as whether to agree to the change, the changeable play mode, and the changeable VR content can be preset, and the processor determines whether to agree to the change request based on the preset conditions. If the judgment result of agreeing to change the play mode is sent without user input, the other device can notify the user of the change at the end of the content. In addition, if the judgment result of agreeing to change the VR content is sent without user input, the other device preferably notifies the user of the VR content change before switching VR content.

[0138] Additionally, when notifying users to choose in S1207 whether to coordinate with other devices to adjust the VR experience area, in S1216 whether to request other devices to change the play mode, and in S1218 whether to request other devices to change the VR content, at least some of the information obtained in S1201, such as priority, scheduled usage, payment amount, age, and physical condition, can be displayed. This can be used as a reference when users decide whether to make a request; for example, if their priority is low or other users' end time is approaching, they can decide not to request a change. Alternatively, instead of issuing a notification prompting the user to choose / judge, other information can be used for HMD 1 to determine whether to make adjustments with other devices and whether to send a request.

[0139] Figure 14 This is a flowchart illustrating an example of processing in an HMD that receives a change request from another device. For example, it shows... Figure 5 The HMD 1A shown receives a change request from HMD 1B, which is another device. Figure 12 This is an example of a process executed by the processor 21 of the HMD 1A during S1211, S1217, or S1219. This process can begin when the HMD 1A is started, or when the presence of other devices in the vicinity is detected via the camera 10 or proximity communication.

[0140] HMD 1 passed Figure 10B The process shown involves sending its own VR experience area information, location information, and ongoing VR content information to other devices (S1401). When HMD 1 receives a change request from another device (S1402), it determines the received request (S1403~S1405). Alternatively, it may receive the VR experience area information, location information, and scheduled VR content information from other devices along with the change request. Furthermore, although not shown, it may also begin when HMD 1 detects the presence of other devices in the vicinity. Figure 14 During processing, regardless of whether a change request is received from other devices, information such as the VR experience area is obtained from other devices.

[0141] When the change request is to reduce the VR experience area (S1403 - Yes), HMD 1 determines whether a change permission has been set (S1411). By allowing the user to pre-configure permission or prohibition settings, the immersion of the user experiencing VR content can be maintained. Furthermore, since it is determined in S1413 whether area reduction will affect the user's VR content experience, it can be initially set to allow the change. Alternatively, a notification can be sent in S1411 to prompt the user to choose whether to allow the change.

[0142] If the change is permitted (S1411 - Yes), HMD 1 calculates a region change plan regarding the VR experience area change (S1412). That is, as the region change plan, HMD 1 calculates the size of the VR experience area after the region change, the shape of the VR experience area after the region change, and the center of gravity position of the VR experience area after the VR experience area is moved, etc. Additionally, in S1402, HMD 1 receives, along with the change request, information from other devices of the requesting party, such as... Figure 12In the case of the data for the changed area calculated in S1208, this data can be used to calculate the area change plan. HMD 1 determines whether the size of the changed VR experience area is greater than or equal to the size of the VR experience requirement area of ​​the currently executed VR content when the VR experience area is changed based on the calculated area change plan (S1413). In addition, this example only describes the comparison between the VR experience area and the VR experience requirement area, but if there is an unused area in the real space that is not used as a VR experience area by any device, it is possible to determine whether the VR experience area can be changed while considering the use of the unused area.

[0143] HMD 1 determines that even if the VR experience area is changed, it will still be larger than the required VR experience area and will not affect the user's VR content experience (S1413 - Yes), and implements the change of the VR experience area based on the area change plan (S1414). For example, in Figure 13 In the illustrated case, HMD 1A can be modified by shrinking the VR experience area P1 to the VR experience requirement area P1R, or it can be modified to a VR experience area P11 (which only shrinks the portion in the direction of the overlapping area OS). Furthermore, the VR experience area is not limited to a quadrilateral; it can also be modified to a hexagon or other shapes while removing only the overlapping area OS. Alternatively, by changing the size and shape of the VR experience area P1, it can be moved away from the VR experience area of ​​HMD 1B, thus creating an interference area. When the VR experience areas of different users are in contact with each other, the possibility of collisions due to users leaving their VR experience areas increases. Therefore, it is preferable to set interference areas between the VR experience areas to separate them. Regarding this interference area, a predetermined distance can be ensured, or a user-defined distance can be used to ensure the distance. Furthermore, by adjusting the VR experience area so that the user's current position is included within the VR experience area when moving it, content playback can continue. Additionally, the adjustment of the VR experience area can be implemented in stages multiple times to increase the distance between users.

[0144] Then, HMD 1 sends the change result to other devices of the requesting party (S1406). Additionally, the change result in S1406 can be sent via short-range communication to devices located within a specified range of HMD 1. Furthermore, before implementing the VR experience area change in S1414, HMD 1 can also notify the user of information regarding area shrinkage or movement. For example, if the shrinkage rate of the changed VR experience area or the amount of movement of the VR experience area's center of gravity exceeds a specified threshold, it is preferable for HMD 1 to notify the user. Additionally, if the center of gravity of the changed VR experience area moves more than a specified distance from the user's position, a notification can be issued to draw the user's attention.

[0145] On the other hand, if the change is not permitted (S1411-No), HMD 1 sends a message to the requesting device stating that the VR experience area cannot be changed (S1441). Additionally, if it is determined that the size of the changed VR experience area is smaller than the size of the VR experience requirement area for the currently executing VR content, and that the change will affect the user's VR content experience (S1413-No), HMD 1 also sends a message to the requesting device stating that the VR experience area cannot be changed (S1441).

[0146] Although not in Figure 14 As shown, if it is determined that a change in region would affect the user's VR content experience (S1413-No), it can also, according to preset settings, transfer to the processing steps S1421 or S1432 below to change the play mode or content, regardless of the content of the change request from other devices. In this case, HMD 1 can prompt the user through images or sounds whether to change the play mode or content.

[0147] If the request received in S1402 is determined to be a change of play mode (S1404-Yes), HMD 1 determines whether there are other play modes that can be changed in the VR content being executed (S1421).

[0148] If it is determined that the user can experience the content in another play mode (S1421 - Yes), HMD 1 notifies the user that it has received a request to change the play mode of the VR content (S1422). At this time, HMD 1 may include, for example, a message prompting the user to choose whether to change the play mode and the specific changes (e.g., changing from room mode to standing mode). Alternatively, HMD 1 may notify the user by displaying the message on display 12.

[0149] If the user grants permission to change the play mode (S1423 - Yes), HMD 1 implements the play mode change (S1424) and sends the change result to the requesting device. Alternatively, after being notified of receiving the play mode change request in S1422, in S1423, the user can grant permission to change the play mode and input a play mode different from the current play mode. In this case, it is preferable to display a list of candidate play modes containing only those with a VR experience requirement area smaller than the current play mode, based on the determination result of S1421, or to set it so that the user cannot select a play mode other than the candidate play modes. Furthermore, similar to S1414, before implementing the VR experience area change in S1424, HMD 1 can notify the user of information regarding the area reduction.

[0150] Additionally, HMD 1 controls the implementation of a play mode change in S1424, and changes the VR experience area accordingly to match the VR experience requirements of the changed play mode. The calculation of the area change scheme is performed, for example, in S1421.

[0151] On the other hand, if it is determined that there is no changeable play mode for the VR content being executed (S1421-No), HMD 1 sends the result that the play mode cannot be changed to the requesting device (S1441). In addition, if the user inputs that the play mode cannot be changed (S1423-No), HMD 1 also sends the result that it cannot be changed to the requesting device (S1441).

[0152] Although not in Figure 14 As shown, if it is determined that the currently executing VR content does not have a play mode that can be changed (S1421-No), the process can be transferred to S1432 below to change the content, regardless of the content of the change request from other devices, according to the preset settings. At this time, HMD 1 can prompt the user whether to change the content through images or sounds.

[0153] Furthermore, similar to S1411, the setting for changing the play mode can be done through initial settings or pre-set user settings, omitting the processing in S1422~S1423. In this case, the immersion of the user experiencing VR content can be maintained.

[0154] If the request received in S1402 is determined to be a content change (S1405 - Yes), HMD 1 notifies the user that a request has been received to change the content being experienced (S1431). HMD 1 displays, for example, a message on display 12 indicating that a content change request has been received, as well as a message prompting the user to choose whether to make the content change.

[0155] If the user enters permission to change the content (S1432 - Yes), HMD 1 implements the content change (S1433) and sends the change result to the requesting device (S1406).

[0156] Additionally, HMD 1 controls the implementation of the content change in S1433, and changes the VR experience area accordingly to match the VR experience requirement area of ​​the changed content. Furthermore, the notification in S1431 may include a notification prompting the user to select other content, and in S1432, the user enters a change permission and selects content different from the current content. In this case, it is preferable to display a list of candidate content for changes in the notification in S1431 that contains only content with a VR experience requirement area smaller than the VR experience requirement area of ​​the currently executed content, or to configure the system so that the user cannot select content other than the candidate content for changes.

[0157] In addition, if the user inputs content that cannot be changed (S1432-No), HMD 1 sends a message to the requesting device that the content cannot be changed (S1441).

[0158] In addition, although not in Figure 14 As shown, if the execution of VR content has ended in HMD 1 when a change request is received in S1402, it is preferable to control the process after S1403 to not perform the subsequent processing, but instead notify the requesting device that the execution of VR content has ended.

[0159] As described above, according to the second embodiment, by using the experience area information and content information of its own device and other devices, and cooperating with other devices to adjust the area, even if it is impossible to set a suitable VR experience area by itself alone, it is possible to obtain the opportunity to set a suitable VR experience area for the user's content experience.

[0160] Furthermore, in the above description, the processing of setting the VR experience area, obtaining information from other devices, and providing information to other devices are performed on an information display terminal such as an HMD, but are not limited thereto. Similar to the first embodiment, for example, content information, location information, etc., sent from information display terminals such as HMD 1A and HMD 1B can also be used to enable... Figure 12 , Figure 14At least part of the processing is performed by the server and content distribution device. For example, when the server and content distribution device select whether to coordinate with other devices to adjust the VR experience area in S1207, whether to request other devices to change the play mode in S1216, and whether to request other devices to change the VR content in S1218, they can use information such as priority, usage schedule, payment amount, age, and physique obtained from HMD 1A and HMD 1B. In this case, information display terminals with high priority or high user payment amounts may not be requested to make change requests, while information terminals with low priority or low payment amounts may be set to be unable to refuse change requests. In addition, the server and content distribution device can also use the value of the VR experience demand area, which is reduced according to age, physique, etc., to make judgments in S1205, S1210, and S1413. When a usage reservation has been pre-entered, the priority of users who have entered reservations can be set higher, or a threshold such as 2 hours per day can be set for the user's VR content experience time, and the priority of information display terminals that exceed the threshold can be set lower.

[0161] According to the description of the second embodiment, as an example, an information display terminal is provided. This information display terminal includes a display, a communication interface, an input interface used by the user of the information display terminal for input operations, a camera for capturing real-world space, and a processor. The processor communicates with other devices (located in the same real-world space as the information display terminal) via the communication interface, receives information indicating a first experience area set for experiencing first content executed on other devices, and displays the first experience area and the image of the real-world space captured by the camera superimposed on the display using the received information when the user selects an area via the input interface. Additionally, the processor receives information indicating a second experience area selected by the user via the input interface, and information about the second content selected by the user, and compares the size of the second experience area with the size of a second experience requirement area corresponding to the second content. Then, if the size of the second experience area is smaller than the size of the second experience requirement area, the processor controls the display to show a message prompting the user to select whether to perform adjustments with other devices (i.e., coordinated adjustments). Furthermore, an information processing method executed on the information display terminal and an information processing system having an information display terminal and a server or content distribution device are also provided.

[0162] [Third Implementation Method]

[0163] The third embodiment will now be described. Furthermore, for structures and processes identical to those in the first or second embodiment, descriptions will be omitted or briefly described. In the third embodiment, the HMD combines the experience areas of its own device with those of other devices to reset the experience area. For example, Figure 5 The HMD 1B is combined with other devices such as the HMD 1A to create a composite experience area.

[0164] Figure 15 This is a flowchart illustrating an example of VR experience area setting processing. Similar to the first and second embodiments, this processing is executed by processor 21, for example, in... Figure 5 HMD 1B starts processing automatically upon startup, or it can be started by user B's operation.

[0165] and Figure 7 S701~S703, Figure 12 Similarly, in S1201-S1203, HMD 1 acquires information from other devices (S1501) and overlays the acquired VR experience areas of other devices onto the camera image acquired by camera 10 (S1502). Then, HMD 1 sets the VR experience area based on the user's area selection (S1503). Here, the information acquired by HMD 1 includes, for example, information about the VR experience areas of other devices (i.e., play area information 103), location information (i.e., player location information 102), and information about the VR content being executed (i.e., content information 101).

[0166] HMD 1 displays information to prompt the user to select the VR content they wish to experience, and accepts the user's selection (S1504). Here, if the VR content has multiple play modes, HMD 1 also prompts the user to select the play mode of the VR content and accepts their selection. Additionally, although not illustrated, in S1503... Figure 7 Similarly, S704 can determine whether the set VR experience area overlaps with the VR experience area of ​​other devices, and only proceed to S1504 when it is determined that there is no overlap.

[0167] Next, HMD 1 uses the content information obtained in S1501 to determine whether the VR content received in S1504 is the same as the VR content being executed by other devices (S1505). If it is determined that their respective VR content is the same (S1505 - Yes), HMD 1 notifies the user, prompting the user to choose whether to experience the VR content with other users (S1506).

[0168] If a user of HMD 1 chooses to experience the content with other users (S1506 - Yes), HMD 1 determines whether multiplayer gameplay of VR content can be performed collaboratively (online) with other devices (S1507). HMD 1 uses, for example, its own content information and the content information of other devices obtained in S1501, to determine whether multiplayer gameplay is possible based on whether the conditions for supporting a certain number of players are met. Additionally, HMD 1 can use the content progress information contained in the content information to determine whether the difference in content progress between its own device and other devices is within a specified range. Furthermore, HMD 1 can determine whether collaborative gameplay with other devices is possible based on, for example, each user's player level, level information, and the area currently being played within the content.

[0169] If HMD 1 determines that it can collaborate with other devices to execute content (S1507 - Yes), it sends a collaboration request for content execution to other devices (S1508). Alternatively, the order of S1506 and S1507 can be reversed, and if HMD 1 determines that it can collaborate with other devices to execute content, it can notify the user to choose whether to experience VR content with other users.

[0170] Additionally, upon receiving a request from HMD 1, other devices notify the user of that device, prompting them to choose whether to experience VR content with other users. Then, after the user enters their selection, the result is sent back to the requesting HMD 1.

[0171] Upon receiving a selection result from another device that enables collaborative execution of content (S1509 - Yes), HMD 1 combines its own VR experience area with the VR experience areas of other devices (S1510). For example, as Figure 16As shown, a VR experience area P2 containing HMD 1B and a VR experience area P1 containing HMD 1A can be set up, and a VR experience area P31 can be synthesized in a maximized manner within the real space S. Alternatively, as shown in VR experience area P32, VR experience areas P1 and P2 can be synthesized by connecting them using a connecting area with a pre-defined width and shape. The connecting area is preferably set to a width and shape that allows user passage. Alternatively, as shown in VR experience area P33, VR experience areas P1 and P2 can be synthesized by connecting them with a vertex angle exceeding 90 degrees at the connection point. Alternatively, as shown in VR experience area P34, regardless of the position and shape of VR experience areas P1 and P2, they can be set as recommended areas that meet the requirements for multiplayer gameplay. For example, if the length of the two sides of the recommended area is 3m × 3m, VR experience area P34 is set to contain a 3m × 3m area. Furthermore, if the VR experience area is set to a complex shape, users will find it difficult to grasp its shape. Therefore, it is preferable to set the VR experience area to a polygon with no more than 4 vertices when compositing it. In addition, if there are obstacles such as televisions or tables, or other people besides users A and B, in the real space S, it is preferable to compose the VR experience area in a way that does not include their area.

[0172] In S1511, HMD 1 determines whether the synthesized VR experience area is larger than or equal to the size of the VR experience area required for multiplayer gameplay of the selected VR content. If the size of the synthesized VR experience area meets the VR experience requirements for multiplayer gameplay (S1511 - Yes), HMD 1 changes the VR experience area in a manner corresponding to the synthesized VR experience area (S1512). After changing the VR experience area, HMD 1 begins executing the content (S1513).

[0173] Additionally, in S1512, HMD 1 sends data about the synthesized VR experience area to other devices. Here, if the HMD 1 and other devices have different play modes, it can be processed to change them to the same play mode, or the user can be notified before changing to the same play mode. Alternatively, the different play modes can remain unchanged, allowing for multiplayer gameplay between HMD 1 and other devices for reasons related to real-world space or the user.

[0174] If the size of the synthesized VR experience area is determined to be smaller than the size required for a multi-player VR experience (S1511 - No), HMD 1 uses measurement information of the real space obtained by camera 10 or range sensor 11 to determine whether there is an area in the real space that can be corrected (S1521). For example, if HMD 1 determines that there is an area in the real space that is not used as a VR experience area and there is an area that can be used to expand the VR experience area (S1521 - Yes), it corrects and resets the VR experience area (S1522) and then starts executing the content (S1513). In addition, the owner of the content advancement in the VR experience area is the user who was previously playing the content, and the users who join later follow the user who is the owner. The owner's rights can be transferred from the user with rights to other users, or it can be determined using information such as priority, usage schedule, payment amount, age, and physique obtained from HMD 1A and HMD 1B. For example, users with high priority or high payment amount can be preferentially set as the owner in charge of content advancement.

[0175] In addition, although not in Figure 15 As shown, if the size of the synthesized VR experience area is determined to be smaller than the size of the VR experience area required for multiplayer games (S1511-No), the VR content's play mode can be automatically changed or the play mode can be changed based on user input to reduce the VR experience area required for multiplayer games. Additionally, although not in... Figure 15 As shown, if a user selects a synthesized VR experience area during the content progression, the VR experience area can be redefined and re-defined for each user. In this case, it is preferable to divide the synthesized VR experience area equally. Alternatively, when dividing the VR experience area, information such as priority, scheduled usage, payment amount, age, and physical condition obtained from HMD 1A and HMD 1B can be used to determine the division.

[0176] If S1505-No, S1506-No, S1507-No, S1509-No, or S1521-No, HMD 1 ends processing.

[0177] Through such Figure 15 The processing shown can be used to set up VR experience areas for multiplayer gameplay with other devices. Furthermore, the processing is not limited to that shown; it can also be combined and executed, for example. Figure 12 and Figure 15 Part of. For example, in Figure 12If, in S1207, coordination with other devices is selected, HMD 1 can execute the processing after S1505 instead of the processing after S1208. Furthermore, in S1212, if the judgment result sent from other devices indicates that they disagree with the change, HMD 1 can execute the processing after S1505 instead of returning to S1203. Additionally, in the cases of S1505-No, S1506-No, S1507-No, S1509-No, and S1521-No, the processing can continue without ending; instead, the processing after S1203 can be executed to coordinate with other devices to adjust the VR experience area.

[0178] As described above, according to the third embodiment, by combining the experience areas of itself and other devices, it is possible to set a suitable VR experience area when multiple users experience the same content.

[0179] Furthermore, in the above description, the VR experience area setting process is performed on an information display terminal such as an HMD, but it is not limited to this. Similar to the first and second embodiments, Figure 15 The processing can be performed, in part or in whole, by the server or content distribution device. For example, information about the conditions for experiencing content with other users can be sent from information display terminals such as HMD 1A and HMD 1B and pre-registered in the server or content distribution device. Then, when the server or content distribution device determines that HMD 1A and HMD 1B are located in the same real space, it can determine whether to synthesize a VR experience area without requiring the users of the information display terminals to make a selection or without notification, and control HMD 1A and HMD 1B to change the VR experience area based on the determination result, or send information for changing the area.

[0180] According to the description of the third embodiment, as an example, an information display terminal is provided. This information display terminal includes a display, a communication interface, an input interface used by a user of the information display terminal for input operations, a camera for capturing real-world space, and a processor. The processor communicates with other devices (located in the same real-world space as the information display terminal) via the communication interface, receives information indicating first content executed on the other devices, and information indicating a region set for experiencing the first content, i.e., a first experience region. When the user selects a region via the input interface, the processor uses the received information to overlay the first experience region with an image of the real-world space captured by the camera and displays it on the display. Additionally, the processor receives information indicating a region selected by the user via the input interface, i.e., a second experience region, and information indicating second content selected by the user, and compares the first content with the second content. Then, if the first content and the second content are the same, the processor determines whether it can cooperate with other devices to execute the content. If it determines that it can cooperate with other devices to execute the content, it sends a cooperation request to the other devices via the communication interface. Furthermore, an information processing method executed on the information display terminal and an information processing system having an information display terminal and a server or content distribution device are also provided.

[0181] The embodiments have been described above, but the present invention is not limited to the above embodiments, and includes various modifications and equivalent structures within the technical concept of the claimed technical solutions. For example, the above embodiments have been described in detail for ease of understanding, and the present invention is not limited to having all the described structures. In addition, for example, for a part of the structure of the embodiments, other structures may be added, deleted, or replaced. For example, when changing or composing the VR experience area, the experience area can be adjusted by gradually changing the experience area. In addition, the information display terminal can notify the user of the information display terminal through sound, display, etc. In addition, the information display terminal may include, for example, a vibrator for notification by vibration. In addition, the position information and posture (orientation) information of the information display terminal can be used in the adjustment of the experience area.

[0182] Explanation of reference numerals in the attached figures

[0183] 1 Head-mounted display, 10 Camera, 11 Ranging sensor, 12 Display, 13 Speaker, 14 Microphone, 15 Sensor group, 16 Control unit, 17 Internal bus, 20 Communication unit, 21 Processor, 22 Memory, 23 Storage device, 24 Basic operation program, 25 VR audiovisual program, 26 Proximity transmission program.

Claims

1. An information display terminal, characterized in that, include: monitor; Communication interface; and processor, The processor performs the following controls: The system communicates with other devices located in the same physical space as the information display terminal via the communication interface, and receives information indicating a first experience area defined for experiencing content executed on the other devices. Determine whether the area selected by the user of the information display terminal, i.e., the second experience area, overlaps with the first experience area. If it is determined that the second experience area does not overlap with the first experience area, the size of the experience requirement area needed to experience the content selected by the user is compared with the size of the second experience area. When the size of the second experience area is greater than or equal to the size of the experience requirement area, the second experience area is set as the area for experiencing the content selected by the user.

2. The information display terminal according to claim 1, characterized in that: The processor controls the display to show a message on the screen that prompts the user to select the content they wish to experience using the information display terminal if it determines that the second experience area does not overlap with the first experience area.

3. The information display terminal according to claim 2, characterized in that: The processor controls the display to show a message prompting the user to reselect an area larger than the second experience area or to reselect content different from the requested content when the size of the second experience area is smaller than the size of the experience demand area.

4. The information display terminal according to claim 3, characterized in that: The processor controls the display to prevent the user from selecting content that requires a larger area than the second experience area when a message prompting the user to reselect content different from the content is displayed on the display.

5. The information display terminal according to claim 1, characterized in that: It also includes a camera for capturing real-world images and an input interface for the user to perform input operations. The processor controls the display to overlay an image of the first experience area onto the screen using the received information, when the user selects an area via the input interface.

6. The information display terminal according to claim 1, characterized in that: It also includes memory, The memory stores setting information, which includes information indicating whether to automatically acquire information and information indicating the content of the information to be acquired. The processor sends information requests to the other devices via the communication interface based on the setting information stored in the memory.

7. An information display terminal, characterized in that, include: monitor; Communication interface; A camera used to film real-world spaces; and processor, The processor performs the following controls: The device communicates with other devices located in the same physical space as the information display terminal via the communication interface, and receives information indicating a first experience area, i.e., a region set up for experiencing first content executed on the other devices. When a user selects an area on the information display terminal, the received information is used to overlay the first experience area with the image of the real space captured by the camera and display it on the screen. Receive information indicating the area selected by the user, namely the second experience area, and the second content selected by the user. The size of the second experience area is compared with the size of the second experience demand area corresponding to the second content. If the size of the second experience area is smaller than the size of the second experience demand area, a message is displayed on the display prompting the user to choose whether to perform adjustments with the other devices.

8. The information display terminal according to claim 7, characterized in that: If the user selects to perform adjustments with the other device, the processor sends at least one of the following requests to the other device via the communication interface: a request to change the first experience area, a request to change the play mode of the first content, and a request to change the first content.

9. The information display terminal according to claim 7, characterized in that: The processor performs the following controls: If the user selects to perform adjustments with the other devices, the processor calculates the difference between the size of the second experience demand area and the size of the second experience area, which is the change area. If the size of the first experience area minus the changed area is greater than or equal to the size of the first experience requirement area corresponding to the first content, a change request for the first experience area is sent to the other device via the communication interface.

10. The information display terminal according to claim 7, characterized in that: The processor controls the receiving, via the communication interface, of information from the other device indicating at least one of a predetermined schedule, priority, payment amount, and age, and includes the received information in a message prompting the user to select whether to perform an adjustment with the other device and displays it on the display.

11. The information display terminal according to claim 7, characterized in that: It also includes an input interface for the user to perform input operations.

12. An information display terminal, characterized in that, include: monitor; Communication interface; A camera used to film real-world spaces; and processor, The processor performs the following controls: The system communicates with other devices located in the same physical space as the information display terminal via the communication interface, and receives information indicating first content being executed on the other devices and information indicating a first experience area set up for experiencing the first content. When a user selects an area on the information display terminal, the received information is used to overlay the first experience area with the image of the real space captured by the camera and display it on the screen. Receive information indicating the area selected by the user, namely the second experience area, and the second content selected by the user. Compare the first content with the second content. If the first content and the second content are the same, determine whether the content can be executed collaboratively with the other devices. If it is determined that the content can be executed collaboratively with the other device, a collaboration request is sent to the other device via the communication interface.

13. The information display terminal according to claim 12, characterized in that: Upon receiving a result indicating the ability to cooperate from the other device via the communication interface, the processor sets an experience area that is synthesized from the first experience area and the second experience area.

14. The information display terminal according to claim 12, characterized in that: The processor controls the comparison between the size of the experience requirement area corresponding to the content executed collaboratively by the information display terminal and the other devices and the size of the synthesized experience area. If the size of the synthesized experience area is greater than or equal to the size of the experience requirement area, the first content is executed.

15. The information display terminal according to claim 12, characterized in that: It also includes an input interface for the user to perform input operations.

Images