Information processing device, information processing method, and information processing program

Abstract

To reduce stress imposed on a user by XR content.SOLUTION: An information processing device relates to reproduction of XR content and comprises a switching control unit. The switching control unit switches a display for displaying XR content while taking over displayed content among a plurality of displays provided to be capable of displaying the XR content according to the state of sickness regarding sickness caused by the XR content.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The disclosed embodiments include an information processing device, method and Information Processing Program Regarding. [Background technology]

[0002] Conventionally, there is known a technology that provides users with digital content that includes virtual space experiences such as VR (Virtual Reality), AR (Augmented Reality), and MR (Mixed Reality), so-called XR (Cross Reality) content, using a head mounted display (HMD) etc. XR is a collective term that includes all virtual space technologies including VR, AR, MR, SR (Substitutional Reality), AV (Audio / Visual), etc.

[0003] In addition, in this technology, an XR system has also been proposed that is mounted on a moving body such as a vehicle and that can use the moving body as a motion platform (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2017-102401 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, conventional technologies have room for further improvement in reducing the stress that users experience when using XR content.

[0006] For example, it is known that users who receive XR content may experience "XR sickness," which is similar to motion sickness. XR sickness is a type of motion sickness that can occur due to a mismatch between the image and the user's body movements. In particular, in the case of an XR system installed in a vehicle, the movement of the vehicle is added to the user's body movements, making the mismatch with the image more complex and making XR sickness more likely to occur. Furthermore, since regular motion sickness symptoms are also added, more complicated symptoms of sickness are more likely to occur.

[0007] When XR sickness is likely to occur, the user may take actions such as removing the HMD in order to reduce the stress caused by XR sickness. However, in this case, although the user can reduce the stress caused by XR sickness, they will instead experience the stress of having their viewing of the XR content interrupted.

[0008] One aspect of the embodiment has been made in consideration of the above, and aims to provide an information processing device, an information processing system, and a method for playing XR content that can reduce the stress that a user experiences due to XR content. [Means for solving the problem]

[0009] According to an embodiment, the information processing device is an information processing device for XR content playback that has a controller and is mounted on a vehicle. The controller determines that the vehicle has reached a point just before entering the sickness area based on map data including information about a sickness area that is an area where sickness due to XR content is likely to occur, and in response to the vehicle reaching a point just before entering the sickness area, The display device used in the vehicle interior for displaying the above information can be switched from the HMD to a HUD, ceiling monitor, side window monitor, tablet, or smartphone while continuing to display the information. . Effect of the Invention

[0010] According to one aspect of the embodiment, the stress experienced by the user by XR content can be reduced. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of an information processing system according to an embodiment. [Diagram 2] FIG. 2 is an explanatory diagram of XR sickness. [Diagram 3] FIG. 3 is a diagram (part 1) outlining the information processing method according to the embodiment. [Figure 4] FIG. 4 is a schematic explanatory diagram (part 2) of the information processing method according to the embodiment. [Diagram 5] FIG. 5 is a schematic explanatory diagram (part 3) of the information processing method according to the embodiment. [Figure 6] FIG. 6 is a block diagram illustrating an example of the configuration of the information processing device according to the embodiment. [Figure 7] FIG. 7 is a block diagram showing an example of the configuration of the display unit 3. As shown in FIG. [Figure 8] FIG. 8 is an explanatory diagram (part 1) of the execution timing of the switching control by the switching control unit. [Figure 9] FIG. 9 is a diagram (part 2) illustrating the execution timing of the switching control by the switching control unit. [Figure 10] FIG. 10 is a diagram showing an example in which a ceiling monitor is used. [Figure 11] FIG. 11 is a diagram showing a modified example of the HMD. [Figure 12] FIG. 12 is an explanatory diagram (part 1) of playback control of XR content for multiple users U. [Figure 13] FIG. 13 is an explanatory diagram (part 2) of playback control of XR content for multiple users U. [Figure 14] FIG. 14 is an explanatory diagram (part 3) of playback control of XR content for multiple users U. [Figure 15] FIG. 15 is an explanatory diagram (part 4) of playback control of XR content for multiple users U. [Figure 16] FIG. 16 is a flowchart showing a processing procedure executed by the information processing device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, embodiments of an information processing device, an information processing system, and an XR content playback method disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments described below.

[0013] In the following description, multiple components having substantially the same functional configuration may be distinguished by adding different numbers with a hyphen after the same reference symbol. For example, multiple components having substantially the same functional configuration may be distinguished as necessary, such as information processing device 10-1 and information processing device 10-2. However, when there is no need to particularly distinguish between multiple components having substantially the same functional configuration, only the same reference symbol is used. For example, when there is no need to particularly distinguish between information processing device 10-1 and information processing device 10-2, they are simply referred to as information processing device 10.

[0014] In the following, the information processing system 1 according to the embodiment will be described as an XR system that is mounted on a vehicle and provides XR content to a user. In the following, the "XR content playback method" will be referred to as the "information processing method."

[0015] First, an overview of an information processing method according to an embodiment will be described with reference to Figs. 1 to 5. Fig. 1 is a diagram showing a schematic configuration of an information processing system 1 according to an embodiment. Fig. 2 is an explanatory diagram of XR sickness. Figs. 3 to 5 are explanatory diagrams (part 1) to (part 3) showing an overview of the information processing method according to the embodiment.

[0016] As shown in FIG. 1, an information processing system 1 according to the embodiment includes a display unit 3 and an information processing device 10.

[0017] The display unit 3 includes a plurality of display devices. As one of them, as shown in FIG. 1, the display unit 3 includes an HMD 31. The HMD 31 is an information processing terminal that presents XR content provided by the information processing device 10 to the user U and allows the user to enjoy an XR experience. The HMD 31 is a wearable computer that is worn on the head of the user U and is a goggle-type computer in the example of FIG. 1. The HMD 31 may be a glasses-type computer or a hat-type computer.

[0018] The HMD 31 includes a display 31a, a speaker 31b, and a sensor unit 31c. The display 31a is provided so as to be placed in front of the eyes of the user U, and displays images included in the XR content provided by the information processing device 10.

[0019] In the example of Fig. 1, the display 31a is provided in front of each of the left and right eyes of the user U, but there may be only one display. The display 31a may be a non-transmissive type that completely covers the field of view, or may be a video-transmissive type or an optical-transmissive type. In this embodiment, the display 31a is a non-transmissive type.

[0020] 1, the speaker 31b is provided in a headphone type, for example, and is worn on the ear of the user U. The speaker 31b outputs sound included in the XR content provided from the information processing device 10.

[0021] The sensor unit 31c is a device that detects changes in the inside and outside circumstances of the user U, and includes, for example, a camera, a motion sensor, and the like.

[0022] The information processing device 10 is a computer, for example, an in-vehicle device mounted on a vehicle, connected to the HMD 31 by wire or wirelessly, and provides XR content to the HMD 31. In addition, the information processing device 10 acquires changes in the situation detected by the sensor unit 31c at any time, and reflects such changes in the situation in the XR content.

[0023] For example, the information processing device 10 can change the direction of the field of view in the virtual space of the XR content in response to a change in the head or line of sight of the user U detected by the sensor unit 31c.

[0024] Incidentally, it is known that when XR content is provided using such an HMD 31, the user U may experience "XR sickness," which is similar to motion sickness.

[0025] As shown in Fig. 2, XR sickness can occur due to factors such as a mismatch between the image and the body movements of the user U. In addition, in the case of an XR system mounted on a vehicle, the body movements of the user U are further complicated by the movement of the vehicle due to changes in the surrounding environment, making XR sickness more likely to occur.

[0026] When such XR sickness is likely to occur, the user U may take an action such as removing the HMD in order to reduce the stress caused by the XR sickness. However, in this case, although the user U can reduce the stress caused by XR sickness, he or she will instead be stressed by the interruption of viewing the XR content.

[0027] Therefore, the information processing method of the embodiment is an information processing method executed by an information processing device 10 mounted on a vehicle V, which acquires the inside and outside conditions of the vehicle V, and switches the displays based on the acquired conditions while handing over the display content between multiple displays capable of displaying XR content.

[0028] Specifically, as shown in FIG. 3, in the information processing method according to the embodiment, the information processing device 10 acquires the inside and outside conditions of the vehicle V at any time, in other words, the conditions related to the user U who uses the XR content, and estimates the condition related to XR sickness (step S1). The condition related to XR sickness may be rephrased as a sickness condition related to sickness caused by the XR content. The information processing device 10 estimates the condition related to XR sickness based on, for example, a report from the user U. In addition, the information processing device 10 estimates the condition related to XR sickness by, for example, detecting a change in the physical condition of the user U.

[0029] In addition, the information processing device 10 estimates a state related to XR sickness based on the usage state of the XR content, such as the type of XR content being provided, the state of the video, and the state of the audio.

[0030] The information processing device 10 also estimates a situation related to XR sickness based on the driving conditions of the vehicle V, such as road conditions, the conditions of the vehicle V, and the operation conditions. The information processing device 10 also estimates a situation related to XR sickness based on user information including various parameters indicating the susceptibility of each user U, for example.

[0031] In addition, in the process of estimating the situation related to XR sickness, the information processing device 10 can use an estimation model generated using, for example, a machine learning algorithm. The estimation model is appropriately reinforced learning based on the actual estimation result. As a result of the reinforcement learning, for example, a judgment threshold for estimating the situation related to XR sickness, etc. is appropriately updated.

[0032] Then, the information processing device 10 executes playback control of the XR content so as to reduce XR sickness based on the situation related to XR sickness estimated in step S1 (step S2). As shown in the figure, the playback control of the XR content is, for example, display switching control (step S21).

[0033] As described above, the display unit 3 includes a plurality of display devices, one of which is the HMD 31. The display unit 3 further includes a display device other than the HMD 31. The display device other than the HMD 31 is, for example, a HUD (Head-Up Display) that projects image information onto the front window of the vehicle V.

[0034] The information processing device 10 executes a process of switching the display of the XR content between the HMD 31 and the HUD as a display switching control. At this time, the information processing device 10 also displays at least a part of the virtual object in the XR content between the HMD 31 and the HUD while handing it over.

[0035] Specific examples are shown in Figures 4 and 5. First, as shown in the upper part of Figure 4, it is assumed that the information processing device 10 displays AR content on the display 31a of the HMD 31. The example of Figure 4 shows an example in which virtual objects Vo1, Vo2, and Vo3 are displayed against a background of a photographed image of real space captured by an on-board camera of a vehicle V, for example.

[0036] As shown in the figure, the virtual object Vo1 is, for example, a character in the XR content. The virtual object Vo2 is, for example, a message regarding the progress of the XR content. The virtual object Vo3 is, for example, a guidance message for the user U who is watching the XR content.

[0037] As indicated by the virtual object Vo3, when the sickness level indicating the degree of XR sickness of the user U increases based on, for example, the estimation result in step S1 described above, the information processing device 10 causes the HMD 31 to display a guidance message encouraging the user to remove the HMD 31.

[0038] When the user U removes the HMD 31 in response to the guidance message, as shown in the lower part of the figure, the information processing device 10 causes the virtual objects Vo1 and Vo2 that had been displayed on the HMD 31 up until that point to be displayed on the HUD 32. Note that at this time, the information processing device 10 causes the virtual objects Vo1 and Vo2 to be displayed on the HUD 32 such that the positions and sizes of the virtual objects Vo1 and Vo2 as seen by the user U are carried over from the HMD 31 to the HUD 32 without change.

[0039] In addition, when the user U removes the HMD 31, the A-pillar in the vehicle cabin and the outside scenery through the front window come into his or her field of vision and can be recognized, as shown in the figure.

[0040] As a result, the information processing device 10 can reduce the stress that the user U experiences due to XR sickness by freeing the user U from the blocked field of view using the HMD 31. On the other hand, the information processing device 10 can reduce the stress that the user U experiences due to interruption in viewing of the XR content by allowing the user U to continue viewing the XR content even after removing the HMD 31.

[0041] In addition, in Fig. 4, an example in which AR content is displayed on the display 31a of the HMD 31 is given, but the same applies to XR content other than AR content. For example, as shown in the upper part of Fig. 5, the information processing device 10 displays VR content on the display 31a of the HMD 31. In the example of Fig. 5, the background is not a photographed image of a real space captured by an on-board camera of the vehicle V, but the background itself is represented by a group of virtual objects (here, mountains, trees, etc.).

[0042] Then, similarly to the example of FIG. 4, when the user U removes the HMD 31 in response to the guidance message of the virtual object Vo3, the information processing device 10 displays the virtual objects Vo1 and Vo2 that were displayed on the HMD 31 up until then on the HUD 32 while maintaining their positions and sizes, as shown in the lower part of FIG. 5. At this time, the information processing device 10 does not display the background of the virtual objects on the HUD 32. That is, as shown in the figure, the information processing device 10 basically allows the user U to view the real background. Note that instead of not displaying the background using the virtual objects at all, the information processing device 10 may display the background using the virtual objects with increased transparency.

[0043] 4, the information processing device 10 can reduce the stress that the user U experiences due to XR sickness by releasing the user U from the blocked field of view using the HMD 31. On the other hand, the information processing device 10 can reduce the stress that the user U experiences due to the interruption of viewing of the XR content by allowing the user U to continue viewing the XR content even after removing the HMD 31.

[0044] Thus, the information processing method of the embodiment is an information processing method executed by an information processing device 10 mounted on a vehicle V, which acquires the inside and outside conditions of the vehicle V, and switches the displays based on the acquired conditions while handing over the display content between multiple displays capable of displaying XR content.

[0045] Therefore, according to the information processing method according to the embodiment, it is possible to reduce the stress that the user U experiences due to the XR content. Hereinafter, a configuration example of the information processing system 1 to which the information processing method according to the embodiment is applied will be described in more detail.

[0046] Fig. 6 is a block diagram showing a configuration example of an information processing device 10 according to an embodiment. Fig. 7 is a block diagram showing a configuration example of a display unit 3. Note that Figs. 6 and 7 show only components necessary for explaining the features of the embodiment, and descriptions of general components are omitted.

[0047] In other words, each component shown in Figures 6 and 7 is a functional concept, and does not necessarily have to be physically configured as shown in the figures. For example, the specific form of distribution and integration of each block is not limited to that shown in the figures, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0048] In addition, in the description using FIG. 6 and FIG. 7, the description of components that have already been described may be simplified or omitted.

[0049] 6, an information processing device 10 according to an embodiment includes a storage unit 11 and a control unit 12. The information processing device 10 is also connected to a display unit 3, a sensor unit 5, and a seat mechanism 7 via a wired network such as a Controller Area Network (CAN), wirelessly, or directly.

[0050] As described above, the display unit 3 includes a plurality of displays. As shown in Fig. 7, the display unit 3 includes, for example, an HMD 31, a HUD 32, a ceiling monitor 33, a side window monitor 34, a tablet 35, a smartphone 36, and the like.

[0051] The ceiling monitor 33 is a monitor device formed on the ceiling of the vehicle cabin. Similarly, the side window monitor 34 is a monitor device formed on a side window of the vehicle cabin. Note that, like the HUD 32, the ceiling monitor 33 and the side window monitor 34 may be devices onto which image information is projected by, for example, a projector provided separately. The tablet 35 and the smartphone 36 are examples of mobile terminal devices carried by the user U.

[0052] Returning to the description of Fig. 6, the sensor unit 5 is a group of various sensors mounted on the vehicle V, and includes, for example, a microphone, a camera, an acceleration sensor, a steering angle sensor, a vital sensor, and the like.

[0053] The microphone collects sounds emitted within the vehicle cabin, such as the speech of the user U. The cameras include a front camera, a rear camera, a side camera, an interior camera, and the like, mounted on the vehicle V, and capture images of the inside and outside of the vehicle V. The interior camera captures images of the user U's condition, for example.

[0054] The acceleration sensor measures the acceleration and vehicle speed applied to the vehicle V. The steering angle sensor measures the steering angle of the vehicle V.

[0055] The vital sensor is a sensor that detects the physical condition of the user U, and is, for example, worn by the user U to measure vital data such as the heart rate, brain waves, blood oxygen concentration, and sweat of the user U. Of course, the sensor unit 5 may appropriately include various sensors other than those described above. The seat mechanism 7 is an adjustment mechanism for the reclining seat of the vehicle V.

[0056] The storage unit 11 is realized, for example, by a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. In the example of Fig. 6, the storage unit 11 stores an XR content DB (Database) 11a, user information 11b, an estimation model 11c, and a map DB 11d.

[0057] The XR content DB 11a is a database that stores a group of XR contents to be displayed on the display unit 3. The user information 11b is information about a user U who uses the display unit 3, and includes various parameters that indicate the susceptibility to sickness for each user U described above. The user information 11b is appropriately updated based on the estimation results of the user U's past conditions related to XR sickness.

[0058] The estimation model 11c is an estimation model generated using the above-mentioned machine learning algorithm. For example, the estimation model 11c calculates and outputs a value indicating the situation related to XR sickness of the user U (for example, a "sickness level" that is a level value indicating the degree of XR sickness) by inputting data indicating various situations inside and outside the vehicle V acquired by the acquisition unit 12b described later.

[0059] The map DB 11d is a database in which map data is stored. The map data may include not only map information, but also information on the material and undulations of the road, for example. This makes it possible to calculate in detail the road conditions during driving, for example, when estimating the situation related to XR sickness.

[0060] The map DB 11d also includes information on sickness areas where the user U is likely to get sick. The sickness areas are appropriately updated based on the estimation results of the past XR sickness-related conditions of the user U. Note that a server device (not shown) or the like may be used to collect sickness areas from big data of the estimation results of XR sickness of a large number of users U at any time, and the map DB 11d may be appropriately updated based on the collected results.

[0061] The control unit 12 is a controller, and is realized, for example, by a CPU (Central Processing Unit), MPU (Micro Processing Unit), etc., executing various programs (not shown) stored in the storage unit 11 using a RAM as a working area. The control unit 12 can also be realized, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0062] The control unit 12 has a reproduction control unit 12a, an acquisition unit 12b, an estimation unit 12c, a switching control unit 12d, and a seat control unit 12e, and realizes or executes the functions and actions of information processing described below.

[0063] The playback control unit 12a executes playback control for displaying and playing back the XR content stored in the XR content DB 11a on the display unit 3. The playback control unit 12a also reflects changes in various situations detected by the sensor unit 31c and sensor unit 5 of the HMD 31 described above and acquired by the acquisition unit 12b described later, in the playback state of the XR content.

[0064] The acquisition unit 12b acquires sensing data from the sensor unit 31c of the HMD 31 and the sensor unit 5 at any time. The acquisition unit 12b also acquires the usage status of the XR content, such as the type of the XR content being played, the video status, and the audio status, from the playback control unit 12a at any time. The acquisition unit 12b also outputs the various acquired data to the estimation unit 12c.

[0065] The estimation unit 12c estimates the state of XR sickness of the user U using the estimation model 11c based on various data acquired by the acquisition unit 12b. The estimation unit 12c performs voice recognition processing and natural language processing on the contents of the utterance of the user U, and estimates, based on the results, for example, whether or not the user U reports that he or she has experienced XR sickness, and if so, the level of sickness, etc., as described above.

[0066] The switching control unit 12d executes the above-mentioned switching control of the display device based on the estimation result of the estimation unit 12c, and reflects the result in the playback control of the XR content by the playback control unit 12a. Here, Fig. 8 and Fig. 9 are explanatory diagrams (part 1) and (part 2) of the execution timing of the switching control by the switching control unit 12d. Also, Fig. 10 is a diagram showing an example in which the ceiling monitor 33 is used.

[0067] As shown in FIG. 8, the switching control unit 12d executes switching control when the sickness level of the user U calculated by the estimation unit 12c exceeds a predetermined first threshold, that is, at time T1 in FIG.

[0068] 9, when a sickness area is present on the route of the vehicle V, the switching control unit 12d executes switching control of the display near point A just before entering the sickness area. That is, the switching control unit 12d executes switching control based on an estimation of sickness using map data or the like. That is, the information processing device 10 includes a future prediction in the sickness level of the user U and switches the display so as not to make the user U sick or, if he / she does get sick, to make his / her condition as bad as possible.

[0069] At this time, the switching control unit 12d causes the HMD 31 to display, via the reproduction control unit 12a, a guidance message prompting the user to remove the HMD 31 on the display 31a of the HMD 31, as shown as a virtual object Vo3 in FIG. 4 and FIG.

[0070] In addition, the switching control unit 12d causes the playback control unit 12a to display, for example, the virtual objects Vo1 and Vo2 shown in FIG. 4 and FIG.

[0071] In addition, after switching the display, when the sickness level of the user U falls below the first threshold, that is, at time T2 in FIG. 8, the switching control unit 12d returns (restores) the switched display. Alternatively, when the sickness level falls below a second threshold lower than the first threshold, the switching control unit 12d returns the switched display to its original state. That is, in this case, the switching control unit 12d restores the display at time T3 as the restoration timing. In this way, by setting the threshold at the time of restoration lower, hysteresis at the time of restoration can be taken into consideration. In addition, when multiple thresholds are provided, the display may be switched stepwise according to the sickness level, for example, from display A to display B to display C, and at the time of restoration, each display may be restored stepwise according to its suitability for the sickness level, for example, from display C to display B to display A.

[0072] Alternatively, after switching the display, the switching control unit 12d restores the switched display when the vehicle V passes through the sickness area, that is, near point B in Fig. 9. Even when restoring the display, the switching control unit 12d causes the playback control unit 12a to display the display on the HMD 31 while transferring the display content between the HUD 32 and the HMD 31, for example.

[0073] Of course, the switching control unit 12d can switch between any of the displays included in the display unit 3, not just between the HMD 31 to HUD 32 described above, so as to reduce the stress that the user U experiences due to XR sickness.

[0074] For example, as shown in Fig. 10, the virtual objects Vo1 and Vo2 displayed on the HMD 31 may be handed over not only to the HUD 32 but also to the ceiling monitor 33. Also, they may be handed over as appropriate to the above-mentioned side window monitor 34, tablet 35, smartphone 36, or other devices such as a rear window monitor or pillar monitor. This makes it possible to reproduce a virtual space that can be expressed in all directions of 360 degrees on the display 31a of the HMD 31, for example, in the vehicle interior space. Also, when switching, it is preferable to overlap the display between the display before and after switching for a predetermined time to avoid any gaps.

[0075] Returning to the description of Fig. 6, the seat control unit 12e controls the seat mechanism 7 based on the estimation result of the estimation unit 12c. For example, when the sickness level of the user U is high or when the user U is entering the sickness area, the seat control unit 12e reclines the seat to make the user U take a relaxed posture. This makes it possible to reduce the stress that the user U receives due to XR sickness by changing the posture of the user U.

[0076] When coordinating display using the ceiling monitor 33, the seat control unit 12e may recline the seat so that the user U is in a position that is easy for the user to view.

[0077] Next, some modified examples of the embodiments described above will be described with reference to Figures 11 to 15. Note that Figures 11 to 14 show only schematic views of the vehicle interior space as viewed from above.

[0078] Fig. 11 is a diagram showing a modified example of the HMD 31. As shown in the upper part of Fig. 11, the HMD 31 may be configured as a screen device 31A on which an image is simply projected from a projector P provided, for example, near a headrest of a front seat. This makes it possible to display XR content toward the ceiling, for example, by simply moving the head of a user U-2 in the figure, without using a ceiling monitor 33 or the like.

[0079] Also, as shown in the lower part of the same figure, for example, if the sickness level of user U-2 increases, user U-2 can be asked to remove the screen device 31A, and projector P can continue to project the image onto the front window or other window.

[0080] At this time, the location to be projected can be selected according to the sickness level of the user U-2, for example. For example, the greater the sickness level of the user U-2, the more effectively the XR sickness is reduced if the image is projected at a location farther from the user U-2.

[0081] 12 to 15 are explanatory diagrams (part 1) to (part 4) of the playback control of XR content for multiple users U. In the explanation using Figs. 12 to 15, it is assumed that the vehicle V is, for example, a fully autonomous vehicle, and some or all of the occupants can be provided with the same XR content at the same time.

[0082] 12, for example, all of users U-1 to U-4 are provided with the same XR content on a display D provided on the front window. In this case, the estimation unit 12c estimates the state of XR sickness for each of the users U at any time.

[0083] Then, the switching control unit 12d and the playback control unit 12a receive the estimation result of the estimation unit 12c, and perform playback control of the XR content so as to reduce XR sickness according to the user U with the highest sickness level, as shown in FIG. 12.

[0084] For example, in the example of Fig. 12, the playback control unit 12a controls the playback of the XR content so that the stimulation received by the user U-4 from the XR content is reduced according to the sickness level of the user U-2. For example, the playback control unit 12a controls the playback of the XR content so as to reduce the playback speed, brightness, color tone, and the like of the XR content.

[0085] When controlling the playback of XR content to reduce XR sickness, for example, the weather, the brightness of the surroundings, etc. may be taken into consideration. In addition, for example, the control may be linked to the operation of the headlights of the vehicle V.

[0086] 13, for example, all users U wear the HMD 31 and are provided with the same XR content. In this case, the switching control unit 12d performs switching control for user U-4, who has the highest sickness level, to transfer the display of the XR content from the above-mentioned HMD 31 to the HUD 32 or the like in the example of FIG.

[0087] 14, for example, the display D is a dual view type that can display different contents depending on the direction on one screen. In this case, in the example of the figure, the playback control unit 12a controls the playback of the XR content so as to reduce stimulation of the image displayed to user U-4, who has a higher sickness level, compared to the image displayed to user U-1, even if the XR content is the same.

[0088] 15, for example, when a user U receives the same XR content on different displays, one display may be divided into two to display two of the same XR content. In such a case, as shown in the figure, the playback of the XR content may be controlled so that one of the two displays provides less stimulation than the other, and when the user U's sickness level increases, the user U may be prompted to view the display with the reduced stimulation.

[0089] Although not shown in the figures, the following modified examples are also possible. For example, in the explanation using Figures 8 and 9, the case where the display switching control is automatically executed according to the sickness level and sickness area has been given as an example, but the display switching control may be executed automatically when the user U removes the HMD 31. Also, a hardware switch may be provided and the display switching control may be executed manually.

[0090] Also, when the display is switched, both displays may display images before and after the switch. That is, as already mentioned, it is preferable that the display before and after the switch overlap for a predetermined time so that no gap occurs when switching. Also, it is preferable that the display that displayed the image before the switch is turned off after the switch. In this case, for example, it is preferable that the display of the HMD 31 is stopped after the user U removes the HMD 31. The timing of removing the HMD 31 can be detected, for example, by sensing the illuminance inside the HMD 31 with a sensor provided in the HMD 31.

[0091] Next, a processing procedure executed by the information processing device 10 according to the embodiment will be described with reference to Fig. 16. Fig. 16 is a flowchart showing a processing procedure executed by the information processing device 10 according to the embodiment. Note that the processing procedure shown in Fig. 16 is repeated as needed while the XR content is being provided.

[0092] 16, first, the acquisition unit 12b acquires the inside and outside conditions of the vehicle V (step S101). Then, the estimation unit 12c estimates a condition related to XR sickness based on the acquired conditions (step S102).

[0093] Then, the switching control unit 12d executes switching control of the display so as to reduce XR sickness based on the estimation result estimated by the estimation unit 12c (step S103). Then, the process ends.

[0094] As described above, the information processing device 10 according to the embodiment is an information processing device for XR content playback, and includes the switching control unit 12d. The switching control unit 12d switches the display device that displays the XR content while taking over the display content between multiple display devices that are provided to be able to display the XR content, depending on the sickness level related to sickness caused by the XR content.

[0095] Therefore, according to the information processing device 10 according to the embodiment, it is possible to reduce the stress that the user experiences due to XR content.

[0096] The information processing device 10 further includes an acquisition unit 12b and an estimation unit 12c. The acquisition unit 12b acquires a situation related to a user U who uses XR content. The estimation unit 12c estimates the sickness state based on the situation acquired by the acquisition unit 12b. The switching control unit 12d switches the display device according to the estimation result of the estimation unit 12c.

[0097] Therefore, according to the information processing device 10 according to the embodiment, it is possible to reduce the stress experienced by the user U due to a situation related to XR sickness.

[0098] In addition, the switching control unit 12d switches the display when the sickness level of the user U estimated by the estimation unit 12c exceeds a first threshold, and after switching the display, returns the switched display to the original when the sickness level falls below a second threshold that is equal to or lower than the first threshold.

[0099] Therefore, according to the information processing device 10 according to the embodiment, it is possible to reduce the stress that the user U experiences due to the onset of XR sickness. Furthermore, when the XR sickness is resolved, it is possible for the user U to continue enjoying the XR content as before.

[0100] In addition, the switching control unit 12d switches the display when the estimation unit 12c estimates that the user U will enter a sickness area (corresponding to an example of an "area where sickness due to XR content is likely to occur"), and returns the switched display to its original state when the user U passes through the sickness area.

[0101] Therefore, according to the information processing device 10 of the embodiment, by switching the display depending on the situation in which XR sickness is predicted to occur, the stress that the user U will likely experience can be reduced, and after the situation is resolved, the user U can continue to enjoy the XR content as before.

[0102] Furthermore, when switching the display device, if the user U is wearing the HMD 31 as a display device, the switching control unit 12d causes the HMD 31 to display a guidance message encouraging the user to remove the HMD 31.

[0103] Therefore, according to the information processing device 10 according to the embodiment, the user U can follow the guidance to switch the display at an appropriate timing and continue enjoying the XR content.

[0104] In the above-described embodiment, an example was given of a configuration in which each display device and the information processing device 10 are separate, but this is not limited to this, and the configuration may also be such that at least one of the display devices and the information processing device 10 are integrated.

[0105] In addition, in the above-described embodiment, the HMD 31 is given as an example of one of the presentation devices that presents the XR content provided to the user U from the information processing device 10, but the presentation device is not limited to this and may include, for example, a bone conduction speaker or a vibration presentation device that presents vibrations such as bodysonics.

[0106] In the above-described embodiment, the information processing device 10 holds the XR content, but the XR content may be acquired from an external device as necessary. For example, after enjoying the XR content at a dedicated entertainment facility that provides the XR content, if authentication is received at the entertainment facility, the same XR content may be acquired and played back on the same day on the vehicle V. In such a case, the information processing device 10 acquires the corresponding XR content from the device at the entertainment facility via the network during that day.

[0107] Further advantages and modifications may readily occur to those skilled in the art. Thus, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and equivalents thereof. [Explanation of symbols]

[0108] 1. Information Processing Systems 3 Display section 5 Sensor section 7 Seat mechanism 10. Information processing device 11 Storage section 11a XR Content DB 11b User Information 11c Estimation model 11d Map DB 12 Control section 12a Playback control section 12b Acquisition part 12c Estimation part 12d Switching control section 12e Seat control unit 31 HMD 32 HUD 33 Ceiling Monitor 34 Side window monitor 35 tablets 36 Smartphone

Claims

1. An information processing device for XR content playback that has a controller and is mounted on a vehicle, The controller: Based on map data including information on a sickness area, which is an area where sickness due to XR content is likely to occur, it is determined that the vehicle has reached a point just before entering the sickness area; In response to the vehicle reaching a point just before entering the sickness area, a display device used in the vehicle cabin for displaying XR content is switched from an HMD to a HUD, a ceiling monitor, a side window monitor, a tablet, or a smartphone while continuing to display the content. Information processing device.

2. An information processing device for XR content playback that has a controller and is mounted on a vehicle, The controller: Based on map data including information on a sickness area, which is an area where sickness due to XR content is likely to occur, it is determined that the vehicle has reached a point just before entering the sickness area; In response to the vehicle reaching a point just before entering the sickness area, a display device used in the vehicle cabin for displaying XR content is switched from an HMD to an HUD while continuing to display the content. Information processing device.

3. An information processing method executed by the controller of the information processing device described in claim 1 or 2.

4. An information processing program executed by the controller of the information processing device described in claim 1 or 2.

Images