Information processing system, information processing method, display device, and computer program

Abstract

[Problem] To provide a technical improvement for solving or mitigating at least some of the problems of the prior art. [Solution] An information processing system according to the present invention comprises one or a plurality of computer processors. The one or plurality of computer processors are characterized by comprising: a transmission unit that transmits, to a prescribed display device, information for displaying a virtual space provided with a display region in which specific content can be reproduced; a reception unit that receives behavior information about a user during reproduction of first content in the display region transmitted from the prescribed display device; and a determination unit that determines the state of the user on the basis of the behavior information about the user received by the reception unit.

Description

Information processing system, information processing method, display device, and computer program

[0001] The present invention relates to an information processing system, an information processing method, and a display device.

[0002] In recent years, there has been active development of content that can be displayed by a head-mounted display device (HMD).

[0003] Furthermore, virtual reality (VR) technology is also being considered for use in the fields of medical care and nursing care. As an example, Patent Document 1 discloses a rehabilitation support system using an HMD.

[0004] JP 2023-058516 A

[0005] However, the content disclosed in the above document is intended to improve symptoms of a specific disease that has already been diagnosed, and has not been provided as content for general users.

[0006] In recent years, the development of technologies to detect pre-symptomatic and asymptomatic diseases, and technologies to prevent these detected pre-symptomatic and symptomatic conditions from becoming serious, has become a challenge.

[0007] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide technical improvements that solve or alleviate at least some of the problems of the prior art mentioned above.

[0008] A more specific object of the present disclosure is to provide an information processing system, an information processing method, a display device, and a computer program that are capable of appropriately determining the state of a user.

[0009] The information processing system of the present invention is an information processing system comprising one or more computer processors, each of which comprises a transmitting unit that transmits information to a specified display device for displaying a virtual space having a display area in which specific content can be played back, a receiving unit that receives behavior information of a user playing first content in the display area, transmitted from the specified display device, and a determining unit that determines the state of the user based on the user behavior information received by the receiving unit.

[0010] The user behavior information may be gaze information relating to the user's gaze, operation information relating to the user's operation, and / or speech information relating to the user's speech.

[0011] The state of the user may be a state relating to the function of the user's eyes, brain and / or mind.

[0012] The first content may be game content or video content for obtaining information for determining the state of the user.

[0013] The one or more computer processors may further include a gaze determination unit that determines whether the user's gaze position is within a first area set in the center of the display area based on gaze information regarding the user's gaze, which is user behavior information received by the receiving unit.

[0014] The one or more computer processors may further include an association unit that associates a first reward with the user information of the user when the gaze determination unit determines that the gaze position is within the first area.

[0015] The one or more computer processors may further include a playback unit that plays a specified sound on a specified display device when the gaze determination unit determines that the gaze position is in a second area outside the first area.

[0016] The one or more computer processors may further include a display processing unit that displays in the display area a screen including a first object displayed within a first area set in the center of the display area, a second object different from the first object that moves at least within the first area, and a third object different from the first object and the second object that is displayed for a predetermined period of time in a second area outside the first area.

[0017] The virtual space can be provided with a first space having a display area in which specific content can be played back.

[0018] The one or more computer processors may further include a generating unit that generates information to be provided to the user based on the result of the determining unit.

[0019] The provided information can be information guiding to a service that provides information about the user's status in the virtual space.

[0020] The provided information may be guidance information for second content for maintaining or improving the user's condition in the virtual space.

[0021] The second content may be game content or video content dedicated to maintaining or improving the user's condition.

[0022] The first content may be game content or video content dedicated to obtaining information for determining the user's state.

[0023] The one or more computer processors can change the display position of the display area in response to a movement of the user's head while the first content is being played.

[0024] The one or more computer processors can change the display position of the display area in response to a movement of the user's line of sight while the first content is being played back.

[0025] A character object controlled by one or more computer processors is displayed in the virtual space, and the character object can act in accordance with user behavior information.

[0026] The receiving unit further receives biometric information of the user, and the determining unit can determine the state of the user based on the behavior information of the user and the biometric information of the user received by the receiving unit.

[0027] The one or more computer processors may further include a notification unit that notifies the result of the determination by the determination unit to user terminals of other users associated with the user.

[0028] The one or more computer processors further include a generation unit that generates provision information to be provided to the user based on the results of the determination unit, the provision information being guidance information to services that provide information regarding the user's status in the virtual space, and the notification unit can further notify the user of the provision status of the service based on the guidance information.

[0029] The one or more computer processors may allow access to the virtual space from the user terminals of other users associated with the user, enabling conversation between the user and other users within the virtual space and / or display of the user's avatar and the avatars of other users.

[0030] The service that provides information about the user's condition in the virtual space may be a professional consultation service, a food delivery service and / or an insurance service.

[0031] The information processing method of the present disclosure is characterized in that it has one or more computer processors execute a transmitting step of transmitting information to a specified display device for displaying a virtual space having a display area in which specific content can be played, a receiving step of receiving behavior information of a user playing a first content in the display area, transmitted from the specified display device, and a determining step of determining the state of the user based on the user behavior information received in the receiving step.

[0032] The display device of the present disclosure is characterized by comprising a display processing unit that causes a display unit to display a virtual space having a display area in which specific content can be played, an acquisition unit that acquires behavior information of a user while a first content is being played in the display area, and a judgment unit that judges the user's state based on the user behavior information acquired by the acquisition unit.

[0033] The computer program of the present disclosure is characterized by having one or more computer processors realize a display processing function that causes a virtual space having a display area in which specific content can be played on a display unit, an acquisition function that acquires behavior information of a user while a first content is being played in the display area, and a judgment function that judges the user's state based on the user behavior information acquired by the acquisition function.

[0034] The present invention provides a technical improvement that solves or alleviates at least some of the problems of the prior art discussed above.

[0035] Specifically, according to the present invention, it is possible to provide an information processing system, an information processing method, a display device, and a computer program that are capable of appropriately determining the state of a user.

[0036] Furthermore, according to the present invention, it is possible to appropriately determine the state of the user depending on the particularities of the content and / or the determination environment.

[0037] Furthermore, according to one embodiment of the present invention, it is possible to provide the user with an appropriate suggestion based on the result of the above determination.

[0038] Furthermore, according to one embodiment of the present invention, it is possible to provide a technology for detecting pre-symptomatic conditions and diseases without subjective symptoms, and a technology for preventing these detected pre-symptomatic conditions and diseases from becoming serious.

[0039] FIG. 1 is a system configuration diagram showing an example of a system configuration of an information processing system according to the present disclosure. FIG. 2 is a system configuration diagram showing another example of a system configuration of an information processing system according to the present disclosure. FIG. 3 is a configuration diagram showing an example of a hardware configuration of a server device according to the present disclosure. FIG. 4 is a configuration diagram showing an example of a functional configuration of an information processing system according to the present disclosure. FIG. 5 is a conceptual diagram showing an example of a screen displayed on a display device according to the present disclosure. FIG. 6 is a conceptual diagram showing another example of the functional configuration of an information processing system according to the present disclosure. FIG. 7 is a conceptual diagram showing an example of a screen displayed on a display device according to the present disclosure. FIG. 8 is a conceptual diagram showing an example of a screen displayed on a display device according to the present disclosure. FIG. 9 is a flow diagram showing an example of an information processing method in an information processing system according to the present disclosure. FIG. 10 is a circuit configuration diagram showing an example of a circuit configuration for realizing a computer program executed in an information processing system according to the present disclosure. FIG. 11 is a functional configuration diagram showing an example of a functional configuration of a display device according to the present disclosure. FIG. 12 is a flow diagram showing an example of an information processing method in a display device according to the present disclosure. FIG. 13 is a circuit configuration diagram showing an example of a circuit configuration for realizing a computer program executed in a display device according to the present disclosure. FIG. 14 is an image diagram showing a light source position and partitioned areas used in evaluating the visual field of a user's right eye.

[0040] An embodiment of an information processing system according to the present disclosure will be described with reference to the drawings.

[0041] <System Configuration> As shown as an example in Fig. 1 , an information processing system 1000 according to the present disclosure may include a server device 100 connected to a display device 200 via a network. As shown in Fig. 2 , the information processing system 1000 may also include the display device 200 in addition to the server device 100.

[0042] The display device 200 may be a television device, a smartphone (multifunction telephone terminal), a tablet terminal, a personal computer, a console game machine, a head-mounted display (HMD), a wearable computer such as a glasses-type wearable terminal (AR glasses, etc.), or an information processing device other than these devices that can play videos. Furthermore, these terminals may be standalone devices that operate independently, or may be composed of multiple devices connected to each other so that various data can be transmitted and received.

[0043] <Hardware Configuration> Here, the hardware configuration of the server device 100 will be described with reference to Fig. 3. The server device 100 includes a processor 101, a memory 102, a storage 103, an input / output interface (input / output I / F) 104, and a communication interface (communication I / F) 105. The components are connected to each other via a bus B.

[0044] The server device 100 can realize the functions and methods described in this embodiment through cooperation between the processor 101, memory 102, storage 103, input / output I / F 104, and communication I / F 105.

[0045] The processor 101 executes functions and / or methods implemented by codes or instructions included in a program stored in the storage 103. The processor 101 may include, for example, a central processing unit (CPU), a microprocessing unit (MPU), a graphics processing unit (GPU), a microprocessor, a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc., and may implement each process disclosed in each embodiment using a logic circuit (hardware) or a dedicated circuit formed in an integrated circuit (an integrated circuit (IC) chip, a large-scale integration (LSI)), etc. Furthermore, these circuits may be implemented using one or more integrated circuits, and multiple processes described in each embodiment may be implemented using a single integrated circuit. Furthermore, LSIs may also be referred to as VLSIs, super LSIs, ultra LSIs, etc. depending on the degree of integration.

[0046] The memory 102 temporarily stores programs loaded from the storage 103 and provides a working area for the processor 101. The memory 102 also temporarily stores various data generated while the processor 101 is executing the programs. The memory 102 includes, for example, a random access memory (RAM) and a read-only memory (ROM).

[0047] The storage 103 stores programs and includes, for example, a hard disk drive (HDD), a solid state drive (SSD), a flash memory, and the like.

[0048] The communication I / F 105 is implemented as hardware such as a network adapter, communication software, or a combination thereof, and transmits and receives various data via a network. The communication may be performed either wired or wirelessly, and any communication protocol may be used as long as mutual communication is possible. The communication I / F 105 communicates with other devices via the network. The communication I / F 105 transmits various data to other devices in accordance with instructions from the processor 101. The communication I / F 105 also receives various data transmitted from other devices and transmits it to the processor 101.

[0049] The input / output I / F 104 includes an input device for inputting various operations to the server device 100, and an output device for outputting processing results processed by the server device 100. The input / output I / F 104 may be an integrated input device and output device, or may be separate input device and output device.

[0050] The input device is realized by any one or combination of all types of devices that can accept input from a user and transmit information related to the input to the processor 101. Examples of the input device include hardware keys such as a touch panel, a touch display, and a keyboard, a pointing device such as a mouse, a camera (for inputting operations via images), and a microphone (for inputting operations via voice).

[0051] Moreover, the display device 200 according to the present disclosure can also be configured with the same hardware configuration as that shown in FIG. 3, unless otherwise noted.

[0052] Next, various functions that can be executed by one or more computer processors included in an information processing system 1000 according to an embodiment of the present disclosure will be described with reference to the drawings.

[0053] As shown in FIG. 1 , an information processing system 1000 according to an embodiment of the present disclosure includes at least a server device 100 .

[0054] The one or more computer processors included in the information processing system 1000 include a transmitting unit 110, a receiving unit 120, and a determining unit 130, as shown as an example in FIG.

[0055] As an example, the transmitting unit 110, the receiving unit 120, and the determining unit 130 will be described as being included in the server device 100, but some or all of them may be included in the display device 200.

[0056] The transmitting unit 110 transmits information for displaying the virtual space to the display device 200 .

[0057] As an example, the display device 200 may be a wearable display device. Examples of wearable display devices include a head mounted display (HMD) and a glasses-type display.

[0058] The display device 200 can include at least a display unit, a processing unit, and a communication unit.

[0059] The display device 200 may also include an acceleration sensor and an attitude sensor for detecting the movement of the user's head.

[0060] The display device 200 may also include sensors for detecting the movement of the user's line of sight, the state of the pupils, and the user's facial expression. That is, the display device 200 may have an eye tracking function and a facial tracking function.

[0061] The display device 200 can be connected to a predetermined controller that can be operated by a user via a wired or wireless connection.

[0062] The virtual space has a display area in which specific content can be played back.

[0063] As an example, the virtual space may include at least a first space, which may include a display area in which the specific content can be played back.

[0064] 5 is an image diagram showing an example of the virtual space 10 displayed on the display unit of the display device 200. As an example, the first space 11 is a space representing the inside of a spaceship, and the display area 12 is represented as a window of the spaceship. Then, it is assumed that the user can view outer space, which is a virtual space, or specific content from inside the spaceship through the window of the spaceship.

[0065] The specific content may be, for example, game content or video content, but is not particularly limited as long as it includes the first content described below.

[0066] The specific content can be set to start playing in response to a user's operation on an operation panel object 13 arranged inside the spaceship.

[0067] Alternatively, the playback of a particular piece of content may be triggered by the user taking an action to sit on a chair object 14 placed inside the spaceship.

[0068] As an example, as shown in FIG. 6, the display area 12 during playback of a particular content can be made to cover substantially the entire field of view of the user.

[0069] Furthermore, the reproduction of a specific content can be stopped or ended in response to a user's operation on an operation UI object 15 that is displayed during the reproduction of the specific content.

[0070] Alternatively, the playback of a particular piece of content may be stopped or ended when the user stands up from a chair object placed inside the spaceship.

[0071] The receiving unit 120 receives behavior information of the user during playback of the first content, which is transmitted from the display device 200 .

[0072] The first content is one of the specific contents.

[0073] Specifically, the first content may be game content or video content dedicated to obtaining information for determining the user's state, as will be described in detail later.

[0074] The user behavior information may be, for example, gaze information relating to the user's gaze, operation information relating to the user's operation, and / or speech information relating to the user's speech, as will be described in detail later.

[0075] The determining unit 130 determines the state of the user based on the behavior information of the user received by the receiving unit 120 .

[0076] The user's condition may be, for example, a condition related to the function of the user's eyes, brain, and / or mind, as will be described in more detail below.

[0077] The above configuration can provide a technical improvement that solves or alleviates at least some of the problems of the prior art described above.

[0078] Specifically, with the above configuration, it is possible to appropriately determine the state of the user based on the behavior information of the user while the first content is being played back.

[0079] In particular, when determining a user's condition (for example, field of view) using a head-mounted display device such as an HMD, although the distance between the eyes and the screen can be kept constant, in order to obtain more accurate determination results, it is necessary to keep the user's line of sight fixed so that it does not move from the center of the display area.

[0080] Therefore, by providing content that is suited to a special determination environment, it becomes possible to appropriately determine the state of the user.

[0081] As described above, the user's behavior information may be gaze information relating to the user's gaze, operation information relating to the user's operation, and / or speech information relating to the user's speech.

[0082] The gaze information regarding the user's gaze can be acquired by a camera included in the display device 200. Furthermore, in the case of the display device 200 having an infrared irradiation function, data such as the gaze direction, the degree of eye opening, and the size and position of the pupils can be acquired by using the dual camera and infrared irradiation.

[0083] The user behavior information may also include facial expression information relating to the user's facial expression. The facial expression information relating to the user's facial expression may be acquired by a camera included in the display device 200. Specifically, data relating to the user's facial expression and mouth movement may be acquired by tracking each part of the user's face.

[0084] The operation information regarding the user's operation may be acquired by a controller connected to the display device 200. Alternatively, the operation information may be acquired as the user's hand movement captured by a camera provided in the display device 200.

[0085] The speech information regarding the user's speech can be acquired by a microphone provided in the display device 200.

[0086] As mentioned above, the user's condition may be a condition relating to the function of the user's eyes, brain and / or mind.

[0087] The condition related to the function of the eye may be, for example, the condition related to the presence or absence of symptoms of glaucoma, the condition related to eyesight, or the like.

[0088] The brain function can be, for example, a state related to the presence or absence of symptoms of dementia.

[0089] As an example, the mental function can be a state related to the presence or absence of symptoms of a mental illness.

[0090] The one or more computer processors in the present disclosure may further include a generating unit 140, as shown in FIG. 7 as an example.

[0091] The generating unit 140 generates information to be provided to the user based on the result of the determining unit.

[0092] As an example, the provided information may be information guiding to a service that provides information about the user's status in the virtual space.

[0093] A service that provides information about a user's condition is, for example, a service that allows the user to receive consultation or medical treatment about their condition in a virtual space, such as a hospital in a virtual space.

[0094] Moreover, the provided information may be, for example, guidance information for the second content.

[0095] The second content may be, for example, game content or video content.

[0096] Specifically, the second content may be game content or video content dedicated to maintaining or improving the user's condition, as will be described in more detail below.

[0097] As described above, the provided information can be guidance information to a service that provides information about the user's status in the virtual space.

[0098] A service that provides information about a user's condition is, for example, a service that allows the user to receive consultation or medical treatment about their condition in a virtual space, such as a hospital in a virtual space. Such a hospital in a virtual space can be called a metaverse hospital or a virtual hospital.

[0099] The guidance information may be information for displaying or navigating a route to a location in the virtual space where the service is provided, or information for reserving the service.

[0100] FIG. 8 shows an example of a display of a route for a spaceship to reach a service providing location P1, and the route is displayed in the display area 12 using an arrow P2 and the like.

[0101] The guidance information may also be link information for warping to a location in the virtual space where the service is provided.

[0102] In addition, the guidance information may be information (such as a button) for displaying the user (doctor, etc.) involved in the service provision or the user's avatar CO in the display area 12 and enabling conversation, as shown as an example in Figure 9.

[0103] For example, if it is determined that the user's eye function is abnormal, guidance information to an ophthalmologist in the virtual space is displayed. The user can access the ophthalmologist in the virtual space by following the guidance information. The user can also make an appointment to visit the ophthalmologist in the virtual space by following the reservation method indicated in the guidance information. At the ophthalmologist, the user can consult with a doctor or the like about their symptoms and receive online medical treatment. In this case, the user and / or the doctor may be displayed as an avatar. The eye function can be determined more accurately by using a head-mounted display device as the display device.

[0104] For example, if it is determined that the user's brain function is abnormal, guidance information to a psychiatrist, psychosomatic medicine specialist, neurosurgeon, geriatrician, etc. in the virtual space is displayed. The user can access a psychiatrist, psychosomatic medicine specialist, neurosurgeon, geriatrician, etc. in the virtual space by following the guidance information. At these locations, the user can consult with a doctor or the like and receive online medical treatment. In this case, the user and / or the doctor may be displayed as an avatar. Brain function can be determined more accurately by using a head-mounted display device.

[0105] For example, if it is determined that the user's mental function is abnormal, guidance information to a psychiatrist, neuropsychiatrist, etc. in the virtual space is displayed. The user can access the psychiatrist, neuropsychiatrist, etc. in the virtual space by following the guidance information. There, the user can consult with a doctor or the like and receive online medical treatment. In this case, the user and / or the doctor may be displayed as an avatar. The mental function can be determined more accurately by using a head-mounted display device.

[0106] As described above, the provided information can be guidance information for second content for maintaining or improving the user's condition in the virtual space.

[0107] Here, the first content and the second content will be described in detail.

[0108] The first content is game content or video content dedicated to obtaining information for determining the state of the user.

[0109] The second content may be game content or video content dedicated to maintaining or improving the user's condition.

[0110] Game content can be content that progresses in response to user operations, but it can also be game content that is not for entertainment purposes, but is dedicated to obtaining information for determining the user's condition or for maintaining or improving the user's condition.

[0111] Video content can be content that a user watches, but not video content for entertainment purposes, but video content dedicated to obtaining information for determining a user's condition or for maintaining or improving a user's condition.

[0112] For example, the first content may be content for evaluating the user's field of view, such as game content simulating a shooting game, as shown in FIG.

[0113] Such game content is a game in which, for example, the user inputs by pressing a first button on the controller (SHOOT) when a moving first object 17 overlaps with a crosshair 16 fixed in the central region R1 of the display area, and the user inputs by pressing a second button on the controller (CAPTURE) when a second object (light body) 18 appears in the peripheral region R2 of the display area, thereby adding up scores and progressing the game.

[0114] As will be described in more detail below, when assessing a user's field of vision, it is necessary to fix the center of the user's field of vision and assess the range of peripheral vision; if the head and / or gaze move from a fixed position, it may not be possible to obtain appropriate data to use in assessing the field of vision.

[0115] Therefore, in the first content, various configurations can be adopted to prevent the user's line of sight from moving from the central region R1 of the display area.

[0116] For example, as described above, a configuration can be adopted in which the user inputs by pressing a first button on the controller when a moving first object (second object) 17 overlaps with a crosshair (first object) 16 fixed in the central region R1 of the display area, and the user inputs by pressing a second button on the controller when a second object (third object) 18 appears in the peripheral region R2 of the display area, thereby adding up the score.

[0117] At this time, a configuration can be adopted in which the user's gaze is tracked, and if the user's gaze is in the central region R1, an additional score is added.

[0118] Alternatively, instead of operating the controller, a configuration can be adopted in which a sight that follows the user's line of sight is displayed, and a score is added when the sight that follows the line of sight is in the central region R1 and overlaps with the first object 17.

[0119] Furthermore, when the distance between the first object 17 as the target and the crosshair 16 or the central region R1 becomes equal to or less than a predetermined value, a predetermined sound (for example, an alarm sound) may be played. Also, the time until the first object 17 as the target collides with the central region R1 may be displayed on the screen.

[0120] Furthermore, the sound effect can be changed according to the distance between the first object 17, which is the target, and the crosshair 16 or the central region R1. Examples of how the sound effect can be changed include changing the frequency or rhythm, and the sound effect can be configured to gradually increase the sense of tension.

[0121] The position where the second object (light body) 18 appears will be described in detail. The three-dimensional coordinate position where the light body appears in the three-dimensional virtual space is set to be a position that is 45 m away from the coordinates (0,0,0) of the user's viewpoint and is spaced 6 degrees apart from the coordinates (0,0,0) of the user's viewpoint.

[0122] These values ​​(45 m and 6 degrees) are not strictly limited, but were adopted in accordance with the ease of viewing and the Humphrey perimeter.

[0123] Logs can be collected for each of the multiple areas enclosed by dashed rectangles in Fig. 17 regarding the time from the appearance of the light object to the successful or unsuccessful capture. Fig. 17 shows the light object position and the partitioned areas used when evaluating the visual field of the user's right eye, and illustrates the coordinate range in which the light object is displayed converted from distance (m) to visual field angle (degrees).

[0124] The determining unit 130 can determine the state of the user based on the information relating to time in addition to the user behavior information received by the receiving unit 120 .

[0125] With the above configuration, it is possible to cover the evaluation of the visual field in the range corresponding to the visual field test of the medical device, except for the central region R1.

[0126] In addition, the brightness of the light source can be specified in RGB in the virtual space, and by later converting from RGB to HSL, it can be converted into retinal sensitivity using the formula dB = ln((maximum index brightness) / (presented index brightness)), which can be used to correspond to visual field tests in ophthalmology.

[0127] Retinal sensitivity is a parameter actually measured by a perimeter used in ophthalmology, and it is determined whether each measurement point is more difficult to see than a healthy person based on a database of normal values ​​for each age. This was adopted to make it easier to determine the usefulness of the present invention from a medical perspective.

[0128] In addition, it is possible to use disease data managed by the government, medical institutions, etc. that associates the eye health status of actual patients with other diseases and stores it, and to use artificial intelligence to predict other related diseases of users (such as "sleep apnea syndrome," "low blood pressure," "sensitive to cold," and "migraine") from the acquired game logs.

[0129] That is, the one or more computer processors in the present disclosure may further include an acquisition unit and a prediction unit, wherein the acquisition unit acquires disease data that associates patient information with disease information stored in a specified storage device, and the prediction unit can predict other diseases of the user based on the disease data acquired by the acquisition unit and the user's condition determined by the determination unit.

[0130] As another example, the first content may be content for evaluating the user's eyesight, such as game content simulating a vision test, as shown in FIG.

[0131] Such game content is a game that progresses by, for example, the user pressing a corresponding button on the controller or uttering a corresponding word in accordance with the shape of the target icon 19 displayed in the display area.

[0132] As an example, the second content may be content for maintaining or improving the user's field of vision, such as game content simulating a shooting game, as shown in FIG.

[0133] In such a game, for example, a configuration can be applied that allows the user to exercise by moving their eyeballs widely up and down and left and right.

[0134] Therefore, in the second content, various configurations can be adopted to move the user's line of sight to the surrounding area R2 of the display area.

[0135] For example, a configuration can be adopted in which the first object 17 moves in the surrounding region R2, and a score is added when a crosshair that follows the line of sight overlaps the first object 17. In this case, the appearance of the second object 18 is not essential.

[0136] The one or more computer processors in the present disclosure may further include a gaze determination unit 150, as shown in FIG. 7 as an example.

[0137] The gaze determination unit 150 determines whether the user's gaze position is within a first region R1 (Figure 10) set in the center of the display area 12 based on gaze information regarding the user's gaze, which is user behavior information received by the receiving unit 120.

[0138] The one or more computer processors in the present disclosure may further include an associating unit 160, as shown in FIG. 7 as an example.

[0139] When the gaze determination unit 150 determines that the gaze position is within the first region R1, the associating unit 160 associates the first reward with the user information of the user.

[0140] Such a first reward may be an addition to the score described above.

[0141] The one or more computer processors in the present disclosure may further include a playback unit 170, as shown in FIG. 7 as an example.

[0142] The playback unit 170 causes a predetermined display device 200 to play a predetermined sound when the gaze determination unit 150 determines that the gaze position is in the second region R2 (FIG. 10).

[0143] The predetermined sound may be the alarm sound described above.

[0144] The one or more computer processors in the present disclosure may further include a display processing unit 180, as shown in FIG. 7 as an example.

[0145] As shown as an example in Figure 10, the display processing unit 180 displays in the display area 12 a screen including a first object 16 displayed within a first area R1 set in the center of the display area, a second object 17 different from the first object and moving at least within the first area R1, and a third object 18 different from the first object 16 and the second object 17 and displayed for at least a predetermined period of time in a second area R2 outside the first area R1.

[0146] The one or more computer processors in the present disclosure can change the display position of the display area in response to a movement of the user's head while the first content is being played.

[0147] In other words, from the start of playback of the first content until playback of the first content is stopped or ended, the center of the display area can always be displayed in the center (front) of the user's field of vision, even if the user moves their head.

[0148] With this configuration, it is possible to prevent a decrease in the accuracy of determining the user's state (particularly the state of the field of view) due to movement of the user's head.

[0149] Alternatively, one or more computer processors in the present disclosure may change the display position of the display area in response to the movement of the user's line of sight while a particular piece of content is being played.

[0150] In other words, from the start of playback of the first content until playback of the first content is stopped or finished, the center of the display area can always be displayed in front of the user's line of sight, even if the user moves their line of sight.

[0151] With this configuration, it is possible to prevent a decrease in the accuracy of determining the user's state (particularly the state of the field of view) due to movement of the user's head.

[0152] For example, when assessing a user's field of vision, it is necessary to fix the center of the user's field of vision and assess the range of peripheral vision, but if the head and / or gaze move from a fixed position, it may not be possible to obtain appropriate data to use in assessing the field of vision.

[0153] However, with the above configuration, even if the user's head and / or line of sight move from a fixed position, the display position of the display area can be appropriately adjusted to appropriately determine the user's condition.

[0154] A character object that moves in accordance with user behavior information can be displayed in the virtual space.

[0155] Furthermore, the receiving unit 120 further receives biometric information of the user, and the determining unit 130 can determine the state of the user based on the behavior information of the user and the biometric information of the user received by the receiving unit 120 .

[0156] The biological information referred to here refers to biological information other than the biological information included in the behavior information described above. For example, the biological information may include information related to brain waves, pulse, body temperature, etc.

[0157] By using this biological information together with the behavior information described above, it becomes possible to determine the state of the user with higher accuracy.

[0158] The one or more computer processors in the present disclosure may further include a notification unit 150, as shown in FIG. 17 as an example.

[0159] The notification unit 150 notifies the result of the determination by the determination unit 130 to the user terminals of other users associated with the user.

[0160] The other users may be users who are related to the user in the real world, such as the user's family, friends, acquaintances, etc. The other users may also be administrators of real-world communities (companies, schools, nursing homes, hospitals, etc.) to which the user belongs.

[0161] With this configuration, it is possible to notify users who are related to the user in the real world of the user's state determined in the virtual space.

[0162] The notification unit 150 may not notify other users of all the results of the determination made by the determination unit 130, but may decide whether or not to notify them depending on the results of the determination.

[0163] When the provided information is guidance information to a service that provides information about the user's status in the virtual space, the notification unit 150 can further notify the user of the status of the service provided to the user based on the guidance information.

[0164] The service provision status refers to whether or not the service has been provided to the user, that is, whether or not the user has received the service based on the guidance information.

[0165] This configuration can encourage users to receive the service.

[0166] One or more computer processors in the present disclosure may allow access to the virtual space from user terminals of other users associated with the user.

[0167] In this case, one or more computer processors in the present disclosure may enable conversation between the user and other users and / or display of the user's avatar and other users' avatars in the virtual space.

[0168] The other users may be associated with the user and are not limited to the above-mentioned family members, administrative users, etc. In other words, the other users referred to here may also include the user's friends in the virtual space who have no relationship with the user in the real world.

[0169] According to the above configuration, a plurality of users, including the user and other users, can access the same virtual space.

[0170] At this time, the avatars of the user and the other users may be displayed in the virtual space.

[0171] Furthermore, the avatar may reflect the facial expressions and / or movements of the user and other users, or may move in response to operations by the user and other users.

[0172] Furthermore, the conversation between the user and other users includes conversation via voice chat or text conversation via comment input.

[0173] According to the above configuration, communication with other users in the virtual space becomes more active, which can be useful in activating the user's brain.

[0174] The service that provides information about the user's condition in the virtual space may be a consultation service with an expert, a food delivery service, and / or an insurance service.

[0175] The consultation service with experts is a service that allows users to consult with experts such as doctors, nurses, and counselors about their own condition via virtual space.

[0176] The food delivery service allows users to request delivery of products necessary to improve their condition via virtual space.

[0177] The insurance service is a service that proposes insurance tailored to the user's condition via a virtual space. The terms of insurance (such as the amount of insurance) may vary depending on the user's condition and the usage of the service.

[0178] In this way, according to the invention of the present disclosure, the user can receive services in the virtual space that correspond to the user's state.

[0179] Next, an information processing method according to the present disclosure will be described with reference to the drawings.

[0180] The information processing method according to the present disclosure is characterized in that one or more computer processors execute a sending step S110, a receiving step S120, and a determining step S130, as shown as an example in FIG.

[0181] A transmission step S110 transmits, to a predetermined display device, information for displaying a virtual space having a display area capable of reproducing specific content.

[0182] The transmitting step S110 can be performed by the above-described transmitting unit 110. Details of the transmitting unit 110 are as described above.

[0183] A receiving step S120 receives behavior information of a user who is reproducing a first content in a display area, the behavior information being transmitted from a predetermined display device.

[0184] The receiving step S120 can be performed by the above-described receiving unit 120. Details of the receiving unit 120 are as described above.

[0185] In the determining step S130, the state of the user is determined based on the behavior information of the user received in the receiving step S120.

[0186] The determination step S130 can be performed by the above-described determination unit 130. Details of the determination unit 130 are as described above.

[0187] The above configuration can provide a technical improvement that solves or alleviates at least some of the problems of the prior art described above.

[0188] Next, the computer program according to the present disclosure will be described with reference to the drawings.

[0189] The computer program in the present disclosure is characterized by causing one or more computer processors to implement a transmitting function, a receiving function, and a determining function.

[0190] The transmission function transmits, to a predetermined display device, information for displaying a virtual space having a display area capable of reproducing specific content.

[0191] The receiving function receives behavior information of a user who is reproducing a first content in a display area, the behavior information being transmitted from a predetermined display device.

[0192] The determining function determines the state of the user based on the user behavior information received by the receiving function.

[0193] The above functions can be realized by circuits 1110 to 1130 shown in Fig. 13. The transmitting circuit 1110, the receiving circuit 1120, and the determining circuit 1130 are realized by the above-mentioned transmitting unit 110, receiving unit 120, and determining unit 130, respectively. Details of each unit are as described above.

[0194] The above configuration can provide a technical improvement that solves or alleviates at least some of the problems of the prior art described above.

[0195] Next, the display device 200 according to the present disclosure will be described with reference to the drawings.

[0196] The display device 200 according to the present disclosure is characterized by including a display processing unit 210, an acquisition unit 220, and a determination unit 230, as shown in FIG.

[0197] The display processing unit 210 displays a virtual space having a display area in which specific content can be played back on the display unit.

[0198] The acquisition unit 220 acquires behavior information of a user while the first content is being reproduced in the display area.

[0199] The determination unit 230 determines the state of the user based on the user behavior information acquired by the acquisition unit 220. The determination unit 230 can have the same configuration as the determination unit 130 described above.

[0200] The above configuration can provide a technical improvement that solves or alleviates at least some of the problems of the prior art described above.

[0201] Next, an information processing method according to the present disclosure will be described with reference to the drawings.

[0202] The information processing method according to the present disclosure is characterized in that, as shown in FIG. 15, one or more computer processors are caused to execute a display processing step S210, an acquisition step S220, and a determination step S230.

[0203] In the display processing step S210, a virtual space having a display area in which specific content can be played back is displayed on the display unit.

[0204] The display processing step S210 can be executed by the above-described display processing unit 210. Details of the display processing unit 210 are as described above.

[0205] In the acquisition step S220, behavior information of the user while the first content is being reproduced in the display area is acquired.

[0206] The acquisition step S220 can be performed by the above-described acquisition unit 220. Details of the acquisition unit 220 are as described above.

[0207] In the determining step S230, the state of the user is determined based on the behavior information of the user acquired in the acquiring step S220.

[0208] The determination step S230 can be performed by the above-described determination unit 230. Details of the determination unit 230 are as described above.

[0209] The above configuration can provide a technical improvement that solves or alleviates at least some of the problems of the prior art described above.

[0210] Finally, the computer program of the present disclosure will be described with reference to the drawings.

[0211] The computer program in the present disclosure is characterized by causing one or more computer processors to realize a display processing function, an acquisition function, and a determination function.

[0212] The display processing function causes the display unit to display a virtual space in which a first space having a display area capable of reproducing specific content is provided.

[0213] The acquisition function acquires behavior information of a user while the first content is being reproduced in the display area.

[0214] The determination function determines the state of the user based on the user behavior information acquired by the acquisition function.

[0215] The above functions can be realized by circuits 1210 to 1230 shown in Fig. 16. The display processing circuit 1210, acquisition circuit 1220, and determination circuit 1230 are realized by the above-mentioned display processing unit 210, acquisition unit 220, and determination unit 230, respectively. Details of each unit are as described above.

[0216] The above configuration can provide a technical improvement that solves or alleviates at least some of the problems of the prior art described above.

[0217] Furthermore, an information processing device such as a computer or a mobile phone can be suitably used to function as the server device or terminal device according to the above-described embodiments. Such an information processing device can be realized by storing a program describing the processing content for realizing each function of the server device or terminal device according to the embodiments in a storage unit of the information processing device, and having the CPU of the information processing device read and execute the program.

[0218] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims.

[0219] Furthermore, the methods described in the embodiments can be stored as a program executable by a computer on a recording medium such as a magnetic disk (e.g., a floppy disk, a hard disk, etc.), an optical disk (e.g., a CD-ROM, a DVD, an MO, etc.), or a semiconductor memory (e.g., a ROM, a RAM, a flash memory, etc.), and can also be distributed by transmitting it via a communication medium. The program stored on the medium also includes a configuration program that configures software means (including not only executable programs but also tables and data structures) within the computer for execution. The computer implementing this device loads the program stored on the recording medium and, in some cases, configures the software means using the configuration program, and executes the above-described processing by controlling the operation of this software means. Note that the term "recording medium" as used herein is not limited to a storage medium for distribution, but also includes storage media such as a magnetic disk or semiconductor memory installed within the computer or in a device connected via a network. The storage unit may function as, for example, a main storage device, an auxiliary storage device, or a cache memory.

[0220] 100 Server device 110 Transmission unit 120 Reception unit 130 Determination unit 200 Display device

Claims

1. An information processing system having one or more computer processors, the one or more computer processors having: a transmitting unit that transmits information for displaying a virtual space having a display area in which specific content can be played to a specified display device; a receiving unit that receives behavior information of a user who is playing a first content in the display area, transmitted from the specified display device; and a determining unit that determines a state of the user based on the user behavior information received by the receiving unit.

2. An information processing system according to claim 1, characterized in that the user's behavior information is gaze information relating to the user's gaze, operation information relating to the user's operation, and / or speech information relating to the user's speech.

3. The information processing system according to claim 1, wherein the user's condition is a condition related to the function of the user's eyes.

4. An information processing system according to claim 1, characterized in that the user's state is a state relating to the user's brain function and / or mental function.

5. The information processing system according to claim 1, characterized in that the first content is game content or video content for obtaining information for determining the user's state.

6. The information processing system described in claim 1, characterized in that the one or more computer processors further include a gaze determination unit that determines whether the user's gaze position is within a first area set in the center of the display area based on gaze information regarding the user's gaze, which is the user's behavior information received by the receiving unit.

7. The information processing system described in claim 6, characterized in that the one or more computer processors further include an association unit that associates a first reward with the user information of the user when the gaze position is determined to be within the first area as a result of the gaze determination unit.

8. The information processing system according to claim 6, further characterized in that the one or more computer processors are provided with a playback unit that plays a specified sound on the specified display device when the gaze position is determined to be in a second area outside the first area as a result of determination by the gaze determination unit.

9. The information processing system according to claim 1, wherein the virtual space includes a first space having a display area capable of playing back the specific content.

10. An information processing system as described in claim 9, characterized in that the display area is an area represented as one area among one or more areas provided in the first space and facing outside the first space from within the first space.

11. An information processing system as described in claim 9, characterized in that before playback of the specific content begins, an image of a space outside the first space is displayed in the display area, and the specific content is played in the display area in response to an operation or action of the user.

12. An information processing system according to claim 9, wherein the first space is a space representing the interior of a spacecraft.

13. An information processing system according to claim 9, characterized in that the display area and a part of the first space are displayed in the user's field of view image during playback of the specific content.

14. The information processing system according to claim 9, characterized in that the specific content is started to be played in the display area in response to an operation or action of the user.

15. The information processing system of claim 9, wherein the specific content is triggered to start playing in response to a user's operation on an operation panel object arranged in the first space, or when the user performs an action of sitting on a chair object arranged in the first space.

16. The information processing system according to claim 1, wherein the one or more computer processors further include a generation unit that generates information to be provided to the user based on the result of the determination unit.

17. An information processing system according to claim 16, wherein the provided information is guidance information to a service that provides information regarding the user's status in the virtual space.

18. An information processing system according to claim 17, wherein said guidance information is information for displaying a route to a location in said virtual space where said service is provided.

19. An information processing system according to claim 17, wherein said guide information is information for reserving said service in said virtual space.

20. An information processing system according to claim 16, characterized in that the provided information is guidance information for a second content for maintaining or improving the user's condition in the virtual space.

21. The information processing system according to claim 20, wherein the second content is game content or video content dedicated to maintaining or improving the user's condition.

22. The information processing system according to claim 1, characterized in that the one or more computer processors display a fixed central area of ​​the display area at a central position of the user's field of view during playback of the first content.

23. The information processing system according to claim 1, characterized in that the one or more computer processors change the display position by fixing the central area of ​​the display area in front of the user's line of sight during playback of the first content.

24. The information processing system according to claim 1, characterized in that a character object controlled by the one or more computer processors is displayed in the virtual space, and the character object behaves according to behavior information of the user.

25. The information processing system of claim 1, wherein the receiving unit further receives biometric information of the user, and the determination unit determines the state of the user based on the behavior information of the user and the biometric information of the user received by the receiving unit.

26. An information processing system according to claim 25, wherein the biological information is pulse rate and / or body temperature.

27. The information processing system according to claim 1, characterized in that the one or more computer processors further include a notification unit that notifies a user terminal of another user associated with the user of the result of the determination by the determination unit.

28. The information processing system described in claim 27, wherein the one or more computer processors further include a generation unit that generates provision information to be provided to the user based on the result of the judgment unit, the provision information being guidance information to a service that provides information regarding the user's status in the virtual space, and the notification unit further notifies the other users of the provision status of the service to the user based on the guidance information.

29. An information processing system according to claim 28, wherein the service provision status is a status as to whether or not the service has been provided to the user.

30. The information processing system according to claim 1, characterized in that the one or more computer processors allow access to the virtual space from a user terminal of another user associated with the user, and enable conversation between the user and the other user within the virtual space and / or display of the user's avatar and the other user's avatar.

31. The information processing system according to claim 17, wherein the service that provides information about the user's condition in the virtual space is a consultation service provided by an expert.

32. The information processing system according to claim 17, wherein the service that provides information about the user's condition in the virtual space is a food delivery service and / or an insurance service.

33. The information processing system according to claim 1, characterized in that the one or more computer processors further comprise: an acquisition unit that acquires disease data that associates patient information with disease information stored in a specified storage device; and a prediction unit that predicts other diseases of the user based on the disease data acquired by the acquisition unit and the user's condition determined by the determination unit.

34. An information processing method that causes one or more computer processors to execute the following steps: a transmitting step of transmitting, to a specified display device, information for displaying a virtual space having a display area capable of playing specific content; a receiving step of receiving, from the specified display device, behavior information of a user currently playing a first content in the display area; and a determining step of determining a state of the user based on the behavior information of the user received in the receiving step.

35. A display device comprising: a display processing unit that causes a display unit to display a virtual space having a display area in which specific content can be played; an acquisition unit that acquires behavior information of a user while a first content is being played in the display area; and a judgment unit that judges the state of the user based on the behavior information of the user acquired by the acquisition unit.

36. A computer program that causes one or more computer processors to realize the following functions: a display processing function that causes a display unit to display a virtual space having a display area in which specific content can be played; an acquisition function that acquires behavior information of a user while a first content is being played in the display area; and a judgment function that judges the state of the user based on the behavior information of the user acquired by the acquisition function.

Images