Information processing system, information processing method, and recording medium

Abstract

According to one embodiment of the present invention, provided is an information processing system. In this information processing system, a control is performed so as to display a virtual space image on a display unit that can be viewed by a user. A combinations of a painting image of a first category and a painting image of a second category different from the first category is sequentially displayed on the virtual space image. Information pertaining to the stress of a user is determined on the basis of information regarding the position of the line of sight of the user with respect to the painting image of the first category and the painting image of the second category that have been sequentially displayed.

Description

Information processing system, information processing method, and recording medium

[0001] The present invention relates to an information processing system, an information processing method, and a program.

[0002] Patent Document 1 discloses a healthcare system that supports the prediction of health-related items for members.

[0003] Japanese Patent Application Laid-Open No. 2022-080198

[0004] In Patent Document 1, a large amount of learning data is required, such as medical examination data, stress check data, work data, personnel data, salary data, lifestyle data, medical data, and company data. Generally, a large amount of data is required for learning. Furthermore, to improve accuracy, learning and evaluation of the results are necessary, which is costly.

[0005] According to one aspect of the present invention, an information processing system is provided. The information processing system controls the display of a virtual space image on a display unit visible to a user. The virtual space image sequentially displays a combination of a first category of pictorial images and a second category of pictorial images different from the first category. Information about the user's stress is obtained based on information about the user's gaze position relative to the sequentially displayed pictorial images of the first category and the second category.

[0006] FIG. 1 is a diagram illustrating an example of a user wearing an HMD. FIG. 2 is a diagram illustrating an example of the hardware configuration of an HMD. FIG. 3 is a main flowchart illustrating an example of information processing in an HMD. FIG. 4 is a diagram illustrating an example of a three-dimensional virtual space image displayed after login. FIG. 5 is a flowchart illustrating an example of stress check processing. FIG. 6 is a diagram illustrating an example of sequentially displaying sets of combinations of painting images of a first category and painting images of a second category displayed in a three-dimensional virtual space image. FIG. 7 is a flowchart illustrating an example of vision training processing. FIG. 8 is a diagram illustrating an example of a process in which at least one object image is selected from a plurality of object images displayed in a three-dimensional virtual space image and displayed in a manner distinguishable from the other object images.

[0007] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the accompanying drawings. Various features shown in the following embodiments can be combined with each other.

[0008] In this specification, the term "unit" may include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In addition, while various types of information are handled in the first embodiment, this information is represented by high or low signal values ​​as a binary bit set consisting of 0 or 1, and communication and calculations can be performed on the circuit in the broad sense.

[0009] Furthermore, a circuit in the broad sense is a circuit realized by at least an appropriate combination of a circuit, circuitry, a processor, a memory, etc. That is, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0010] <First Embodiment> 1. HMD Fig. 1 is a diagram showing an example of a user 10 wearing an HMD (Head Mounted Display) 100. As shown in Fig. 1, in the first embodiment, the description will be given assuming that the user 10 wears the HMD 100 and that the HMD 100 performs processing, etc., which will be described later.

[0011] Here, the claimed information processing system may be composed of multiple devices or may be composed of one device. When the claimed information processing system is composed of one device, an example of the device is the HMD 100. When the claimed information processing system is composed of multiple devices, an example of the multiple devices is the HMD 100 and a server device capable of communicating with the HMD 100.

[0012] In the first embodiment, the HMD 100 will be described as an example, but any device may be used as long as it allows the user 10 to perform actions in a three-dimensional virtual space constructed on the Internet. Another example of the HMD 100 is smart glasses.

[0013] 2. Hardware Configuration Fig. 2 is a diagram showing an example of the hardware configuration of the HMD 100. As shown in Fig. 2, the HMD 100 includes, as its hardware configuration, a control unit 210, a storage unit 220, an imaging unit 230, an input unit 240, an output unit 250, a communication unit 260, a gyro sensor 270, and an acceleration sensor 280.

[0014] The control unit 210 is a CPU (Central Processing Unit) or the like, and controls the entire HMD 100 .

[0015] The storage unit 220 is any one of a hard disk drive (HDD), a read-only memory (ROM), a random access memory (RAM), a solid state drive (SSD), etc., or any combination thereof, and stores programs and data used by the control unit 210 when executing processes based on the programs. The storage unit 220 is an example of a storage medium. In the specification, data used by the control unit 210 when executing processes based on the programs (e.g., a three-dimensional virtual space image, a set of pictorial images of a first category and a second category described below, object images described below, thresholds, setting values, etc.) is described as being stored in the storage unit 220. However, the data may also be stored in a storage unit of another device that can communicate with the HMD 100. In other words, the data may be stored in a storage unit of any device that can be accessed by the control unit 210. The control unit 210 executes processing based on the program stored in the memory unit 220, thereby realizing the functions of the HMD 100 and the information processing of the flowcharts shown in Figures 3, 5, and 7, which will be described later.

[0016] The imaging unit 230 captures an image of the user 10 and generates a captured image. The captured image includes both eyes of the user 10.

[0017] The input unit 240 is a button or the like provided on the HMD 100, and inputs operation information of the user 10 to the HMD 100. The output unit 250 is a display or the like, and displays a three-dimensional virtual space image or the like generated by the control unit 210. The output unit 250 is an example of a display unit. The communication unit 260 connects the HMD 100 to a network and controls communication with other devices.

[0018] The gyro sensor 270 is a sensor that detects changes in the rotation and orientation of the HMD 100 as angular velocity and outputs an electrical signal to the control unit 210, etc. The acceleration sensor 280 is a sensor that detects the inertial force that occurs when the movement speed of the HMD 100 changes and outputs an electrical signal as acceleration.

[0019] The hardware configuration of the HMD 100 includes two lenses in addition to those shown in Fig. 2. The left eye of the user 10 views the virtual space image through the left lens, and the right eye views the virtual space image through the right lens.

[0020] 3. Information Processing The information processing of the first embodiment will now be described.

[0021] 3 is a main flowchart showing an example of information processing in the HMD 100. When the user 10 logs in to the system, the control unit 210 executes a calibration process in step S310. As part of the calibration process, the control unit 210 determines the position of the eyeballs of the user 10 based on a captured image including the iris or pupil of the user 10, and corrects misalignment between the HMD 100 and the head of the user 10 in the horizontal and vertical directions.

[0022] In step S320, control unit 210 generates a three-dimensional virtual space image and performs control to display it on output unit 250. Control unit 210 determines the line of sight of user 10 based on the captured image captured by imaging unit 230, generates a three-dimensional virtual space image according to the line of sight, and performs control to display it on output unit 250. The processing of step S320 is an example of a process of controlling to display a virtual space image on a display unit visible to the user.

[0023] FIG. 4 is a diagram showing an example of a three-dimensional virtual space image displayed after login. While FIG. 4 shows an example of a person being drawn, the person is merely an image. In reality, the user 10 is in the three-dimensional virtual space via the output unit 250 of the HMD 100, and visually observes images and the like displayed in the three-dimensional virtual space according to the user's line of sight. Multiple signboards are displayed in the air within the three-dimensional virtual space image. In the example of FIG. 4, a stress check signboard 410 and a vision training signboard 420 are displayed as examples of multiple signboards. The multiple signboards are an example of multiple graphical user interface components. The stress check signboard 410 is an example of a graphical user interface component related to a stress check. The vision training signboard 420 is an example of a graphical user interface component related to vision training.

[0024] That is, the processing of step S320 is an example of processing for displaying multiple graphical user interface components in a virtual space image. The multiple graphical user interface components include graphical user interface components related to stress checks. In the example of FIG. 4, a stress check information board 410 and a vision training information board 420 are displayed as examples of multiple information boards, but other information boards may also be included. However, for the sake of simplicity, the following description will be given using the stress check information board 410 and the vision training information board 420 as examples of multiple information boards.

[0025] In step S330, the control unit 210 determines which of the displayed guide boards has been selected based on the line of sight of the user 10. For example, if the control unit 210 determines based on the line of sight of the user 10 that the gaze area of ​​the user 10 is on the stress check guide board 410, the control unit 210 determines that the stress check has been selected, and if the control unit 210 determines that the gaze area of ​​the user 10 is on the vision training guide board 420, the control unit 210 determines that the vision training has been selected. If the control unit 210 determines that the stress check has been selected, the process proceeds to step S340, and if the control unit 210 determines that the vision training has been selected, the process proceeds to step S350.

[0026] As another example, the control unit 210 may determine which signboard has been selected based on the line of sight of the user 10 when receiving an input operation via the input unit 240. When the control unit 210 determines that the gaze area of ​​the user 10 is on the stress check signboard 410 when receiving an input operation via the input unit 240, it determines that the stress check has been selected, and when it determines that the gaze area of ​​the user 10 is on the vision training signboard 420, it determines that the vision training has been selected.

[0027] In step S340, the control unit 210 executes a stress check process.

[0028] (2) Flowchart of Stress Check Details of the stress check process will be explained using the flowchart shown in FIG. 5. FIG. 5 is a flowchart showing an example of the stress check process. In step S510, the control unit 210 displays a combination of a first category of painting images and a second category of painting images different from the first category within a three-dimensional virtual space image. The first category of painting images are, for example, images of paintings drawn by people who are not depressed. The second category of painting images are, for example, images of paintings drawn by people who are depressed.

[0029] In step S520, the control unit 210 determines the gaze position of the user 10 based on the captured image including both eyes of the user 10. The control unit 210 determines whether the gaze position is more prevalent on the pictorial images in the first category or the pictorial images in the second category, and stores the determined result in a predetermined storage area such as the storage unit 220.

[0030] In step S530, the control unit 210 determines whether the processes of steps S510 and S520 have been executed a predetermined number of times. If the control unit 210 determines that the processes of steps S510 and S520 have been executed a predetermined number of times, the control unit 210 proceeds to step S540. If the control unit 210 determines that the processes of steps S510 and S520 have not been executed the predetermined number of times, the control unit 210 returns to step S510 and displays the next combination of a first category pictorial image and a second category pictorial image different from the first category within the three-dimensional virtual space image. The predetermined number of times is, for example, five times. However, this does not limit the first embodiment. Information about the predetermined number of times may be written in a predetermined file in the storage unit 220, etc.

[0031] The process of performing the processes of steps S510 and S520 a predetermined number of times is an example of a process of sequentially displaying, in the virtual space image, combinations of pictorial images of a first category and pictorial images of a second category different from the first category. The process of control unit 210 determining that stress check information board 410 has been selected on the screen as shown in FIG. 4 , proceeding to step S340, and repeatedly performing the processes of steps S510 and S520 is an example of a process of sequentially displaying, in the virtual space image, combinations of pictorial images of the first category and pictorial images of the second category when a graphical user interface component related to a stress check is selected from among multiple graphical user interface components based on information about the user's gaze position.

[0032] 6 is a diagram showing an example in which a set of combinations of pictorial images of a first category and pictorial images of a second category are displayed in sequence in a three-dimensional virtual space image. Image 610 is a three-dimensional virtual space image. Picture image 611 and picture image 612 are displayed in image 610. Picture image 611 is a pictorial image of the first category. Picture image 612 is a pictorial image of the second category. Both pictorial images are landscape images. For example, picture image 611 and picture image 612 are displayed by the first processing of step S510 by the control unit 210. The control unit 210 determines whether the user 10's gaze position is more often on picture image 611 or picture image 612, and stores the determined result in a predetermined storage area such as the storage unit 220.

[0033] Image 620 is a three-dimensional virtual space image. Image 620 is displayed a predetermined time after image 610 is displayed. Painting image 621 and painting image 622 are displayed in image 620. Painting image 621 is a painting image of the second category. Painting image 622 is a painting image of the first category. For example, painting image 621 and painting image 622 are displayed by the control unit 210 performing the processing of step S510 a second time. The control unit 210 determines whether the gaze of user 10 is more likely to be on painting image 621 or painting image 622, and stores the determined result in a predetermined storage area such as the storage unit 220.

[0034] Image 630 is a three-dimensional virtual space image. Image 630 is displayed a predetermined time after image 620 is displayed. Painting image 631 and painting image 632 are displayed in image 630. Painting image 631 is a painting image of the first category. Painting image 632 is a painting image of the second category. For example, painting image 631 and painting image 632 are displayed by the control unit 210 performing the processing of step S510 a third time. The control unit 210 determines whether the gaze of user 10 is more likely to be on painting image 631 or painting image 632, and stores the determined result in a predetermined storage area such as the storage unit 220.

[0035] In step S540, the control unit 210 calculates a stress level based on information stored in a predetermined storage area, such as the storage unit 220, indicating whether the user 10 looked more frequently at the pictorial images in the first category or the pictorial images in the second category in each set of pictorial images. The stress level is a value indicating the user's stress level. The stress level is an example of information related to stress. For example, the control unit 210 calculates the stress level of the user 10 based on information on the number of times the user looked at the pictorial images in the first category or the pictorial images in the second category and reference information. The reference information is information stored in the storage unit 220 or the like that associates information on the number of times the user looked at the pictorial images in the first category or the pictorial images in the second category with the stress level. The processing of step S540 is an example of processing for calculating information related to the user's stress based on information on the position of the user's gaze relative to the sequentially displayed pictorial images in the first category and the pictorial images in the second category.

[0036] In step S550, control unit 210 outputs the stress level calculated in step S540. Displaying the stress level by control unit 210 on output unit 250 is an example of output. Writing the stress level in a file or the like by control unit 210 and storing it in memory unit 220 is also an example of output. Sending data including the stress level by control unit 210 to another device via communication unit 260 is also an example of output. The processing of step S550 is an example of processing to output information related to stress.

[0037] 5, the stress level of user 10 can be calculated without collecting various data such as medical examination data, stress check data, work data, personnel data, salary data, lifestyle data, medical data, and company data. Furthermore, since there is no need to collect and learn various data, the stress level can be calculated immediately without requiring learning time. Furthermore, the function of calculating the stress level of user 10 can be realized at low cost.

[0038] In step S350 of FIG. 3, the control unit 210 executes a vision training process.

[0039] (3) Flowchart of Vision Training Details of the vision training process will be described using the flowchart shown in FIG. 7. FIG. 7 is a flowchart showing an example of the vision training process. In step S710, the control unit 210 displays vision training images within a three-dimensional virtual space image. The vision training images are multiple object images displayed three-dimensionally in a row within the three-dimensional virtual space image. The control unit 210 displays at least one object image from the multiple object images in a manner that makes it distinguishable from the other object images. Each time the control unit 210 executes the process of step S710, it randomly selects at least one object image from the multiple object images and displays it in a manner that makes it distinguishable from the other object images. An example of a distinguishable display is displaying it in a color different from the other object images. Note that this is just one example, and any display may be used as long as it can be distinguished from the other object images. Other examples of a distinguishable display include displaying it in a bold frame, displaying it in a flashing manner, displaying other object images in a different shape, displaying the selected object image and not displaying the other object images, and various other display methods.

[0040] In step S720, the control unit 210 determines the gaze position of the user 10 based on the captured image including both eyes of the user 10. More specifically, the control unit 210 determines the position of the gaze point of the user 10 based on the captured image. Note that, for simplicity of explanation, the position of the gaze point will be simply referred to as the gaze position hereinafter. The control unit 210 determines the distance in the three-dimensional virtual space between the gaze position and the object image displayed so as to be distinguishable from other object images. The control unit 210 then stores information about the determined distance in a predetermined storage area such as the storage unit 220.

[0041] In step S730, the control unit 210 determines whether the processes of steps S710 and S720 have been executed a predetermined number of times. If the control unit 210 determines that the processes of steps S710 and S720 have been executed a predetermined number of times, the control unit 210 proceeds to step S740. If the control unit 210 determines that the processes of steps S510 and S520 have not been executed a predetermined number of times, the control unit 210 returns to step S710, randomly selects at least one object image from among the multiple object images displayed three-dimensionally in a row within the three-dimensional virtual space image, and displays the selected object image in a manner distinguishable from the other object images. The predetermined number of times is, for example, five times. However, this does not limit the first embodiment. Information about the predetermined number of times may be written in a predetermined file in the storage unit 220, etc.

[0042] The control unit 210 determines that the vision training information board 420 has been selected on the screen as shown in Figure 4, proceeds to step S350, and repeatedly executes steps S710 and S720.This is an example of a process in which, when a graphical user interface component related to vision training is selected from multiple graphical user interface components based on information about the position of the user's gaze, object images are displayed sequentially in the virtual space image.

[0043] FIG. 8 is a diagram showing an example in which at least one object image is selected from multiple object images displayed in a three-dimensional virtual space image and displayed in a manner distinguishable from the other object images. Image 810 is a three-dimensional virtual space image. In image 810, object image 811 selected by the control unit 210 through the first processing of step S710 is displayed in a manner distinguishable from the other object images. Image 812 is an image indicating the gaze position of user 10. Image 812 is shown for illustrative purposes and is not actually displayed in the virtual space image. The object image is not limited to that shown in FIG. 8. Any image may be used as long as it can indicate the presence of an object in the virtual space. Other examples of object images include balls and shuttlecocks used in sports (baseballs, volleyballs, basketballs, tennis balls, table tennis balls, badminton shuttlecocks, etc.). By repeatedly executing step S710, the control unit 210 may display the selected object image (e.g., balls or shuttlecocks) so that they move sequentially from the depth direction to the front direction in the three-dimensional virtual space.

[0044] Image 820 is a three-dimensional virtual space image. Image 820 is displayed after a predetermined time (e.g., one second) has elapsed since image 810 was displayed. In image 820, an object image 821 selected by the control unit 210 in the second processing of step S710 is displayed so as to be distinguishable from other object images. Note that image 822 is an image indicating the position of the line of sight of user 10. Image 822 is shown for explanatory purposes and is not actually displayed in the virtual space image.

[0045] Image 830 is a three-dimensional virtual space image. Image 830 is displayed a predetermined time after image 820 is displayed. In image 830, an object image 831 selected by the control unit 210 in the third processing of step S710 is displayed so as to be distinguishable from other object images. Note that image 832 is an image indicating the position of the line of sight of user 10. Image 832 is shown for explanatory purposes and is not actually displayed in the virtual space image.

[0046] In step S740, the control unit 210 calculates the stereoscopic vision based on information about the distance between each selected object image and the gaze position of the user 10, which information is stored in a predetermined storage area such as the storage unit 220. The processing of step S740 is an example of processing for calculating information about the stereoscopic vision of the user based on the gaze position of the user attempting to view the object image and the position of the object image.

[0047] In step S750, the control unit 210 outputs the stereopsis calculated in step S740. Displaying the stereopsis by the control unit 210 on the output unit 250 is an example of output. Writing the stereopsis by the control unit 210 in a file or the like and storing it in the memory unit 220 is also an example of output. Sending data including the stereopsis by the control unit 210 to another device via the communication unit 260 is also an example of output. The processing of step S750 is an example of processing to output information related to the user's stereopsis.

[0048] According to the process shown in FIG. 7, the user's stereoscopic acuity can be determined without collecting various data such as medical examination data, stress check data, work data, personnel data, salary data, lifestyle data, medical data, and company data. Furthermore, since there is no need to collect and learn various data, the stereoscopic acuity can be determined immediately without requiring learning time. Furthermore, the function of determining the stereoscopic acuity of the user 10 can be realized at low cost. The control unit 210 may also perform a calibration process again immediately before executing the process of step S710. This eliminates discrepancies and allows the user's stereoscopic acuity, etc. to be determined more accurately.

[0049] In step S360 of Fig. 3, the control unit 210 determines whether or not to end the processing of the flowchart shown in Fig. 3. For example, if the user's line of sight is on an image displayed in the three-dimensional virtual space image for a predetermined period of time, the control unit 210 determines to end the processing of the flowchart shown in Fig. 3. If the control unit 210 determines to end the processing of the flowchart shown in Fig. 3, it ends the processing of the flowchart shown in Fig. 3. If the control unit 210 determines not to end the processing of the flowchart shown in Fig. 3, the process returns to step S320.

[0050] According to the first embodiment, the stress level and stereopsis of the user 10 can be determined without collecting various data such as medical examination data, stress check data, work data, personnel data, salary data, lifestyle data, medical data, company data, etc. Furthermore, since there is no need to collect and learn various data, the stress level and stereopsis can be determined immediately without requiring learning time. Furthermore, the function of determining the stress level and stereopsis of the user 10 can be realized at low cost.

[0051] (Variation 1) Variation 1 of embodiment 1 will be described. Variation 1 will mainly describe the differences from embodiment 1. The same applies to the following variations. In variation 1, the control unit 210 inputs information about the user's gaze position obtained in step S520 into the trained model. The trained model is a trained model that has been trained using information about the user's gaze position with respect to the pictorial images of the first category and the pictorial images of the second category as input data and a value indicating the level of stress as output data. The control unit 210 obtains information about the user's stress by acquiring data indicating the level of stress from the trained model.

[0052] Modification 1 also makes it possible to determine the stress level of user 10 without collecting a variety of data. Furthermore, since the amount of data to be learned is smaller than in Patent Document 1, it is possible to shorten the learning time and reduce costs. Furthermore, by learning information on the user's gaze position, not only on whether the user looked more frequently at the first category pictorial images or the second category pictorial images, but also on where the user looked at each pictorial image as input data, it becomes possible to obtain a more precise value indicating the stress level.

[0053] (Variation 2) Variation 2 of embodiment 1 will be described. The control unit 210 of variation 2 inputs the position of the user's gaze and the position of an object image displayed among a plurality of object images so as to be distinguishable from other object images to the trained model. The trained model is a trained model that uses the position of the gaze of the user attempting to view an object image for sale displayed in a three-dimensional virtual space image (e.g., x, y, z coordinates in three-dimensional space) and the position of the object image displayed among the plurality of object images in the three-dimensional virtual space image so as to be distinguishable from other object images (e.g., x, y, z coordinates in three-dimensional space) as input data, and information about the user's stereopsis as output data. The control unit 210 obtains information about the user's stereopsis from the trained model.

[0054] Modification 2 also makes it possible to determine the depth perception of the user 10 without collecting various data. Furthermore, since the amount of data to be learned is smaller than in Patent Document 1, it is possible to shorten the learning time and reduce costs.

[0055] (Variation 3) Variation 3 of embodiment 1 will be described. In variation 3, the control unit 210 may store information about the stereopsis of the user 10 together with information about the date and time when the stereopsis was calculated as history in the storage unit 220, etc. Then, when outputting the stereopsis in step S750, the control unit 210 may output it together with the past history.

[0056] By outputting the data together with past history, it is possible to check the progress of depth perception.

[0057] (Variation 4) Variation 4 of embodiment 1 will be described. In the processing of the flowchart shown in FIG. 7 , if the distance in the three-dimensional virtual space between the gaze position and an object image displayed in a manner distinguishable from other object images is within a predetermined distance, the control unit 210 of Variation 4 calculates the time it takes from the display of the selected object image until the calculated distance falls within the predetermined distance, and stores the calculated time in a predetermined storage area such as the storage unit 220 in association with distance information. If the time is stored in association with the distance information in the predetermined storage area such as the storage unit 220, the control unit 210 calculates the dynamic visual acuity of the user 10 based on the stored time. If the time information is not stored in association with the distance information, the control unit 210 can determine that the user 10 was unable to gaze at the object image displayed in a manner distinguishable from other object images within the three-dimensional virtual space within the predetermined time. Furthermore, if the time information is stored in association with the distance information, the control unit 210 can determine that the user 10 was able to gaze at the object image displayed in a manner distinguishable from other object images within the three-dimensional virtual space within the predetermined time. The control unit 210 can calculate how many times the user 10 was able to gaze at an object image that is displayed in a manner that is distinguishable from other object images and the average time it took for the user 10 to gaze at the object image, based on information stored in a predetermined storage area such as the storage unit 220. The control unit 210 calculates the dynamic visual acuity of the user 10 based on the calculated information.

[0058] The control unit 210 outputs the obtained information on the dynamic visual acuity of the user 10. The control unit 210 may store the obtained information on the dynamic visual acuity of the user 10 together with the date and time information when the dynamic visual acuity was obtained as history in the storage unit 220 or the like. Then, when the control unit 210 outputs the dynamic visual acuity in step S750, it may output it together with the past history.

[0059] According to the fourth modification, the dynamic visual acuity of the user 10 can be calculated without collecting various data. Furthermore, the amount of data to be learned is smaller than in Patent Document 1, which reduces learning time and costs. Furthermore, by outputting the results together with past history, the progress of dynamic visual acuity can be confirmed.

[0060] (Variation 5) Variation 5 of the first embodiment will be described. A single HMD 100 may be shared by multiple users. In such a usage method, the user reads a two-dimensional code or the like attached to the HMD 100 with the user's smartphone or the like. User information may then be sent from the smartphone to the HMD 100, making it possible to identify which user is logged in to the HMD 100 system. When multiple users share the HMD 100, the above-mentioned information (for example, stereoscopic vision, stress level, etc.) is stored in the storage unit 220 or the like for each user.

[0061] (Variation 6) Variation 6 of the first embodiment will be described. In Variation 6, a server device that acquires information from the HMD 100 may perform all or part of the processing performed by the HMD 100 described in the first embodiment, including the above-mentioned variations. By performing processing on the server device side rather than on the client device side, the load on the client side (i.e., the HMD 100) is reduced. Furthermore, software upgrades and software modifications can be performed on the server device side, allowing for quick identification of the cause of a problem and recovery. The server device has at least a control unit, a storage unit, and a communication unit as hardware components. The control unit is a CPU or the like, and realizes the functions of the server device by executing processing based on a program stored in the storage unit. The storage unit stores programs and data used when the control unit executes processing based on the program. The communication unit connects the server device to a network or the like and manages communication with other devices.

[0062] The server device may store data resulting from the execution of the HMD 100 (e.g., stress check results and vision training results) in a memory unit or the like of the server device in association with identification information identifying the user. The control unit of the server device may perform analysis based on the data resulting from the execution of the HMD 100 stored in the memory unit or the like, generate analysis results, and make the analysis results available for viewing from the HMD 100. The control unit of the server device analyzes the data and generates, for example, a graph showing changes over a predetermined number of days (e.g., per week), or a graph showing how the data resulting from the execution of the HMD 100 has changed in association with biometric data (e.g., ) related to the user, and stores the graph in a memory unit or the like. Note that the biometric data is measured and acquired by various sensors included in the HMD 100 and / or various sensors capable of communicating with the HMD 100 via a network, and transmitted to the server device. The various sensors may include, for example, a sensor for measuring body temperature, a sensor for measuring pulse rate, a sensor for measuring respiratory rate, a sensor for measuring brain waves, a sensor for measuring the temperature around the user, a sensor for measuring sounds around the user, etc. Furthermore, if the results of vision training are better than the average value of past vision training, the control unit of the server device may store the sensor information obtained when the good results were obtained. The stored sensor information may then be transmitted to the HMD 100 and the sensor (or a device that controls the sensor, etc.). With this configuration, it is possible to recreate, for example, an environment around the user when the user is in a zone and has achieved good results.

[0063] (Variation 7) Variation 7 of embodiment 1 will be described. In the vision training process, the control unit 210 of variation 7 determines the user's gaze point based on the captured image, and determines that the image is in focus if the gaze point hits an image (index image) that serves as an index displayed in the three-dimensional virtual space image. The control unit 210 moves the index image and determines the user's dynamic visual acuity based on the position and speed of the moved index image and information about the user's gaze (position and speed of the gaze point). The control unit 210 also moves the index image and determines the user's stereopsis based on the position and speed of the moved index image in the depth direction in the virtual space and information about the user's gaze in the depth direction (position and speed of the gaze point). Variation 7 also makes it possible to determine the user's dynamic visual acuity and stereopsis.

[0064] (Variation 8) Variation 8 of Embodiment 1 will be described. In Variation 8, the control unit 210 may repeatedly execute step S710 of the vision training process to display a selected object image (e.g., a ball or a shuttlecock) so that it moves sequentially from the depth direction to the front direction of the three-dimensional virtual space. When sequentially moving the selected object image from the depth direction to the front direction of the three-dimensional virtual space, the control unit 210 may move it horizontally and also vertically. The control unit 210 then determines the user's stereoscopic acuity and kinetic acuity when the selected object image is moved horizontally. Similarly, the control unit 210 may determine the user's stereoscopic acuity and kinetic acuity when the selected object image is moved horizontally. The control unit 210 may then compare the stereoscopic acuity and kinetic acuity in the horizontal direction with the stereoscopic acuity and kinetic acuity in the vertical direction to analyze whether they are balanced, etc. The control unit 210 may compare the horizontal stereoscopic acuity and dynamic visual acuity with the vertical stereoscopic acuity and dynamic visual acuity, and if it determines that one of the values ​​of stereoscopic acuity and dynamic visual acuity is lower than the other, it may increase the movement of the lower value (for example, vertical movement of the selected image or horizontal movement of the selected image) during the next vision training. In other words, the control unit 210 may change the vertical and horizontal movement of the selected image (such as a ball or a shuttlecock) during vision training based on the past values ​​of the vertical and horizontal stereoscopic acuity and dynamic visual acuity.

[0065] According to the eighth modification, it is possible to determine the depth perception and dynamic visual acuity for vertical and horizontal movements, and if the performance for one of the movements is poor, the next vision training can be performed by increasing the movement of the ball, shuttlecock, etc. that is performing poorly. For example, if the ball is a baseball and the user is a baseball player, if the depth perception and dynamic visual acuity for the vertical movement of the ball are poor, the depth perception and dynamic visual acuity for the vertical movement of the ball can be trained. Note that the control unit 210 may determine which of the vertical and horizontal movements to increase in response to the user's input operation via the input unit 240.

[0066] <Additional Notes> The invention may be provided in the following aspects.

[0067] (1) An information processing system that controls the display of a virtual space image on a display unit visible to a user, sequentially displays combinations of painting images of a first category and painting images of a second category different from the first category in the virtual space image, and obtains information about the user's stress based on information about the user's gaze position with respect to the sequentially displayed painting images of the first category and the second category.

[0068] (2) In the information processing system described in (1) above, information on the position of the user's gaze is input into a trained model, and the trained model is a trained model that uses information on the position of the user's gaze regarding the painting image of the first category and the painting image of the second category as input data and a value indicating the degree of stress as output data, and information regarding the user's stress is obtained by obtaining the value indicating the degree of stress from the trained model.

[0069] (3) The information processing system according to (1) above, wherein the information processing system outputs information relating to the stress.

[0070] (4) In the information processing system described in (1) above, a plurality of graphical user interface components are displayed on the virtual space image, and the plurality of graphical user interface components include a graphical user interface component related to a stress check. When the graphical user interface component related to the stress check is selected from the plurality of graphical user interface components based on information on the position of the user's gaze, a combination of a pictorial image of the first category and a pictorial image of the second category is sequentially displayed on the virtual space image.

[0071] (5) In the information processing system described in (4) above, the plurality of graphical user interface components include a graphical user interface component related to vision training, and when the graphical user interface component related to vision training is selected from the plurality of graphical user interface components based on information about the position of the user's gaze, object images are displayed sequentially in the virtual space image, and information about the user's stereoscopic vision is obtained based on the position of the user's gaze when attempting to view the object image and the position of the object image, and the virtual space image is a three-dimensional virtual space image.

[0072] (6) In the information processing system described in (5) above, the position of the user's gaze and the position of the object image are input into a trained model, and the trained model is trained using the position of the user's gaze when attempting to view an object image for sale displayed in a three-dimensional virtual space image and the position of the object image in the three-dimensional virtual space image as input data, and information regarding the user's stereopsis as output data, and information regarding the user's stereopsis is obtained from the trained model to determine information regarding the user's stereopsis.

[0073] (7) An information processing system according to (5) above, which outputs information about the user's stereoscopic vision.

[0074] (8) An information processing method executed by an information processing system, which controls the display of a virtual space image on a display unit visible to the user, sequentially displays combinations of painting images of a first category and painting images of a second category different from the first category in the virtual space image, and obtains information regarding the user's stress based on information about the user's gaze position with respect to the sequentially displayed painting images of the first category and the painting images of the second category.

[0075] (9) A program for causing a computer to function as the information processing system according to any one of (1) to (7) above. Of course, this is not a limitation.

[0076] For example, the camera 120 may execute part or all of the processing of the HMD 100. The control unit 210 may implement any combination of the first embodiment and the modifications described above.

[0077] Finally, while various embodiments of the present invention have been described, these are presented by way of example only and are not intended to limit the scope of the invention. The novel embodiments may be embodied in various other forms, and various omissions, substitutions, and modifications may be made without departing from the spirit of the invention. The embodiments and their modifications are intended to be included within the scope and spirit of the invention, as well as within the scope of the inventions and their equivalents as defined in the appended claims.

Claims

1. An information processing system comprising: a control unit, the control unit sequentially displays a combination of a first pictorial image and a second pictorial image on a display unit visible to a user, wherein the first pictorial image belongs to a first category, and the second pictorial image belongs to a second category different from the first category, the control unit inputs information regarding the user's gaze position to a trained model, wherein the information regarding the gaze position includes information regarding the number of times the user looked at either the first pictorial image or the second pictorial image more, the trained model being a trained model that uses the information regarding the gaze position as input data and learns a value indicating a level of stress as output data, and the control unit outputs the information regarding the stress of the user by obtaining the value indicating the level of stress from the trained model.

2. An information processing system as described in claim 1, wherein the control unit causes the display unit to display graphical user interface components, wherein the graphical user interface components include graphical user interface components relating to a stress check, and when the graphical user interface component relating to the stress check is selected from the graphical user interface components based on information about the user's gaze position, the control unit causes the display unit to sequentially display combinations of the first pictorial image and the second pictorial image.

3. An information processing system according to claim 1, wherein the information regarding the user's stress is output based on information regarding the position of the user's gaze, which is obtained based on an image captured by an imaging unit.

4. An information processing method executed by an information processing system, comprising, as steps, each process executed by the control unit of the information processing system according to any one of claims 1 to 3.

5. A recording medium on which a program is recorded, the recording medium recording a program for causing a computer to function as the information processing system according to any one of claims 1 to 3.

Images