Information processing system, information processing method, and program

Abstract

To provide a device which acquires a large volume of data for learning, performs learning and result evaluation for improving accuracy, and reduces costs.SOLUTION: An information processing system is provided. The information processing system performs control so as to display a virtual space image on a display unit visible by a user. The system successively displays in the virtual space image the combination of a picture image of a first category and a picture image of a second category different from the first category. The system obtains information on a user's stress based on information of the position of the visual line of the user concerning the picture image of the first category and the picture image of the second category displayed successively.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 discloses a health care system that supports the assessment and prediction of items related to the health of members. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-080198 Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Document 1, a lot of learning data is required, such as medical examination data, stress check data, work data, personnel data, salary data, lifestyle data, medical data, and company data. Generally, a large amount of data is required for learning. In addition, in order to improve accuracy, learning and evaluation of the results are necessary, which is costly. [Means for solving the problem]

[0005] According to one aspect of the present invention, there is provided an information processing system. In this information processing system, a virtual space image is controlled to be displayed on a display unit visible to a user. A combination of a first category of painting images and a second category of painting images different from the first category is sequentially displayed in the virtual space image. Information regarding the user's stress is obtained based on information regarding the position of the user's gaze with respect to the sequentially displayed first category painting images and second category painting images. [Brief description of the drawings]

[0006] [Figure 1]FIG. 1 is a diagram showing an example of a user wearing an HMD. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of the HMD. [Diagram 3] FIG. 3 is a main flowchart showing an example of information processing in the HMD. [Figure 4] FIG. 4 is a diagram showing an example of a three-dimensional virtual space image displayed after login. [Diagram 5] FIG. 5 is a flowchart showing an example of a stress check process. [Figure 6] FIG. 6 is a diagram showing an example in which a set of combinations of a pictorial image of a first category and a pictorial image of a second category are displayed in sequence in a three-dimensional virtual space image. [Figure 7] FIG. 7 is a flowchart showing an example of a vision training process. [Figure 8] FIG. 8 is a diagram showing an example in which at least one object image is selected from a plurality of object images displayed in a three-dimensional virtual space image and is displayed so as to be distinguishable from the other object images. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described with reference to the drawings. Various characteristic features shown in the following embodiments can be combined with each other.

[0008] In this specification, the term "unit" may include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In addition, while various information is handled in the first embodiment, this information is represented by high and low signal values ​​as a binary bit collection consisting of 0 or 1, and communication and calculation can be performed on the circuit in the broad sense.

[0009] In addition, a circuit in the broad sense is a circuit realized by at least appropriately combining a circuit, circuitry, a processor, a memory, etc. In other words, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0010] <Embodiment 1> 1. HMD Fig. 1 is a diagram showing an example of a user 10 wearing a head mounted display (HMD) 100. As shown in Fig. 1, in the first embodiment, a description will be given on the assumption that the user 10 wears the HMD 100 and the HMD 100 performs processing, etc., which will be described later.

[0011] Here, the information processing system described in the claims may be composed of multiple devices or may be composed of one device. When the information processing system described in the claims is composed of one device, an example of the device is the HMD 100. When the information processing system described in the claims is composed of multiple devices, an example of the multiple devices is the HMD 100 and a server device capable of communicating with the HMD 100, etc.

[0012] In the first embodiment, the HMD 100 will be described as an example, but any device may be used as long as it allows the user 10 to perform actions in a three-dimensional virtual space constructed on the Internet. Another example of the HMD 100 is smart glasses.

[0013] 2. Hardware Configuration FIG. 2 is a diagram illustrating an example of a hardware configuration of the HMD 100. As shown in FIG. As shown in FIG. 2, the HMD 100 includes, as its hardware configuration, a control unit 210, a storage unit 220, an imaging unit 230, an input unit 240, an output unit 250, a communication unit 260, a gyro sensor 270, and an acceleration sensor 280.

[0014] The control unit 210 is a CPU (Central Processing Unit) or the like, and controls the entire HMD 100 .

[0015] The storage unit 220 is any one of a hard disk drive (HDD), a read only memory (ROM), a random access memory (RAM), a solid state drive (SSD), etc., or an arbitrary combination thereof, and stores a program and data used when the control unit 210 executes processing based on the program. The storage unit 220 is an example of a storage medium. In the specification, data used when the control unit 210 executes processing based on the program (e.g., a three-dimensional virtual space image, a set of painting images of a first category painting image and a second category painting image, an object image, a threshold value, a setting value, etc., which will be described later) is described as being stored in the storage unit 220, but may be stored in a storage unit of another device that can communicate with the HMD 100. In other words, the data may be stored in a storage unit of any device as long as the control unit 210 can refer to the data. The control unit 210 executes processing based on the program stored in the storage unit 220, thereby realizing the functions of the HMD 100 and the information processing of the flowcharts shown in Figures 3, 5, and 7, which will be described later.

[0016] The imaging section 230 captures an image of the user 10 and generates a captured image. Both eyes of the user 10 are included in the captured image.

[0017] The input unit 240 is a button or the like provided on the HMD 100 and inputs operation information of the user 10 to the HMD 100 . The output unit 250 is a display or the like, and displays a three-dimensional virtual space image or the like generated by the control unit 210. The output unit 250 is an example of a display unit. The communication unit 260 connects the HMD 100 to a network and controls communication with other devices.

[0018] The gyro sensor 270 is a sensor that detects the rotation and change in orientation of the HMD 100 as an angular velocity, and outputs an electrical signal to the control unit 210 and the like. The acceleration sensor 280 is a sensor that detects the inertial force that occurs when the moving speed of the HMD 100 changes, and outputs the detected force as an electrical signal as acceleration.

[0019] The hardware configuration of the HMD 100 includes two lenses in addition to those shown in Fig. 2. The left eye of the user 10 sees the virtual space image through the left lens, and the right eye sees the virtual space image through the right lens.

[0020] 3. Information Processing The information processing of the first embodiment will be described below.

[0021] (1) Main Flowchart FIG. 3 is a main flowchart showing an example of information processing in the HMD 100. When the user 10 logs in to the system, in step S310, the control unit 210 executes a calibration process. As the calibration process, the control unit 210 determines the position of the eyeball of the user 10 based on a captured image including the iris or pupil of the user 10, and corrects the horizontal and vertical misalignment between the HMD 100 and the head of the user 10.

[0022] In step S320, control unit 210 generates a three-dimensional virtual space image and performs control to display it on output unit 250. Control unit 210 determines the line of sight of user 10 based on the captured image captured by imaging unit 230, generates a three-dimensional virtual space image according to the line of sight, and performs control to display it on output unit 250. The process of step S320 is an example of a process of controlling to display a virtual space image on a display unit visible to the user.

[0023] FIG. 4 is a diagram showing an example of a three-dimensional virtual space image displayed after login. FIG. 4 shows an example in which a person is drawn, but the person is only an image. In reality, the user 10 is in a three-dimensional virtual space via the output unit 250 of the HMD 100, and visually watches images and the like displayed in the three-dimensional virtual space according to the line of sight. Multiple guide boards are displayed in the air in the three-dimensional virtual space image. In the example of FIG. 4, a guide board 410 for a stress check and a guide board 420 for vision training are displayed as examples of multiple guide boards. The multiple guide boards are an example of multiple graphical user interface components. The guide board 410 for a stress check is an example of a graphical user interface component related to a stress check. The guide board 420 for vision training is an example of a graphical user interface component related to vision training.

[0024] That is, the process of step S320 is an example of a process for displaying a plurality of graphical user interface components in a virtual space image. The plurality of graphical user interface components include graphical user interface components related to stress checks. In the example of FIG. 4, a stress check information board 410 and a vision training information board 420 are displayed as examples of the plurality of information boards, but other information boards may be included. However, in the following, in order to simplify the explanation, the explanation will be given using the stress check information board 410 and the vision training information board 420 as examples of the plurality of information boards.

[0025] In step S330, the control unit 210 determines which of the displayed guide boards has been selected based on the line of sight of the user 10. For example, when the control unit 210 determines that the gaze area of ​​the user 10 is on the stress check guide board 410 based on the line of sight of the user 10, it determines that the stress check has been selected, and when the control unit 210 determines that the gaze area of ​​the user 10 is on the vision training guide board 420, it determines that the vision training has been selected. When the control unit 210 determines that the stress check has been selected, it advances the process to step S340, and when it determines that the vision training has been selected, it advances the process to step S350.

[0026] As another example, the control unit 210 may determine which guide board has been selected depending on where the gaze of the user 10 is located when an input operation is received via the input unit 240. When the control unit 210 determines that the gaze area of ​​the user 10 is on the stress check guide board 410 when an input operation is received via the input unit 240, it determines that the stress check has been selected, and when the control unit 210 determines that the gaze area of ​​the user 10 is on the vision training guide board 420, it determines that the vision training has been selected.

[0027] In step S340, the control unit 210 executes a stress check process.

[0028] (2) Stress check flow chart Details of the stress check process will be described with reference to the flowchart shown in Fig. 5. Fig. 5 is a flowchart showing an example of the stress check process. In step S510, the control unit 210 displays a combination of a first category painting image and a second category painting image different from the first category in the three-dimensional virtual space image. The first category painting image is, for example, a painting image drawn by a person who is not depressed. The second category painting image is, for example, a painting image drawn by a person who is depressed.

[0029] In step S520, control unit 210 determines the gaze position of user 10 based on a captured image including both eyes of user 10. Control unit 210 determines whether the gaze position is more prevalent among the pictorial images in the first category or the pictorial images in the second category, and stores the determined result in a predetermined storage area such as storage unit 220.

[0030] In step S530, the control unit 210 determines whether the processes of steps S510 and S520 have been performed a predetermined number of times. If the control unit 210 determines that the processes of steps S510 and S520 have been performed a predetermined number of times, the control unit 210 advances the process to step S540. If the control unit 210 determines that the processes of steps S510 and S520 have not been performed a predetermined number of times, the control unit 210 returns the process to step S510 and displays the next combination of a first category painting image and a second category painting image different from the first category in the three-dimensional virtual space image. The predetermined number of times is, for example, five times. However, this does not limit the first embodiment. Information on the predetermined number of times may be written in a predetermined file in the storage unit 220 or the like.

[0031] The process of executing the processes of steps S510 and S520 a predetermined number of times is an example of a process of sequentially displaying in the virtual space image combinations of painting images of a first category and painting images of a second category different from the first category. The process in which the control unit 210 determines that the stress check information board 410 has been selected on the screen as shown in FIG. 4, proceeds to step S340, and repeatedly executes steps S510 and S520 is an example of a process in which, when a graphical user interface component related to a stress check is selected from among multiple graphical user interface components based on information about the user's gaze position, a combination of a first category painting image and a second category painting image is displayed sequentially in the virtual space image.

[0032] FIG. 6 is a diagram showing an example in which a set of combinations of a pictorial image of a first category and a pictorial image of a second category are displayed in sequence in a three-dimensional virtual space image. Image 610 is a three-dimensional virtual space image. Painting image 611 and painting image 612 are displayed in image 610. Painting image 611 is a painting image of a first category. Painting image 612 is a painting image of a second category. Both painting images are images of landscapes. For example, painting image 611 and painting image 612 are displayed by the first processing of step S510 by control unit 210. Control unit 210 determines whether the gaze position of user 10 is more likely to be on painting image 611 or on painting image 612, and stores the determined result in a predetermined storage area such as storage unit 220.

[0033] Image 620 is a three-dimensional virtual space image. After a predetermined time has elapsed since image 610 was displayed, image 620 is displayed. Painting image 621 and painting image 622 are displayed in image 620. Painting image 621 is a painting image of the second category. Painting image 622 is a painting image of the first category. For example, painting image 621 and painting image 622 are displayed by the second processing of step S510 by control unit 210. Control unit 210 determines whether the gaze position of user 10 is more likely to be on painting image 621 or on painting image 622, and stores the determined result in a predetermined storage area such as storage unit 220.

[0034] Image 630 is a three-dimensional virtual space image. After a predetermined time has elapsed since image 620 was displayed, image 630 is displayed. Painting image 631 and painting image 632 are displayed in image 630. Painting image 631 is a painting image of a first category. Painting image 632 is a painting image of a second category. For example, painting image 631 and painting image 632 are displayed by the third processing of step S510 by control unit 210. Control unit 210 determines whether the gaze position of user 10 is more likely to be on painting image 631 or on painting image 632, and stores the determined result in a predetermined storage area such as storage unit 220.

[0035] In step S540, the control unit 210 obtains a stress level based on information on which of the first category painting images and the second category painting images was viewed more in each set of painting images, which is stored in a predetermined storage area such as the storage unit 220. The stress level is a value indicating the user's stress level. The stress level is an example of information on stress. For example, the control unit 210 obtains the stress level of the user 10 based on information on the number of times that the user viewed which of the first category painting images and the second category painting images was viewed more and reference information. The reference information is information that associates information on the number of times that the user viewed which of the first category painting images and the second category painting images was viewed more and the stress level, which is stored in the storage unit 220 or the like. The process of step S540 is an example of a process of obtaining information on the user's stress based on information on the position of the user's gaze with respect to the painting images of the first category and the painting images of the second category that are displayed sequentially.

[0036] In step S550, control unit 210 outputs the stress level calculated in step S540. Displaying the stress level by control unit 210 on output unit 250 is one example of output. Writing the stress level by control unit 210 in a file or the like and storing it in memory unit 220 or the like is also one example of output. Transmitting data including the stress level by control unit 210 to another device via communication unit 260 is also one example of output. The processing of step S550 is one example of processing to output information related to stress.

[0037] According to the process shown in Fig. 5, the stress level of user 10 can be calculated without collecting various data such as medical examination data, stress check data, work data, personnel data, salary data, lifestyle data, medical data, company data, etc. Also, since there is no need to collect and learn various data, the stress level can be calculated immediately without learning time. Also, the function of calculating the stress level of user 10 can be realized at low cost.

[0038] In step S350 in FIG. 3, control unit 210 executes a vision training process.

[0039] (3) Vision training flowchart Details of the vision training process will be described with reference to the flowchart shown in Fig. 7. Fig. 7 is a flowchart showing an example of the vision training process. In step S710, the control unit 210 displays a vision training image in the three-dimensional virtual space image. The vision training image is a plurality of object images displayed in a three-dimensional virtual space image in a three-dimensional arrangement. The control unit 210 displays at least one object image among the plurality of object images in a manner that is distinguishable from other object images. Each time the control unit 210 executes the process of step S710, the control unit 210 randomly selects at least one object image among the plurality of object images and displays it in a manner that is distinguishable from other object images. An example of a distinguishable display is to display it in a color different from other object images. Note that this is only one example, and any display may be used as long as it can be distinguished from other object images. Other examples of a distinguishable display include displaying it in a thick frame, displaying it by blinking, displaying other object images in a different shape, displaying the selected object image and not displaying other object images, and various other display methods.

[0040] In step S720, the control unit 210 obtains the gaze position of the user 10 based on the captured image including both eyes of the user 10. More specifically, the control unit 210 obtains the position of the gaze point of the user 10 based on the captured image. Note that, for simplicity of explanation, the position of the gaze point will be simply referred to as the gaze position below. The control unit 210 obtains the distance in the three-dimensional virtual space between the gaze position and the object image displayed so as to be distinguishable from other object images. Then, the control unit 210 stores information on the obtained distance in a predetermined storage area such as the storage unit 220.

[0041] In step S730, the control unit 210 judges whether the processes of steps S710 and S720 have been executed a predetermined number of times. When the control unit 210 judges that the processes of steps S710 and S720 have been executed a predetermined number of times, the control unit 210 advances the process to step S740. When the control unit 210 judges that the processes of steps S510 and S520 have not been executed a predetermined number of times, the control unit 210 returns the process to step S710, randomly selects at least one object image from among the multiple object images displayed in a three-dimensional array in the three-dimensional virtual space image, and displays the selected object image in a manner distinguishable from other object images. The predetermined number of times is, for example, five times. However, this does not limit the first embodiment. Information on the predetermined number of times may be written in a predetermined file in the storage unit 220 or the like.

[0042] The process in which control unit 210 determines that vision training guide board 420 has been selected on a screen such as that shown in FIG. 4, proceeds to step S350, and repeatedly executes steps S710 and S720 is an example of a process in which object images are displayed sequentially in the virtual space image when a graphical user interface component related to vision training is selected from among multiple graphical user interface components based on information about the position of the user's gaze.

[0043] FIG. 8 is a diagram showing an example in which at least one object image is selected from a plurality of object images displayed in a three-dimensional virtual space image and is displayed so as to be distinguishable from the other object images. Image 810 is a three-dimensional virtual space image. In image 810, object image 811 selected by the control unit 210 in the first processing of step S710 is displayed so as to be distinguishable from other object images. Image 812 is an image showing the position of the gaze of user 10. Image 812 is shown for the purpose of explanation, and is not displayed in the actual virtual space image. The object image is not limited to that shown in FIG. 8. Any image may be used as long as it can show an object to exist in the virtual space. Other examples of object images include balls and shuttlecocks in sports (baseball ball, volleyball ball, basketball ball, tennis ball, table tennis ball, badminton shuttlecock), etc. By repeatedly executing step S710, the control unit 210 may display the selected object image (for example, a ball or a shuttlecock) so as to move sequentially from the depth direction to the front direction in the three-dimensional virtual space.

[0044] Image 820 is a three-dimensional virtual space image. After a predetermined time (for example, one second) has elapsed since image 810 was displayed, image 820 is displayed. In image 820, object image 821 selected by control unit 210 in the second processing of step S710 is displayed in a manner distinguishable from other object images. Image 822 is an image indicating the position of the line of sight of user 10. Image 822 is shown for the purpose of explanation, and is not actually displayed in the virtual space image.

[0045] Image 830 is a three-dimensional virtual space image. After a predetermined time has elapsed since image 820 was displayed, image 830 is displayed. In image 830, object image 831 selected by control unit 210 in the third processing of step S710 is displayed in a manner distinguishable from other object images. Image 832 is an image indicating the position of the line of sight of user 10. Image 832 is shown for the purpose of explanation, and is not actually displayed in the virtual space image.

[0046] In step S740, the control unit 210 calculates the stereoscopic vision based on information about the distance between each selected object image and the gaze position of the user 10, which information is stored in a predetermined storage area such as the storage unit 220. The process of step S740 is an example of a process for calculating information about the stereoscopic vision of the user based on the gaze position of the user who is trying to view the object image and the position of the object image.

[0047] In step S750, control unit 210 outputs the stereoscopic vision calculated in step S740. Displaying the stereoscopic vision by control unit 210 on output unit 250 is one example of output. Writing the stereoscopic vision by control unit 210 in a file or the like and storing it in memory unit 220 or the like is also one example of output. Transmitting data including the stereoscopic vision by control unit 210 to another device via communication unit 260 is also one example of output. The processing of step S750 is one example of processing to output information related to the user's stereoscopic vision.

[0048] According to the process shown in Fig. 7, the user's stereoscopic vision can be calculated without collecting various data such as medical examination data, stress check data, work data, personnel data, salary data, lifestyle data, medical data, company data, etc. Also, since there is no need to collect and learn various data, the stereoscopic vision can be calculated immediately without learning time. Also, the function of calculating the stereoscopic vision of the user 10 can be realized at low cost. The control unit 210 may perform the calibration process again immediately before performing the process of step S710. By doing so, it is possible to eliminate deviations and obtain the user's stereoscopic vision and the like with higher accuracy.

[0049] In step S360 in Fig. 3, control unit 210 determines whether or not to end the processing of the flowchart shown in Fig. 3. For example, when the user's line of sight is on an image displayed in a three-dimensional virtual space image for a predetermined period of time, control unit 210 determines to end the processing of the flowchart shown in Fig. 3. When control unit 210 determines to end the processing of the flowchart shown in Fig. 3, it ends the processing of the flowchart shown in Fig. 3. When control unit 210 determines not to end the processing of the flowchart shown in Fig. 3, it returns the processing to step S320.

[0050] According to the first embodiment, the stress level and stereopsis of the user 10 can be obtained without collecting various data such as medical examination data, stress check data, work data, personnel data, salary data, lifestyle data, medical data, company data, etc. Also, since it is not necessary to collect and learn various data, the stress level and stereopsis can be obtained immediately without learning time. Also, the function of obtaining the stress level and stereopsis of the user 10 can be realized at low cost.

[0051] (Variation 1) Modification 1 of embodiment 1 will be described below. In modification 1, differences from embodiment 1 will be mainly described. The same applies to the following modifications. Control unit 210 of Modification 1 inputs the information on the position of the user's gaze obtained in step S520 to the trained model. The trained model is a trained model trained using information on the position of the user's gaze with respect to the painting image of the first category and the painting image of the second category as input data and a value indicating the degree of stress as output data. Control unit 210 obtains information on the user's stress by acquiring data indicating the degree of stress from the trained model.

[0052] Modification 1 also makes it possible to obtain the stress level of user 10 without collecting various data. Furthermore, since the amount of data to be learned is less than that of Patent Document 1, it is possible to shorten the learning time and reduce costs. Furthermore, by learning information on the position of the user's gaze, not only on which of the painting images in the first category and the painting images in the second category the user looked at more, but also on which part of each painting image the user looked at as input data, it becomes possible to obtain a value indicating the degree of stress in greater detail.

[0053] (Variation 2) A second modification of the first embodiment will be described. The control unit 210 of the second modification inputs the position of the user's gaze and the position of an object image displayed among the multiple object images so as to be distinguishable from other object images to the trained model. The trained model is a trained model trained using the position of the gaze of the user who is trying to view an object image displayed in a three-dimensional virtual space image (e.g., x, y, z coordinates in three-dimensional space) and the position of the object image displayed among the multiple object images in the three-dimensional virtual space image so as to be distinguishable from other object images (e.g., x, y, z coordinates in three-dimensional space) as input data, and information about the user's stereopsis as output data. The control unit 210 obtains information about the user's stereopsis from the trained model.

[0054] Modification 2 also makes it possible to obtain the stereoscopic vision of the user 10 without collecting various data. In addition, the amount of data to be learned is smaller than that of Patent Document 1, making it possible to reduce the learning time and costs.

[0055] (Variation 3) A third modification of the first embodiment will be described. The control unit 210 of the third modification may store information on the stereoscopic vision of the user 10 together with date and time information on when the stereoscopic vision was calculated as history in the storage unit 220 or the like. Then, when outputting the stereoscopic vision in step S750, the control unit 210 may output the information together with the past history.

[0056] By outputting the data together with past history, it is possible to check the progress of stereoscopic vision.

[0057] (Variation 4) A fourth modification of the first embodiment will be described. In processing the flowchart as shown in FIG. 7, when the distance in the three-dimensional virtual space between the gaze position and an object image displayed so as to be distinguishable from other object images is within a predetermined distance, the control unit 210 of variant example 4 calculates the time taken from when the selected object image is displayed until the calculated distance falls within the predetermined distance, and associates the calculated time with the distance information and stores it in a predetermined memory area such as the memory unit 220. When the time is stored in association with the distance information stored in a predetermined storage area such as the storage unit 220, the control unit 210 calculates the dynamic visual acuity of the user 10 based on the stored time. When the time information is not stored in association with the distance information, the control unit 210 can determine that the user 10 was unable to gaze at an object image displayed in a manner distinguishable from other object images within a predetermined time in a three-dimensional virtual space. When the time information is stored in association with the distance information, the control unit 210 can determine that the user 10 was able to gaze at an object image displayed in a manner distinguishable from other object images within a predetermined time in a three-dimensional virtual space. Based on the information stored in a predetermined storage area such as the storage unit 220, the control unit 210 can calculate how many times the user was able to gaze at an object image displayed in a manner distinguishable from other object images and the average time it took to gaze at the object image. The control unit 210 calculates the dynamic visual acuity of the user 10 based on the calculated information.

[0058] The control unit 210 outputs the obtained information on the dynamic visual acuity of the user 10. The control unit 210 may store the obtained information on the dynamic visual acuity of the user 10 in the storage unit 220 or the like as history together with date and time information on when the dynamic visual acuity was obtained. Then, when the control unit 210 outputs the dynamic visual acuity in step S750, it may output the information together with the past history.

[0059] According to the fourth modification, the dynamic visual acuity of the user 10 can be obtained without collecting various data. Also, the amount of data to be learned is less than that of the patent document 1, so that the learning time and the cost can be reduced. Also, by outputting the data together with the past history, the progress of the dynamic visual acuity can be confirmed.

[0060] (Variation 5) A fifth modification of the first embodiment will be described. A single HMD 100 may be shared by multiple users. In such a usage method, the user reads a two-dimensional code or the like attached to the HMD 100 with the user's smartphone or the like. Then, by transmitting user information from the smartphone to the HMD 100, it may be possible to identify which user is logged in to the HDM 100 system. When the HMD 100 is shared by multiple users, each of the above-mentioned information (e.g., stereoscopic vision, stress level, etc.) is stored in the storage unit 220 or the like for each user.

[0061] (Variation 6) A sixth modification of the first embodiment will be described. In the sixth modification, the server device that acquires information from the HMD 100 may perform all or part of the processing performed by the HMD 100 described in the first embodiment, including the above-mentioned modification. By performing processing on the server device side instead of on the client device side, the load on the client side (i.e., the HMD 100) is reduced. In addition, software upgrades and software corrections can be performed on the server device side, so that the cause of a problem can be quickly identified and recovery can be performed. The server device has at least a control unit, a storage unit, and a communication unit as a hardware configuration. The control unit is a CPU or the like, and realizes the function of the server device by executing processing based on a program stored in the storage unit. The storage unit stores programs and data used when the control unit executes processing based on the programs, and the communication unit connects the server device to a network or the like and controls communication with other devices.

[0062] The server device may store data on the results of the execution of the HMD100 (e.g., the results of the stress check and the results of the vision training) in a storage unit or the like of the server device in association with identification information for identifying the user. The control unit of the server device may perform an analysis based on the data on the results of the execution of the HMD100 stored in the storage unit or the like, generate an analysis result, and make it possible to confirm it from the HMD100. The control unit of the server device analyzes the data, and generates, for example, a graph showing changes for a predetermined number of days (e.g., per week), or a graph showing how the data on the results of the execution of the HMD100 has changed in association with biometric data (e.g., ) related to the user, and stores it in a storage unit or the like. Note that the biometric data is measured and acquired by various sensors included in the HMD100 and / or various sensors capable of communicating with the HMD100 via a network, and transmitted to the server device. The various sensors include, for example, a sensor for measuring body temperature, a sensor for measuring pulse rate, a sensor for measuring the number of breaths, a sensor for measuring brain waves, a sensor for measuring the temperature around the user, a sensor for measuring sounds around the user, and the like. Furthermore, if the results of the vision training are better than the average value of past vision training, the control unit of the server device may store the sensor information when the good results were obtained. The stored sensor information may then be transmitted to the HMD 100 and the sensor (or a device that controls the sensor, etc.). With this configuration, for example, an environment in which the user was in a zone and performed well can be reproduced around the user.

[0063] (Variation 7) A seventh modification of the first embodiment will be described. In the vision training process, the control unit 210 of the seventh modification obtains the user's gaze point based on the captured image, and determines that the image is in focus when the gaze point hits an image (index image) that serves as an index displayed in the three-dimensional virtual space image. The control unit 210 moves the index image, and obtains the user's dynamic visual acuity based on the position and speed of the moved index image and the user's tracking information (position and speed of the gaze point). The control unit 210 also moves the index image, and obtains the user's stereoscopic vision based on the position and speed of the moved index image in the depth direction in the virtual space and the user's tracking information in the depth direction (position and speed of the gaze point). The seventh modification can also determine the user's dynamic visual acuity and stereoscopic vision.

[0064] (Variation 8) An eighth modification of the first embodiment will be described. The control unit 210 of the eighth modified example may display a selected object image (e.g., a ball or a shuttlecock) so as to move sequentially from the depth direction to the front direction of the three-dimensional virtual space by repeatedly executing step S710 of the vision training process. When the control unit 210 moves the selected object image sequentially from the depth direction to the front direction of the three-dimensional virtual space, the control unit 210 may move the selected object image horizontally and vertically. The control unit 210 then obtains the user's stereoscopic vision and kinetic visual acuity when the selected object image is moved horizontally. Similarly, the control unit 210 obtains the user's stereoscopic vision and kinetic visual acuity when the selected object image is moved horizontally. The control unit 210 etc. may then compare the stereoscopic vision and kinetic visual acuity in the horizontal direction with the stereoscopic vision and kinetic visual acuity in the vertical direction, and analyze whether they are balanced, etc. Then, the control unit 210 may compare the stereoscopic acuity and kinetic acuity in the horizontal direction with the stereoscopic acuity and kinetic acuity in the vertical direction, and if it determines that the value of either one of the stereoscopic acuity and kinetic acuity is not better than the other, it may increase the movement of the one with the worse value (for example, the vertical movement of the selected image or the horizontal movement of the selected image) during the next vision training. In other words, the control unit 210 may change the vertical and horizontal movements of the selected image (ball, shuttlecock, etc.) during vision training based on the past values ​​of the stereoscopic acuity and kinetic acuity in the vertical and horizontal directions.

[0065] According to the eighth modified example, it is possible to obtain stereoscopic vision and dynamic visual acuity for vertical and horizontal movements, and if the performance for one of the movements is poor, the next vision training can be trained by increasing the movement of the ball, shuttlecock, etc. for the one with poor performance. For example, if the ball is a baseball and the user is a baseball player, if the stereoscopic vision and dynamic visual acuity for the vertical change of the ball are poor, the stereoscopic vision and dynamic visual acuity for the vertical ball can be trained. Note that, depending on an input operation by the user via the input unit 240, the control unit 210 may determine which of the vertical and horizontal movements should be made more frequent.

[0066] <Additional Notes> The invention may be provided in each of the following aspects:

[0067] (1) An information processing system that controls the display of a virtual space image on a display unit visible to a user, sequentially displays combinations of painting images of a first category and painting images of a second category different from the first category in the virtual space image, and obtains information regarding the user's stress based on information about the user's gaze position with respect to the sequentially displayed painting images of the first category and the painting images of the second category.

[0068] (2) In the information processing system described in (1) above, information on the position of the user's gaze is input into a trained model, and the trained model is a trained model trained using information on the position of the user's gaze with respect to the painting image of the first category and the painting image of the second category as input data and a value indicating the degree of stress as output data, and information regarding the user's stress is obtained by obtaining the value indicating the degree of stress from the trained model.

[0069] (3) The information processing system according to (1) above, which outputs information relating to the stress.

[0070] (4) An information processing system in accordance with the above-mentioned (1), which displays a plurality of graphical user interface components in the virtual space image, the plurality of graphical user interface components including a graphical user interface component related to a stress check, and when the graphical user interface component related to a stress check is selected from the plurality of graphical user interface components based on information on the position of the user's gaze, causes a combination of painting images of the first category and painting images of the second category to be displayed sequentially in the virtual space image.

[0071] (5) In the information processing system described above in (4), the plurality of graphical user interface components include a graphical user interface component related to vision training, and when the graphical user interface component related to vision training is selected from the plurality of graphical user interface components based on information on the position of the user's gaze, object images are displayed sequentially in the virtual space image, and information regarding the user's stereoscopic vision is obtained based on the position of the gaze of the user attempting to view the object image and the position of the object image, and the virtual space image is a three-dimensional virtual space image.

[0072] (6) In the information processing system described above in (5), the position of the user's gaze and the position of the object image are input into a trained model, and the trained model is trained using the position of the gaze of a user attempting to view an object image for sale displayed in a three-dimensional virtual space image and the position of the object image in the three-dimensional virtual space image as input data and information regarding the user's stereoscopic acuity as output data, and information regarding the user's stereoscopic acuity is obtained by obtaining information regarding the stereoscopic acuity from the trained model.

[0073] (7) An information processing system as described in (5) above, which outputs information regarding the user's stereoscopic vision.

[0074] (8) An information processing method executed by an information processing system, comprising: controlling a virtual space image to be displayed on a display unit visible to a user; sequentially displaying combinations of painting images of a first category and painting images of a second category different from the first category in the virtual space image; and determining information regarding the user's stress based on information regarding the position of the user's gaze with respect to the sequentially displayed painting images of the first category and the painting images of the second category.

[0075] (9) A program for causing a computer to function as the information processing system according to any one of (1) to (7) above. Of course, this is not the case.

[0076] For example, some or all of the processing of the HMD 100 may be performed by the camera 120. The control unit 210 may implement the above-described first embodiment and each of the modified examples in any combination.

[0077] Finally, although various embodiments of the present invention have been described, these are presented as examples and are not intended to limit the scope of the invention. The new embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. The embodiments and their modifications are within the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0078] 10: User 100: HMD 120: Camera 210: Control section 220: Storage section 230: Imaging unit 240: Input section 250: Output section 260: Communications Department 270: Gyro sensor 280: Acceleration sensor 410: Information board 420: Information board 610:Image 611: Painting image 612: Painting image 620:Image 621: Painting image 622: Painting image 630:Image 631: Painting image 632: Painting image 810:Image 811: Object image 812:Image 820:Image 821: Object image 822:Image 830:Image 831: Object image 832:Image

Claims

1. An information processing system, Controlling the display of a virtual space image on a display unit visible to a user; sequentially displaying combinations of a first category of painting images and a second category of painting images different from the first category in the virtual space image; determining information about the user's stress based on information about the position of the user's gaze with respect to the first category painting image and the second category painting image that are displayed sequentially; Information processing system.

2. 2. The information processing system according to claim 1, Inputting the information on the user's gaze position into a trained model; The trained model is a trained model trained using information on the position of the user's gaze with respect to the painting image of the first category and the painting image of the second category as input data and a value indicating the degree of stress as output data, Obtaining information about the user's stress by obtaining a value indicating the degree of stress from the trained model; Information processing system.

3. 2. The information processing system according to claim 1, outputting information related to the stress; Information processing system.

4. 2. The information processing system according to claim 1, Displaying a plurality of graphical user interface components on the virtual space image; the plurality of graphical user interface components include a graphical user interface component related to a stress check, when a graphical user interface component relating to the stress check is selected from among the plurality of graphical user interface components based on information on the position of the user's gaze, a combination of a picture image of the first category and a picture image of the second category is sequentially displayed in the virtual space image. Information processing system.

5. 5. The information processing system according to claim 4, the plurality of graphical user interface components include a graphical user interface component related to vision training; When the graphical user interface component related to vision training is selected from among the plurality of graphical user interface components based on information on the position of the user's gaze, object images are sequentially displayed in the virtual space image, and information on the user's stereoscopic acuity is obtained based on the position of the gaze of the user attempting to view the object images and the positions of the object images; The virtual space image is a three-dimensional virtual space image. Information processing system.

6. 6. The information processing system according to claim 5, Inputting the position of the user's gaze and the position of the object image into a trained model; The trained model is a trained model trained using the position of the gaze of a user who is trying to see an object image for sale displayed in a three-dimensional virtual space image and the position of the object image in the three-dimensional virtual space image as input data, and information regarding the user's stereoscopic vision as output data, Obtain information about the user's stereopsis by obtaining information about the stereopsis from the trained model; Information processing system.

7. 6. The information processing system according to claim 5, outputting information regarding the user's stereopsis; Information processing system.

8. An information processing method executed by an information processing system, comprising: Controlling the display of a virtual space image on a display unit visible to a user; sequentially displaying combinations of a first category of painting images and a second category of painting images different from the first category in the virtual space image; determining information about the user's stress based on information about the position of the user's gaze with respect to the first category painting image and the second category painting image that are displayed sequentially; Information processing methods.

9. A program, Computer, A program for causing the information processing system according to any one of claims 1 to 7 to function as such.

Images