Physiological stress of a user of a virtual reality environment

Abstract

A computer-implemented method (100) for detecting physiological stress of a user of a virtual reality environment, wherein the method comprises: Display (310) virtual objects at object positions in the virtual reality environment for the user; Instruction (320) to the user to interact with one of the displayed virtual objects; Capturing (330) at least one parameter of the user’s movement in response to the instruction, wherein the at least one parameter of the movement includes response time or movement speed or movement direction; Determine (340) a measure of physiological stress based on the captured at least one parameter of the user's movement, wherein the determination (340) includes comparing the captured parameters of movement with a reference value associated with a virtual object and / or its object position and / or the user; Obtain (350) historical information relating to previously determined measures of physiological stress and compare a newly determined measure with the historical information to provide a personal baseline measure, including checking or flagging erroneous or deviant values; Identify a target part of the user's body to which no measure of physiological stress is associated and adjust the object position so that the interaction requires the use of the target part; Detect a lack of movement of the target part as an indication of significant physiological stress; Assigning (370) the measure of physiological stress to a body part of the user based on the object position of one of the displayed virtual objects; and Adapting the virtual reality environment based on the measure of physiological stress and the user's body part, wherein the adaptation of the virtual reality environment involves adjusting the object position of one of the displayed virtual objects.

Description

BACKGROUND

[0001] The present invention relates to the detection of physiological stress of a user of a virtual reality environment.

[0002] The provision of virtual reality environments with which users can interact is widely known. Such virtual reality environments, and the devices or systems used to provide them, are enjoying increasing popularity, particularly in the field of computer / electronic games.

[0003] When using virtual reality environments, it is important for users to take regular breaks (e.g., to avoid repetitive strain injuries, physiological damage, and / or fatigue). However, this advice can be overlooked or ignored by users of computer / electronic games, especially when they are deeply immersed in or actively engaged within a virtual environment.

[0004] Furthermore, users in a virtual reality environment often have to interact with virtual objects in three dimensions, from multiple angles, and / or in different orientations relative to their current position. This can be difficult and / or stressful for elderly users and users with physical disabilities and / or injuries.

[0005] Publication US 2018 / 0005443A1 relates to a device. The device comprises: a display configured to show virtual content; an interface communicatively coupled to an input mechanism, configured to capture data associated with the movement of the input mechanism; one or more processors communicatively coupled to the display; and a memory containing computer-executable instructions which, when executed by the one or more processors, cause the device to perform operations that include: receiving initial data associated with an initial movement of the input mechanism, via the interface and from the input mechanism; determining the position of a physical object that can constrain the initial movement of the input mechanism;Analyzing the initial data associated with the first movement of the input mechanism and the location of the physical object to determine at least part of the input mechanism's range of motion; determining a restricted area of ​​the display, where the restricted area comprises part of a display area and interaction with a virtual object represented in the restricted area is limited by the input mechanism's range of motion; setting a parameter that correlates the degree of movement of the input mechanism with the degree of movement of a virtual element represented by the display; receiving secondary data via the interface and from the input mechanism associated with a second movement of the input mechanism;and controlling the virtual element displayed by the screen using the set parameter and based on the second data associated with the second movement of the input mechanism, in order to perform the interaction with the virtual object displayed in the limited area of ​​the screen.

[0006] Document US 2009 / 0124863A1 concerns a procedure for recording a patient's condition. The procedure includes: continuously monitoring a patient's gestures using a video imager; identifying the patient's condition corresponding to at least one clinical factor using at least one monitored gesture; and automatically recording the patient's condition in an electronic medical record.

[0007] Document US 2011 / 0230792A1 relates to a system and procedure for motion assessment used in the medical analysis and treatment of movement limitations. The system is configured to stimulate, monitor, and analyze a person's voluntary movements. A surround display is used to stimulate the person's movement, a motion tracker monitors the person's movements, and a processor receives data from the motion tracker and can be configured to control the surround display.

[0008] Publication JP 2009 - 213 782 A relates to a training support system, a training support procedure, and a computer program that can provide video content suitable for a user's training ability and the individuality of the user's movements. A terminal controller of the training support system determines an area corresponding to each part of the user's body from the images captured by a first and a second camera unit, which use cameras to record images. Based on the position of the determined area in the captured images, the system determines the area corresponding to each part of the user's body in a first display area.The terminal controller determines whether the individual parts of the user's body follow the pattern exercise movement displayed on a screen, and if not, it selects the content to be displayed according to the assessment result, such as reducing the exercise level of the pattern exercise movement, reducing the display speed, and so on.

[0009] Publication US 2019 / 0307384A1 concerns the identification and measurement of body states and feedback systems. A computer system initializes a tracking and measuring device to generate patient data relating to a patient's body state or feedback system. The computer system collects the patient data using the initialized tracking and measuring device and identifies a baseline parameter used to establish a baseline state relative to the collected patient data. The computer system further collects additional patient data using the initialized tracking and measuring device and compares the additionally collected patient data to the identified baseline parameter.If the additionally recorded patient data deviates from the baseline state by a certain threshold, the computer system determines that a change in the patient's physical state or feedback system has occurred, and / or determines the extent of the change in the physical state or feedback system that has occurred. SUMMARY

[0010] The invention is based on the objective of creating a computer-implemented method, a system, and a computer program product for observing a virtual reality environment. This objective is achieved by the features of the corresponding independent claims. The embodiments of the invention are specified in the dependent claims. The computer displays a virtual object to the user at an object position within the virtual reality environment and instructs the user to interact with the displayed virtual object. The computer further acquires at least one parameter of the user's movement in response to the instruction. The computer determines a measure of physiological stress based on the at least one acquired parameter of the user's movement and assigns the measure of physiological stress to a body part of the user based on the object position.

[0011] The above summary is not intended to describe every embodiment or implementation of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings included in this patent application are incorporated into the description and form part of it. They illustrate embodiments of the present invention and, together with the description, serve to explain the basic concepts of the invention. The drawings merely illustrate certain embodiments and do not limit the invention. It should be clear that the figures are drawn schematically and not to scale. Fig. Figure 1 shows a block diagram of an exemplary distributed system in which embodiments of the present invention can be implemented. Fig. Figure 2 is a block diagram of an exemplary system in which embodiments of the present invention can be implemented. Fig. Figure 3 is a flowchart of a method for detecting physiological stress of a user of a virtual reality environment according to embodiments of the present invention. Fig. Figure 4 illustrates a system for detecting physiological stress of a user of a virtual reality environment according to one embodiment.

[0013] Although the invention is open to various modifications and alternative forms, details of which have been shown by way of example in the drawings and are described in detail, it should be clear that the invention is not intended to be limited to the embodiments described in each instance. Rather, the invention is intended to cover all modifications, equivalents, and alternatives that fall within the scope of the invention. DETAILED DESCRIPTION

[0014] In the context of the present invention, where embodiments of the present invention constitute a method, it should be clear that such a method can be a process for execution by a computer, i.e., a method implementable by a computer. The various steps of the method therefore reflect different parts of a computer program, e.g., different parts of one or more algorithms.

[0015] Furthermore, in the context of the present invention, a system can be a single unit or a set of distributed units capable of performing one or more embodiments of the methods of the present invention. For example, a system can be a personal computer (PC), a server, or a set of PCs and / or servers connected via a network such as a local area network, the internet, etc., to jointly perform at least one embodiment of the methods of the present invention.

[0016] This document discloses a concept for detecting physiological stress in a user of a virtual reality environment. Embodiments of the present invention can utilize the nature and / or characteristics of a user's interaction with a virtual object within a virtual reality to provide concepts for identifying areas of the user's body that may be subject to physiological stress or strain. Proposed solutions for detecting physiological stress or strain in a user's body part are thus enabled by the context of a user interacting with a virtual object in a virtual or augmented reality environment (e.g., moving or manipulating it).When a user responds to a command to interact with a virtual object, one or more aspects of the user's movement in response to the command can be captured and used to determine whether the user has experienced physiological stress as a result of the movement. Embodiments of the present invention can therefore provide a solution for the dynamic detection of physiological stress to a user's body while the user is immersed in a virtual reality environment.

[0017] By monitoring body and / or device movements as a user interacts with a virtual object in a virtual or augmented reality environment, information can be obtained that reveals the physiological stress experienced by the user. For example, slow movement of a user's body part can indicate that the body part is experiencing physiological stress or strain. Furthermore, a value of movement speed can be directly (or indirectly) correlated with a value of experienced physiological stress.

[0018] Embodiments can identify a specific body part that is subject to physiological stress. By implementing embodiments of the present invention with respect to several different body parts of a user, a stress map of the user's body can therefore be created and / or updated (for example, by assigning obtained measurements of physiological stress to different body parts of the user). Such information can be managed in a form that allows a graphical representation (e.g., a graphic illustration, chart, or diagram) of a user's body parts and one or more associated measurements of physiological stress.

[0019] Embodiments of the present invention can pursue a dynamic and / or targeted approach to detecting physiological stress on a human body using a virtual reality environment. Body and / or device movements can be captured and monitored as a user interacts with the virtual reality environment, and subsequently, information about the captured and / or monitored movements can be used to derive one or more measures of physiological stress on the user's body. The obtained measure(s) of stress can then be mapped onto one or more body parts to provide body part-specific physiological information to the user. By taking into account a user's reaction time (e.g.,The time it takes the user to move a specific body part in response to an instruction can, for example, provide a measure of the physiological stress the user is under. A slow reaction time (i.e., a long period of time elapses before the user moves in response to an instruction) may indicate significant physiological stress or heavy strain on the moving body part.

[0020] For example, implementations can provide the ability to capture a parameter of the user's movement, such as reaction time, speed, or direction. A captured value of a movement parameter can then be compared to a reference value. Based on the result of such a comparison and taking into account one or more instructions for the user, a measure of the physiological stress on a part of the user's body can be determined. Such an approach can therefore consider a relationship between the user's current movement and a movement reference value (e.g., a previous, recent, or earlier captured movement).

[0021] In some embodiments, the reference value can be associated with the virtual object, the object's position, and / or the user. The process of determining physiological stress can thus be adapted to the specific characteristics of the user, the virtual environment, and / or the virtual object to provide more accurate and / or relevant results.

[0022] However, it is clear that the process of determining a measure of physiological stress based on a captured parameter of user movement, for example, can be adapted or configured by a user.

[0023] In some embodiments, a reference value used by the assessment process can be adjusted or configured by a user. For example, in some embodiments, a reference value can be modified based on an assessed measure of physiological stress. In this way, a reference value can be adjusted or updated to reflect a newly assessed measure of physiological stress, providing a more accurate or relevant value for comparison / assessment purposes.

[0024] Some embodiments may also include the detection of an audible or visual cue from the user in response to the instruction. The determination of a measure of physiological stress may then also be based on the detected audible or visual cue. In this way, embodiments can utilize other sensory inputs and information to provide a more detailed and accurate determination of physiological stress. For example, gasping breaths, groans, or other audible cues indicative of pain may be used to provide additional information for determining a measure of physiological stress. The detection of specific spoken words or phrases (such as "ouch" or "that hurts") may indicate stress or pain when a particular body part is moved. In some embodiments, a user's silence (e.g.,A user who is having a conversation and then stops speaking can be used as an acoustic indicator that the user is experiencing some physiological stress and is concentrating on the task. Established speech recognition techniques and / or a neural network system suitable for detecting and identifying specific sounds or spoken words can be employed to identify such sounds. Such approaches can provide a more detailed and accurate assessment of physiological stress levels.

[0025] Implementations may also include defining a target part of the user's body for which a measure of physiological stress is to be determined. Displaying a virtual object at a specific location within the virtual reality environment for the user can then be based on this target body part. Furthermore, mapping the measure of physiological stress to a user's body part can involve mapping the measure of physiological stress to the target body part. This allows for a dynamic and targeted assessment of physiological stress for different body parts. For example, a stress map of a user's body can be created by examining different body parts using different instructions given to the user (e.g., to identify difficult or hard-to-reach areas / positions).This can be achieved by manipulating the virtual reality environment to obtain information about physiological stress in specific areas or parts of a user's body. Such manipulation of the virtual reality environment can then be mapped to the user's movements (e.g., speed, reaction times, direction of movement, etc.).

[0026] Accordingly, it is clear that various embodiments allow for the adaptation of a virtual reality environment to address different areas / regions of physiological stress that a user may experience. Embodiments can be adapted to target specific areas or parts of a user's body to avoid or reduce physiological stress on a body part and / or to provide rehabilitation or treatment for that body part. For example, based on an identified measure of physiological stress for a user, the placement of a virtual object within a virtual reality environment can be adapted to reduce or minimize physiological stress for the user when interacting with the virtual object.

[0027] Some implementations can adapt and / or manipulate a virtual reality environment to identify different areas of physiological stress. For example, implementations can target specific areas of a user's body to obtain information specific to one or more body parts.

[0028] In some embodiments, specifying a target part of the user's body may involve identifying a body part for which no measure of physiological stress is assigned. This allows for the acquisition of more complete or relevant information regarding physiological stress. For example, it can avoid unnecessary repetition in obtaining information for the same body part and instead allow resources to be adjusted to obtain physiological information that is missing or not present for a user (e.g., not present in the stress map for that user).

[0029] Some embodiments may also include the retrieval of historical information relating to a previously determined measure of physiological stress. The determination of a measure of physiological stress may then also be based on this retrievable historical information. In this way, embodiments can utilize other previously obtained measures and information to provide a more comprehensive and accurate determination of physiological stress.

[0030] Many different approaches can be used to determine a measure of physiological stress based on a captured parameter of the user's movement. However, in some embodiments, a measure of physiological stress can be determined by analyzing a captured movement parameter such as velocity, reaction time, and / or direction of movement. Such an analysis can be based on one or more specific characteristics of how the user moves in response to an instruction to interact with a displayed virtual object. Consequently, in some embodiments, one or more factors relating to how a user interacts with a virtual object can be used to determine a measure of the physiological stress experienced by the user.

[0031] For example, some embodiments can provide extensions to a virtual reality system. Such extensions can ensure that a user receives more effective (e.g., more relevant) treatment. In this way, a user cannot be unduly stressed or treated inappropriately as a result of incorrect or poorly targeted instructions. For example, embodiments can be used to target specific areas of a user's body for virtual treatment methods or to prevent virtual objects from being placed in positions within a virtual environment that are difficult or stressful to reach.

[0032] Illustrative embodiments can therefore provide concepts for controlling the delivery of instructions, physiological treatment, and / or one or more virtual objects to a user of a virtual / augmented reality system. Some embodiments of the present invention can therefore provide dynamic control concepts for virtual / augmented reality.

[0033] Furthermore, modifications to and additional steps to a conventional virtual / augmented reality system can be suggested that can increase the value and usefulness of the proposed concepts.

[0034] Illustrative embodiments can be used in many different types of virtual or augmented reality environments. To provide a context for describing the elements and functionality of the illustrative embodiments, the following are presented: Fig. 1 and Fig. 2 below are provided as exemplary environments in which aspects of the illustrative embodiments can be implemented. It should be clear that the Fig. 1 and Fig. Section 2 merely presents examples and is not intended to impose or suggest any limitations regarding the environments in which aspects and embodiments of the present invention may be implemented. Many modifications can be made to the environments shown without altering the scope of the present invention.

[0035] Fig. Figure 1 shows a block diagram of an exemplary distributed system in which embodiments of the present invention can be implemented. The distributed system 100 can comprise a network of computers in which aspects of the illustrative embodiments can be implemented. The distributed system 100 includes at least one network 102, which is the medium used to provide communication links between different units and computers connected to each other in the distributed data processing system 100. The network 102 can include connections such as wired, wireless communication links, or fiber optic cables.

[0036] In the example shown, a first server 104 and a second server 106, along with a storage unit 108, are connected to network 102. Clients 110, 112, and 114 are also connected to network 102. These clients could be, for example, personal computers, network computers, or similar devices. In this example, the first server 104 can provide clients 110, 112, and 114 with data such as boot files, operating system images, and applications. In this example, clients 110, 112, and 114 are clients of the first server 104. The distributed system 100 can include additional servers, clients, and other units not shown.

[0037] In the example shown, the distributed system 100 can be a subnetwork of the Internet, with the network 102 representing a worldwide set of networks and gateways that use the Transmission Control Protocol / Internet Protocol (TCP / IP) protocol suite to exchange data. At the heart of the Internet is a backbone of high-speed data transmission lines between main nodes or hosts, consisting of thousands of commercial, governmental, educational, and other computer systems that relay data and messages. The distributed system 100 can also be implemented to include several different types of networks, such as an intranet, a local area network (LAN), a wide area network (WAN), or similar. As outlined above, Fig. 1 serve as an example, not as an architectural limitation for other embodiments of the present invention, and therefore the Fig. The individual elements shown in Figure 1 are not considered to be limiting with regard to environments in which the illustrative embodiments of the present invention can be implemented.

[0038] Fig. Figure 2 is a block diagram of an exemplary system 200 in which embodiments of the present invention can be implemented. System 200 is an example of a computer such as client 110 in Fig. 1 or of server 104 in Fig. 1, which may contain computer-usable code or instructions implementing the processes for illustrative embodiments of the present invention.

[0039] In the example shown, the System 200 uses a hub architecture comprising a northbridge / memory controller hub (NB / MCH) 202 and a southbridge / input / output (I / O) controller hub (SB / ICH) 204. One or more processing units (PPUs) 206, main memory 208, and a graphics processor 210 are connected to the NB / MCH 202. The graphics processor 210 can be connected to the NB / MCH 202 using, for example, Peripheral Component Interconnect Express (PCIe) or an accelerated graphics port (AGP).

[0040] In the example shown, a network adapter 212 (which may be a local area network (LAN) adapter) is connected to the SB / ICH 204. An audio adapter 216, a keyboard and mouse adapter 220, a modem 222, a read-only memory (ROM) 224, a hard disk drive (HDD) 226, a CD-ROM drive 230, one or more universal serial bus (USB) ports, other data transmission ports 232, and PCI / PCIe units 234 are connected to the SB / ICH 204 via a first bus 238 and a second bus 240. The PCI / PCIe units can include, for example, Ethernet adapters, add-in cards, and PC cards for notebook computers. Unlike PCIe, PCI uses a card bus control unit. ROM 224 could, for example, be a Flash Basic Input / Output System (BIOS).

[0041] The HDD 226 and the CD-ROM drive 230 are connected to the SB / ICH 204 via the second bus 240. The HDD 226 and the CD-ROM drive 230 can, for example, use an integrated drive electronics (IDE) or a SATA (Serial Advanced Technology Attachment) interface. A SIO (Super I / O) unit 236 can be connected to the SB / ICH 204.

[0042] An operating system runs in one or more processing units 206. The operating system can control various components in the system 200. Fig. 2. Coordinate and provide. The client operating system can be a standard commercial operating system. In conjunction with the operating system, an object-oriented programming system such as Java™ can be executed and calls from Java™ programs or applications running in System 200 can be transmitted to the operating system.

[0043] The server in System 200 could, for example, be an IBM. ® eServer™ System p ® -Computer system in which the operating system Advanced Interactive Executive (AIX) ® ) or the Linux operating system ® The System 200 can be a symmetric multiprocessor (SMP) system, comprising multiple processors in the Processing Unit 206. Alternatively, a single-processor system can be used.

[0044] Instructions for the operating system, the programming system, and applications or programs can reside in the storage units, such as the HDD 226, and be loaded into the main memory 208 for execution by one or more processing units 206. Likewise, one or more message processing programs, according to one embodiment, can be designed to be stored by the storage units and / or the main memory 208.

[0045] The processes for illustrative embodiments of the present invention can be performed by one or more processing units 206, using computer-usable program code which may be located in a memory such as main memory 208, ROM 224 or one or more peripheral units, including HDD 226 and CD-ROM drive 230.

[0046] A bus system, for example the one in Fig. The first bus 238 or second bus 240 shown in Figure 2 can comprise one or more buses. The bus system can be implemented using any type of communication structure or architecture that provides the conditions for data transmission between different components or units connected to the structure or architecture. A data transmission unit such as the modem 222 or the network adapter 212 from Figure 2 can be used. Fig. 2 can comprise one or more units used for transmitting and receiving data. For example, a memory unit could be the main memory 208, the ROM 224, or a cache, such as the one in the NB / MCH 202 in Fig. 2 can be found.

[0047] Experts will understand that the hardware in the Fig. 1 and Fig. 2 may vary depending on the implementation. Other internal hardware or peripheral units, such as flash memory, equivalent non-volatile memory, or optical disk storage, and the like, may be used in addition to or instead of those specified in the Fig. 1 and Fig. The hardware shown in Figure 2 can be used. Furthermore, the processes of the illustrative embodiments can be applied to a multiprocessor data processing system that differs from the previously mentioned system without deviating from the concept and scope of protection of the present invention.

[0048] Furthermore, System 200 can take the form of any one of a number of different data processing systems, including client computing units, server computing units, tablet computers, laptop computers, a telephone or other data transmission unit, a personal digital assistant (PDA), or the like. In some illustrative examples, System 200 might be a portable data processing unit configured with flash memory to provide non-volatile storage for storing, for example, operating system files and / or user-generated data. Consequently, System 200 can essentially be any known or subsequently developed data processing system without any architectural limitations.

[0049] As explained in detail above, embodiments of the present invention comprise a method for detecting physiological stress in a user of a virtual reality environment. Such a method may include an instruction to the user to interact with a virtual object in a virtual reality environment. The method may then include the acquisition of at least one parameter of the user's movement in response to the instruction. Based on the acquired parameter(s) of the user's movement, a measure of physiological stress can be determined. Proposed embodiments can then be used to determine when a user is experiencing physiological stress or strain as a result of interacting with a virtual reality environment.This can then be used to adjust the virtual reality environment and / or instructions to avoid or prevent a user from experiencing physiological stress or strain.

[0050] Accordingly, a method for detecting physiological stress in a user of a virtual reality environment, according to several embodiments, is described below. As mentioned above, many different approaches can be used to determine a measure of physiological stress resulting from detected user interaction with a virtual object. As an example, and with reference to Fig. Figure 3 shows a flowchart of procedure 300 for detecting physiological stress in a user of a virtual reality environment. Here, the virtual reality environment is designed to provide virtual objects with which a user can interact.

[0051] The process begins at 310 by displaying one or more virtual objects to the user at one or more object positions in the virtual reality environment. This can be implemented, for example, using a conventional virtual reality system with a controller and a head-mounted display unit.

[0052] Next, at 320, the user is instructed to interact with one or more of the displayed virtual objects. Such an instruction can be provided, for example, using an audible and / or visual prompt indicating a virtual object and an action to be performed with that virtual object. However, it is clear that the instruction can take any suitable form of prompt to the user, directing them to interact with the virtual environment. For example, it can instruct the user to select a menu item or to perform a specific movement / action within the virtual environment, game, or application. In embodiments where more than one virtual object is displayed, the instruction can pertain to one of the virtual objects or encompass more than one virtual object (such as an instruction to make the virtual objects interact).The prompt can be provided via a visual and / or acoustic cue and / or via a tactile interface (such as vibrations from a handheld control device).

[0053] Another possibility concerns virtual reality applications that interface with the real world. Here, prompts from the real world could be used. For example, a virtual reality application could be developed that allows a user to turn off the lights in the virtual world, establishing a connection to the real light sources in the room in the real world. An audio prompt could then be delivered from the real world to prompt the user to perform an action in the virtual world.

[0054] At 330, one or more parameters of the user's movement in response to the instruction are then captured. For example, the captured parameter(s) of the user's movement may include reaction time, movement speed, or direction of movement. For this purpose, a tracking component can be used, which is suitable for tracking movements of the control unit, movements of the head-mounted display unit, and body movements of the user. For such motion tracking, the direction, speed, and duration of the movement (and / or any other aspect of the movement) can be measured and monitored. This monitoring can be continuous or comprise a series of data points.

[0055] Movements of the control unit and headset can be tracked, for example, using existing virtual reality technology that is commonly available in existing virtual / augmented reality systems and devices. Body movements can be derived from movements of the control unit and / or headset and / or from optical recognition systems that are commonly available (e.g., in video game consoles and accessories). The illustrated embodiment can therefore utilize existing motion measurement and tracking technologies and also integrate several different solution approaches.

[0056] In addition to the parameters described above, operation 330 may also include the detection and monitoring of reaction times and / or the identification of the moving body part(s).

[0057] In step 340, a measure of physiological stress is then determined based on the detected parameter(s) of the user's movement. In some embodiments, step 340 of the determination involves comparing the detected parameters of the user's movement with a reference value, where the reference value may be related to at least one or more virtual objects, the positions of one or more virtual objects, and / or the user. For example, by comparing a detected velocity of the user's movement with a reference velocity value, a measure of the physiological stress experienced by the user can be determined. A velocity value that is lower than a reference velocity value may, for example, indicate high physiological stress or strain for the user.

[0058] Subsequently, based on the result of such a comparison and taking into account the instructions given to the user, a measure of physiological stress for a part of the user's body can be determined. Such an approach can thus consider a relationship between the recorded movement of the user and a movement reference value (e.g., an average speed value for a population of users).

[0059] Furthermore, operation 340 may, for example, also involve analyzing recorded movement speed, movement patterns, directions, etc., to infer shaky, tense, or unusual movements of the user. Such an analysis may include comparing the recorded movement speed, patterns, and directions with existing reference data. Additionally, or alternatively, biometric data may be obtained and analyzed to derive a measure of physiological stress. Such biometric data may be obtained, for example, by connecting to a wearable sensor (e.g., a heart rate sensor, a blood pressure monitor, etc.) and / or from what is detected by the headset or control unit (e.g., skin temperature).

[0060] Furthermore, some embodiments of the present invention may, for example, include additional steps (as indicated by the dashed boxes of Fig. 3 is shown), which can further improve the accuracy and / or suitability of an established measure of physiological stress.

[0061] In particular, procedure 300 at 350 may also include obtaining historical information regarding a previously determined measure of physiological stress. Step 340, determining a measure of physiological stress, may then also be based on the obtained historical information. For example, obtained historical information may be used to verify or flag a determined measure of physiological stress, taking into account erroneous or deviant values. Furthermore, using previously determined measures of physiological stress may enable embodiments to provide a comparison of the user's performance relative to a previous situation or a "normal" baseline measurement. This may involve comparing the newly obtained measure with the historical data. A result of the comparison may then be communicated to the user (e.g.,"We observe that your left arm appears to be under greater stress than would normally be expected for a user of your size, weight, age, etc."). Additionally, or alternatively, historical data regarding the user's physiological stress can be used to demonstrate progress or improvements in body parts. This can be useful in rehabilitation applications.

[0062] Furthermore, the method may, for example, also include a step 360 of detecting an audible or visual cue from the user in response to the application. Step 340 of determining a measure of physiological stress may then also be based on the detected audible or visual cue. In this way, such embodiments can receive other sensory inputs beyond the user movement parameter(s) detected in step 330, and additional cues that can be used to provide a more detailed and accurate determination of physiological stress. For example, audible and / or visual cues can be used to provide additional information for determining (or identifying) a measure of physiological stress.Recognizing specific spoken words or phrases (such as "ouch" or "that hurts") and / or visual cues displayed by the user (such as a user wincing in pain) can indicate stress or pain when moving a particular body part. Known speech and / or visual cue recognition techniques and / or systems can be used to detect such sounds or images.

[0063] Furthermore, some embodiments may also include a step 370 of assigning the determined measure of physiological stress to a body part of the user, and this may be done, for example, on the basis of the position(s) of one or more objects.

[0064] For example, if a user responds to a command to interact with a virtual object at a specific object position, one or more aspects of the user's movement in response to the command can be captured and used to determine if and which body parts are under physiological stress. The determined level of stress can then be mapped to one or more body parts to provide body-part-specific information to the user.

[0065] Furthermore, a proposed method may also involve defining a target part of the user's body (e.g., a body part for which a measure of physiological stress is lacking or required). In the case of 310, displaying one or more virtual objects at one or more object positions in the virtual reality environment can therefore be based on the user's target body part. This allows for the targeted assessment of physiological stress in different body parts. Additionally, a stress map of the user's body can be created by examining different body parts through various instructions to the user (e.g., to identify areas / positions that are difficult for the user to reach).

[0066] It should therefore be clear that proposed embodiments of the present invention, in combination with a process of manipulating the virtual reality environment, can be used to obtain information on the physiological stress of specific areas or body parts of a user. Such manipulation of the virtual reality environment can then be mapped to the user's movements (e.g., speed, reaction times, direction of movement, etc.) in order to determine a measure of physiological stress for the specific areas or parts of the user's body.

[0067] Using information regarding a determined stress measure for each body part, a map can be created that identifies areas / parts of a user's body that may have difficulty accessing objects in one or more specific positions within a virtual reality environment. For example, if a user has a stiff left shoulder, they may find it difficult to reach objects located on their left side at a certain height relative to their forward-facing posture. A physiological stress measure obtained using Procedure 300 can detect this by inferring from motion data that objects positioned high and to the left take a relatively longer time to interact with and / or move more slowly than objects in a different relative position.By averaging such reaction times and / or movement speeds, a map can be created for each body part.

[0068] For areas / parts of the map that are not assigned a measure(s) of physiological stress, the virtual / augmented reality environment can be adapted and / or interaction instructions can be provided to the user to enable the collection of necessary motion data. Virtual objects can be strategically placed in the virtual / augmented reality environment, for example, so that a user must interact with them using one or more specific target body parts (e.g., body parts for which a measure of physiological stress is to be determined). The resulting speed, direction, pattern, etc., of movement can then be analyzed to determine a measure of physiological stress for the target body part(s). In some implementations, it can also be detected that a user refuses / declines to use a target body part.This can occur at 330, and the detection of one or more parameters of a user's movement may include the detection of a lack of movement; this may indicate significant physiological stress for the target body part.

[0069] The above description suggests that a concept for detecting one or more properties of a user's interaction with a virtual object in a virtual reality environment is proposed as a way to determine a measure of the physiological stress experienced by the user. This can be implemented on a part-by-part basis and / or used for multiple body parts simultaneously. For example, information from captured movements while interacting with multiple virtual objects can be used to determine different measures of physiological stress for different parts of the user's body.

[0070] The description provided above indicates that proposed embodiments can utilize the nature and / or characteristics of a user's movement when interacting with a virtual object in a virtual or augmented reality environment to provide concepts for determining a measure of the physiological stress experienced by the user. This can enable highly efficient instruction and / or treatment of a user in a virtual reality environment, as the delivery of virtual objects and / or instructions can be controlled based on the determined physiological stress experienced by the user and taking into account whether a user activity in the virtual reality environment might lead to undesirable / unacceptable physiological stress.

[0071] Therefore, such implementations can be provided as extensions to existing virtual reality systems. These extensions can deliver targeted (e.g., less intrusive or more favorably positioned) virtual objects and / or notifications to users. In this way, users are not presented with virtual objects and / or instructions that cause unnecessary or excessive physiological stress.

[0072] In some embodiments, a system can be provided that includes a processing arrangement suitable for executing any procedure previously specified with reference to the Fig. 1, Fig. 2 to Fig. 3 was described.

[0073] As in Fig.As shown in Figure 4, embodiments may, for example, include a computer system 70 that can form part of a networked system 7. The components of the computer system 70 may include one or more processing arrangements, which may include, for example, processors or processing units 71, a system memory 74, and a bus 90 that connects various system components, including the system memory 74, to the processing unit, but is not limited to these.

[0074] Bus 90 represents one or more of any type of bus structure, including a memory bus or memory control unit, a peripheral bus, an accelerated graphics port, and a processor or local bus, using any one of a variety of bus architectures. Such architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0075] The Computer System 70 typically includes a variety of computer-readable media. Such media can be any available media that the Computer System 70 can access, including both volatile and non-volatile media, and interchangeable and non-interchangeable media.

[0076] The system memory 74 can include computer system-readable media in the form of volatile memory, such as random access memory (RAM) 75 and / or a cache 76. The computer system 70 can also include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 77 can be provided for reading from and writing to a non-removable, non-volatile magnetic medium (not shown and usually referred to as a "hard disk drive"). Although not shown, a magnetic disk drive can be provided for reading from and writing to a removable, non-volatile magnetic disk (for example, a "floppy disk"), and an optical disk drive can be provided for reading from or writing to a removable, non-volatile optical disk such as a CD-ROM, DVD-ROM, or other optical media.In such cases, these can each be connected to the bus 90 via one or more data media interfaces. As shown and described below, the memory 74 can comprise at least one program product that has a set (e.g., at least one) of program modules configured to perform the functions of embodiments of the invention.

[0077] A program / utility 78 comprising a set (at least one) of program modules 79, as well as an operating system, one or more application programs, other program modules, and program data, may, for example, but are not limited to, be stored in memory 74. The operating system, one or more application programs, other program modules, and / or the program data, or a combination thereof, may comprise an implementation of a networked environment. The program modules 79 generally perform the functions and / or methodologies of embodiments of the invention as described herein.

[0078] The computer system 70 can also exchange data with one or more external units 80, such as a keyboard, pointing unit, display 85, etc.; one or more units that enable a user to interact with the computer system 70; and / or any units (e.g., a network card, modem, etc.) that enable the computer system 70 to exchange data with one or more other data processing units. Such data exchange can take place via input / output (I / O) interfaces 72. Furthermore, the computer system 70 can exchange data with one or more networks, such as a local area network (LAN), a wide area network (WAN), and / or a public network (e.g., the Internet), via a network adapter 73. As shown, the network adapter 73 exchanges data with the other components of the computer system 70 via the bus 90.It should be clear that other hardware and / or software components could be used in conjunction with the Computer System 70, although these are not shown. Examples include, but are not limited to: microcode, unit drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and storage systems for data archiving, etc.

[0079] The present invention may be a system, a method, and / or a computer program product with any possible level of technical integration. The computer program product may include a computer-readable storage medium (or media) on which computer-readable program instructions are stored to induce a processor to execute aspects of the present invention.

[0080] A computer-readable storage medium can be a physical unit capable of retaining and storing instructions for use by an execution unit. For example, a computer-readable storage medium can be an electronic storage unit, a magnetic storage unit, an optical storage unit, an electromagnetic storage unit, a semiconductor storage unit, or any suitable combination thereof, without limitation. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), and erasable programmable read-only memory (EPROM).Flash memory), static random-access memory (SRAM), portable compact storage disk-read-only memory (CD-ROM), DVD (digital versatile disc), USB flash drive, floppy disk, a mechanically coded unit such as punched cards or raised structures in a groove on which instructions are stored, and any suitable combination thereof. A computer-readable storage medium shall not, in its use herein, be understood as volatile signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses traveling through an optical fiber cable), or electrical signals transmitted by a wire.

[0081] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to individual data processing units or, via a network such as the internet, a local area network, a wide area network, and / or a wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission lines, wireless transmission, routing computers, firewalls, switching units, gateway computers, and / or edge servers. A network adapter card or network interface in each data processing unit receives computer-readable program instructions from the network and forwards them for storage on a computer-readable storage medium within the respective data processing unit.

[0082] Computer-readable program instructions for executing the steps of the present invention can be assembly instructions, ISA (Instruction Set Architecture) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., as well as conventional procedural programming languages ​​such as C or similar languages. The computer-readable program instructions can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server.In the latter case, the remotely located computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be established with an external computer (for example, via the internet using an internet service provider). In some embodiments, electronic circuits, including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), can execute computer-readable program instructions by using state information from the computer-readable program instructions to personalize the electronic circuits to perform aspects of the present invention.

[0083] Aspects of the present invention are described herein with reference to flowcharts and / or block diagrams or diagrams of methods, devices (systems), and computer program products according to embodiments of the invention. It is understood that each block of the flowcharts and / or block diagrams or diagrams, as well as combinations of blocks in the flowcharts and / or block diagrams or diagrams, can be executed by means of computer-readable program instructions.

[0084] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a specialized computer, or another programmable data processing device to create a machine, such that the instructions executed via the processor of the computer or other programmable data processing device generate a means of implementing the functions / steps specified in the block(s) of the flowcharts and / or block diagrams or charts.These computer-readable program instructions may also be stored on a computer-readable storage medium capable of controlling a computer, programmable data processing device, and / or other units to function in a particular manner, such that the computer-readable storage medium on which instructions are stored has a manufactured product, including instructions that implement aspects of the function / step specified in the block(s) of the flowchart and / or block diagrams or charts.

[0085] The computer-readable program instructions can also be loaded onto a computer, other programmable data processing device, or other unit to cause the execution of a series of process steps on the computer or other programmable device or other unit in order to generate a process executed on a computer, such that the instructions executed on the computer, other programmable device, or other unit implement the functions / steps specified in the block(s) of the flowcharts and / or block diagrams or charts.

[0086] The flowcharts and block diagrams or charts in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this context, each block in the flowcharts or block diagrams or charts can represent a module, segment, or part of instructions that includes one or more executable instructions for performing the specific logical function(s). In some alternative embodiments, the functions specified in the block may occur in a different order than shown in the figures. For example, two blocks shown consecutively may in reality be executed essentially simultaneously, or the blocks may sometimes be executed in reverse order depending on the corresponding functionality.It should also be noted that each block of the block diagrams or charts and / or flowcharts, as well as combinations of blocks in the block diagrams or charts and / or flowcharts, can be implemented by special hardware-based systems that perform the specified functions or steps, or execute combinations of special hardware and computer instructions.

[0087] The descriptions of the various embodiments of the present invention are shown for illustrative purposes, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and changes are apparent to those skilled in the art without deviating from the scope of application of the described embodiments. The terminology used herein has been chosen to explain the basic concept of the embodiments, their practical application, or technical improvements over commercially available technologies as clearly as possible, or to enable other skilled individuals to understand the embodiments disclosed herein.

Claims

[1] A computer-implemented method (100) for detecting physiological stress of a user of a virtual reality environment, wherein the method comprises: Display (310) virtual objects at object positions in the virtual reality environment for the user; Instruction (320) to the user to interact with one of the displayed virtual objects; Capturing (330) at least one parameter of the user’s movement in response to the instruction, wherein the at least one parameter of the movement includes response time or movement speed or movement direction; Determine (340) a measure of physiological stress based on the captured at least one parameter of the user's movement, wherein the determination (340) includes comparing the captured parameters of movement with a reference value associated with a virtual object and / or its object position and / or the user; Obtain (350) historical information relating to previously determined measures of physiological stress and compare a newly determined measure with the historical information to provide a personal baseline measure, including checking or flagging erroneous or deviant values; Identify a target part of the user's body to which no measure of physiological stress is associated and adjust the object position so that the interaction requires the use of the target part; Detect a lack of movement of the target part as an indication of significant physiological stress; Assigning (370) the measure of physiological stress to a body part of the user based on the object position of one of the displayed virtual objects; and Adapting the virtual reality environment based on the measure of physiological stress and the user's body part, wherein the adaptation of the virtual reality environment involves adjusting the object position of one of the displayed virtual objects. [2] Method according to claim 1, wherein the detected at least one parameter of the user's movement is reaction time and / or movement speed and / or movement direction. [3] The method of claim 1, further comprising: Capture (360) one or more spoken words from the user in response to the instruction, and where the determination of the measure of physiological stress is also based on the one or more recorded spoken words. [4] The method of claim 1, further comprising: Defining a target part of the user's body for which a measure of physiological stress is to be determined, wherein defining the target part of the user's body involves identifying a part of the user's body to which no measure of physiological stress is associated, and wherein the display of the virtual object at the object position in the virtual reality environment is based on the target part of the user's body, and wherein the assignment of the measure of physiological stress to the user's body part involves an assignment of the measure of physiological stress to the user's target body part. [5] The method of claim 1, further comprising: Obtained (350) historical information relating to a previously determined measure of physiological stress, and where the determination of the measure of physiological stress is also based on the historical information obtained. [6] Computer program product comprising a computer-readable storage medium containing program instructions stored thereon, wherein the computer-readable storage medium is not a volatile signal per se, and wherein the computer-executable program instructions execute a method according to any of the preceding claims. [7] System (7) which features: one or more processors (71); and a memory (74) which is connected to one or more processors for data exchange, wherein the memory (74) comprises instructions (79) which, when executed by the one or more processors (71), cause the one or more processors to execute a method according to any one of claims 1 to 5.

Images