Method for operating a virtual reality system

Abstract

Method for operating a virtual reality system (10), comprising the steps: - Specifying a first virtual observation position (P3) depending on a recorded position (P1) of a first person (26) and specifying a second virtual observation position (P4) depending on a recorded position (P2) of a second person (28); - Displaying a virtual object (40) within a virtual environment (38) using a first virtual reality headset (12) worn by the first person (26) from the first virtual observation position (P3); - simultaneous display of the same virtual object (40) within the same virtual environment (38) using a second virtual reality headset (14) worn by the second person (28) from the second virtual observation position (P4); - Capture speech sound (34) from at least the first person (26) and convert the speech sound (34) into a first speech signal using a microphone device (22); - Determining the relative arrangement of the two virtual observation positions (P3, P4) to each other; - Controlling a headphone (18) worn by the second person (28) taking into account the determined relative arrangement of the virtual observation positions (P3, P4), so that the speech signal is reproduced by means of the headphone (18) worn by the second person (28) as if the speech sound (34) at the second virtual observation position (P4) were being received from the first virtual observation position (P3); - wherein the microphone device (22) comprises a dummy head (32) equipped with a binaural recording device and arranged in the same room (30) as the two persons (26, 28), by means of which the speech sound (34, 36) from the first person (26) and / or from the second person (28) is recorded.

Description

[0001] The invention relates to a method for operating a virtual reality system.

[0002] A virtual reality system, as defined by the invention, is a system by means of which a virtual reality can be displayed both visually and acoustically. The virtual reality system can comprise several so-called virtual reality glasses for displaying virtual objects and headphones for sound reproduction.

[0003] Virtual reality glasses are a specific type of head-mounted display, a visual output device worn on the head. They present images on a screen positioned close to the eyes or project them directly onto the retina. Virtual reality glasses typically also have sensors for tracking head movement. This allows the display of the calculated graphics to be adapted to the wearer's movements. Due to the physical proximity, the image areas displayed by head-mounted displays appear considerably larger than those of free-standing screens and, in extreme cases, can even cover the user's entire field of vision. Because the display on virtual reality glasses follows all head movements of the wearer, they have the feeling of moving directly within a computer-generated image landscape.

[0004] Using such virtual reality glasses, a virtual reality can be presented, whereby virtual reality is usually referred to as the representation and simultaneous perception of reality in its physical properties in a real-time computer-generated, interactive virtual environment.

[0005] Such a virtual reality system can be used, for example, in the sale of motor vehicles, to virtually display a motor vehicle using virtual reality headsets. Several people wearing the headset could, for example, simultaneously move around the virtual vehicle within the same displayed virtual environment to view it from different perspectives. It would be desirable if the people could communicate with each other while doing so.

[0006] US 2008 / 0144794 A1 discloses a method for operating a teleconferencing system. Different users of individual PCs can move within a shared virtual environment according to their mouse and keyboard commands. The virtual environment is displayed on monitors connected to the PCs. The users each wear headsets so they can communicate with each other. Audio output via the headsets takes into account the relative positioning of the users within the virtual environment.

[0007] US 5 588 139 A shows a method for displaying a virtual environment. Different people each wear a head-mounted display, which displays the same virtual environment. The positions of the respective head-mounted displays are detected, and changes in the position of the head-mounted displays are converted into respective virtual position changes within the displayed virtual environment. Virtual images of the people are displayed within the virtual environment so that the people can see each other within the virtual environment. The people can converse with one another, and corresponding sounds can be reproduced stereoscopically as if they were coming directly from the mouth of one of the virtual images of the people.

[0008] EP 2 373 055 A2 discloses a headphone device for reproducing binaural spatial audio signals and a system equipped therewith.

[0009] It is the object of the present invention to provide a method for operating a virtual reality system by means of which a conversation is made possible between at least two people who are viewing the same virtual environment together.

[0010] The object is achieved by a method for operating a virtual reality system having the features of the independent patent claim. Advantageous embodiments with expedient and non-trivial refinements of the invention are specified in the dependent claims.

[0011] The method according to the invention for operating a virtual reality system comprises the following steps: - specifying a first virtual observation position depending on a detected position of a first person and specifying a second virtual observation position depending on a detected position of a second person; - Displaying a virtual object within a virtual environment by means of first virtual reality glasses worn by the first person from the first virtual observation position; - simultaneously displaying the same virtual object within the same virtual environment by means of a second pair of virtual reality glasses worn by the second person from the second virtual observation position; - capturing speech sound from at least the first person and converting the speech sound into a first speech signal by means of a microphone device; - Determining the relative arrangement of the two virtual observation positions to each other; - controlling a headset worn by the second person taking into account the determined relative arrangement of the virtual observation positions, so that the speech signal is reproduced by means of the headset worn by the second person as if the speech sound were received at the second virtual observation position from the first virtual observation position.

[0012] Using the method according to the invention, it is thus possible for at least two different people to simultaneously view one and the same virtual environment and a virtual object arranged therein. The people can simultaneously change their virtual observation position within the virtual environment in order to view the virtual object from different virtual observation positions and thus also from different virtual perspectives.

[0013] If such a virtual reality application involves, for example, a sales situation in which a seller and a buyer view the virtual object together, it may be useful for the two people to be able to converse with each other during the virtual reality application. Using the method according to the invention, it is possible for the other person's voice to come from the direction in which the respective person is located within the virtual environment, corresponding to their respective virtual observation position.By means of the method according to the invention, it is at least possible for the speech sound of a first person to be captured and converted into a speech signal, wherein by determining the relative arrangement of the two virtual observation positions of the two persons to one another within the virtual environment, a corresponding control of the headphones worn by the second person is possible such that, taking into account the determined relative arrangement of the virtual observation positions, the speech signal is reproduced via the headphones worn by the second person as if the speech sound at the second virtual observation position were being received from the first virtual observation position. Thus, it feels to the second person as if the speech sound from the first person were reaching them in the same way as the two persons are currently located in the virtual environment.

[0014] Furthermore, the invention provides for the microphone device to have an artificial head equipped with a binaural recording device and located in the same room as the two people, by means of which the speech sound of the first person and / or the second person is captured. This allows for particularly realistic directional speech capture, which has a positive effect on directional speech reproduction via the respective headphones.

[0015] An advantageous embodiment of the invention provides that speech sound from the second person is detected and converted into a second speech signal, wherein a headset worn by the first person is controlled taking into account the determined relative arrangement of the virtual observation position in such a way that the second speech signal is reproduced by means of the headset worn by the first person as if the speech sound at the first virtual observation position were received by the second virtual observation position.In other words, it is possible for the second person to also speak, and for the first person's speech to be reproduced via the headphones in such a way that it sounds to the first person as if the second person's speech within the virtual environment were actually coming from the direction of the second observation position, i.e., the second person's virtual positioning within the virtual environment. This makes it particularly easy for the first and second person to converse with each other during the virtual reality application, with each person hearing the speech reproduced via their respective headphones in the direction in which they are positioned within the virtual environment according to their virtual observation position.In other words, a type of directional speech capture and directional speech reproduction takes place, with the directional information relating to the respective virtual observation positions of the two persons within the displayed virtual environment.

[0016] A further advantageous embodiment of the invention provides that a respective head orientation of the two people is detected and assigned to a respective virtual head orientation within the virtual environment, wherein the respective virtual head orientations of the people are taken into account when controlling the headphones worn by the first person and / or when controlling the headphones worn by the second person. Thus, in addition to purely taking into account the virtual positioning of the two people within the virtual environment, a virtual alignment of the people to one another within the virtual environment is also carried out. This allows speech sound reproduction via the headphones to be particularly detailed in the sense of directional speech detection and reproduction.

[0017] It would also be possible for the microphone device to comprise a first microphone worn by the first person, by means of which the speech sound of the first person is recorded, and / or for the microphone device to comprise a second microphone worn by the second person, by means of which the speech sound of the second person is recorded. Due to the proximity of the microphones to the mouths of the respective people, the speech sound of the people can be recorded particularly undisturbed by other ambient noise. Furthermore, if the people wear respective microphones by means of which their speech sound is recorded, it is possible for the people to be in different rooms and yet still be able to communicate with one another within the same virtual environment.

[0018] According to a further advantageous embodiment of the invention, the first speech signal and / or the second speech signal are converted into a binaural speech signal. This makes it possible to achieve particularly detailed, directional speech reproduction using the headphones, which has a positive effect on the realistic speech perception of the respective people as they move together within the virtual environment.

[0019] A further advantageous embodiment of the invention provides that detected ambient noise that differs from the speech sound of the first and / or second person is filtered out by the microphone device. This essentially transmits only the speech sound from the respective persons to the other person, without the conversation between the persons being impaired by additional ambient noise.

[0020] According to a further advantageous embodiment of the invention, it is provided that a symbol identifying the second person is displayed at the second virtual observation position by means of the first pair of virtual reality glasses, and a symbol identifying the first person is displayed at the first virtual observation position by means of the second pair of virtual reality glasses. This allows the people within the virtual environment to easily recognize where the other person is located within the virtual environment, which further facilitates communication between them. Furthermore, the people can recognize the perspective from which the other person is viewing the currently displayed virtual object, which can further improve communication between them.

[0021] A further advantageous embodiment of the invention provides that a virtual motor vehicle is displayed as the virtual object. This makes it possible, for example, for a salesperson and a potential customer to move together within the virtual environment to jointly explore and discuss a potential purchase object in the form of a virtual motor vehicle. It is possible for the customer and the salesperson to actually be in the same room; alternatively, it is just as possible for the customer and the salesperson to be in different rooms, while still being able to view, discuss, and examine the same virtual motor vehicle within the same virtual environment.

[0022] Further advantages, features, and details of the invention will become apparent from the following description of preferred embodiments and from the drawings. The features and combinations of features mentioned above in the description, as well as the features and combinations of features mentioned below in the description of the figures and / or shown alone in the figures, can be used not only in the respective specified combinations, but also in other combinations or on their own, without departing from the scope of the invention.

[0023] The drawing shows: Fig. 1 a schematic representation of a virtual reality system comprising a detection device, two virtual reality glasses, a microphone device, two headphones and a control device for controlling the virtual reality glasses and the headphones; Fig. 2 a schematic representation of a room in which two people are arranged, each wearing one of the virtual reality glasses and a headset, with an artificial head equipped with a binaural recording device for capturing the speech sound of the two people being arranged between the two people; Fig. 3 a schematic plan view of a virtual environment displayed by means of the virtual reality glasses, within which a virtual motor vehicle is displayed, wherein respective virtual observation positions of the two persons are marked, from whose perspective the persons can each view the virtual motor vehicle through their virtual reality glasses; and in Fig. 4 a schematic representation of two different rooms, wherein one of the persons is arranged in each of the rooms and the persons wear respective microphones by means of which the speech sound of the persons is recorded.

[0024] In the figures, identical or functionally identical elements are provided with the same reference symbols.

[0025] A virtual reality system 10 for displaying a virtual environment with a virtual object arranged therein is shown in a schematic representation in Fig. 1. The virtual reality system 10 comprises a first pair of virtual reality glasses 12, a second pair of virtual reality glasses 14, a first pair of headphones 16, and a second pair of headphones 18. The virtual reality system 10 further comprises a detection device 20, a microphone device 22, and a control device 24, which is designed to control the virtual reality glasses 12, 14 and the headphones 16, 18.

[0026] In Fig. 2 shows a schematic plan view of a first person 26 and a second person 28, wherein the two persons 26, 28 are arranged in one and the same room 30. The first person 26 has put on the first pair of virtual reality glasses 12 and the first headphones 16, while the second person 28 has put on the second pair of virtual reality glasses 14 and the second headphones 18 of the virtual reality system 10. An artificial head 32 equipped with a binaural recording device is arranged between the two persons 26, 28 and is part of the microphone device 22. The artificial head 32 equipped with the binaural recording device is designed to record speech sound 34 from the first person 26 and speech sound 36 from the second person 28. The microphone device 22 is designed to convert the speech sound 34 into a first speech signal and the speech sound 36 into a second speech signal.The microphone device 22 is further configured to convert the respective speech signals into binaural speech signals. Furthermore, the microphone device 22 is also configured to filter out detected ambient noise that differs from the speech sound 34 or the speech sound 36.

[0027] In Fig. Figure 3 shows a schematic top view of a virtual environment 38, which is displayed simultaneously using the two virtual reality glasses 12, 14. A virtual motor vehicle 40 is also displayed within the virtual environment 38 using the virtual reality glasses 12, 14. A method for operating the virtual reality system 10 is explained in more detail below.

[0028] First, a first virtual observation position P3 is specified depending on a real first position P1 of the first person 26 within the room 30, as detected by the detection device 20. Furthermore, a second virtual observation position P3 is specified depending on a real position P2 of the second person 28 within the room 30, as detected by the detection device 20.

[0029] The virtual motor vehicle 40 is displayed within the virtual environment 38 from the first virtual observation position P3 by means of the first virtual reality glasses 12 worn by the first person 26. The first person 26 thus sees the right side of the virtual motor vehicle 40. Simultaneously, the same virtual motor vehicle 40 is displayed within the same virtual environment 38 from the second virtual observation position P4 by means of the second virtual reality glasses 14 worn by the second person 28. The second person 28 thus sees the virtual motor vehicle 40 from the left side view through the second virtual reality glasses 14.

[0030] If the first person 26 should say something, the associated speech sound 34 of the first person 26 is picked up by the microphone device 22 - according to the Fig. 2, i.e., by means of the artificial head 32, and converted into a first speech signal. The relative arrangement of the two virtual observation positions P3, P4 to one another is determined by means of the control device 24. Subsequently, the control device 24 controls the headphones 18 worn by the second person 28, taking into account the determined relative arrangement of the two virtual observation positions P3, P4, such that the speech signal from the speech sound 34 of the first person 26 is reproduced by means of the headphones 18 worn by the second person 28 as if the speech sound 34 were being received at the second virtual observation position P4 from the first virtual observation position P3. In other words, the second person 28 perceives the linguistic utterance of the first person 26 in a direction-dependent manner depending on the virtual positioning within the virtual environment.The reproduction of the speech sound 34 of the first person 26 via the headphones 18, which the second person 28 has put on, thus occurs purely depending on how the virtual observation positions P3, P4 are arranged relative to one another. According to the method shown in . Fig. 3, the second person 28 perceives the speech sound 34, i.e. the linguistic utterance, of the first person 26 as if the first person 26 were positioned opposite the second person 28.

[0031] Conversely, the speech sound 36 from the second person 28 can also be detected and converted into a second speech signal, wherein the headphones 16 worn by the first person 26 are controlled taking into account the determined relative arrangement of the virtual observation positions P3, P4 such that the second speech signal is reproduced by means of the headphones 16 worn by the first person 26 as if the speech sound 36 from the second person 28 at the first virtual observation position P3 were received from the second virtual observation position P4.Due to the use of the artificial head 32, which is equipped with the binaural recording device, a direction-dependent speech recording of the speech sounds 34, 36 of the two persons 26, 28 can be carried out in a simple and particularly detailed manner and can be converted into corresponding binaural speech signals, so that a particularly spatial hearing impression can be ensured during the conversation between the persons 26, 28.

[0032] Furthermore, it can be provided that a respective head orientation of the two persons 26, 28 is detected by the detection device 20 and assigned to a respective virtual head orientation within the virtual environment 38, wherein the respective virtual head orientations of the persons 26, 28 are taken into account when controlling the headphones 16 worn by the first person 26 and / or when controlling the headphones 18 worn by the second person 28. This can further enable improved directional speech detection and speech reproduction via the respective headphones 16, 18.

[0033] In addition, it can be provided that a symbol 42 identifying the second person 28 is displayed at the second virtual observation position P4 by means of the first virtual reality glasses 12, and a symbol 44 identifying the first person 26 is displayed at the first virtual observation position P3 by means of the second virtual reality glasses 14. In other words, the virtual positioning of the second person 28 is displayed to the first person 26 by means of the symbol 42, and the virtual positioning of the first person 26 within the virtual environment 38 is displayed to the second person 28 by means of the symbol 44. Taking into account the respectively detected head orientation of the two people 26, 28, the symbols 42, 44 can also be displayed in such a way that the people 26, 28 are shown, by means of the virtual reality glasses 12, 14, in which direction the respective people 26, 28 are currently looking within the virtual environment 38.

[0034] This further facilitates or simplifies a targeted conversation between the two people 26, 28 while they move within the virtual environment 38. The first person 26 can be, for example, a car salesman, while the second person 28 can be, for example, a potential customer. First, the two people 26, 28 configure the virtual motor vehicle 40 according to the customer's wishes and then begin the virtual reality application, during which the two people 26, 28 put on the virtual reality glasses 12, 14 and display the just-configured virtual motor vehicle 40.The two persons 26, 28 can move freely within the virtual environment 38 and thus view the virtual motor vehicle 40 together from a variety of perspectives and converse with each other, whereby a particularly simple conversation between the two persons 26, 28 is enabled due to the directional speech detection and speech reproduction.

[0035] In Fig.4 shows an alternative scenario not covered by the patent claims, wherein the two persons 26, 28 are located in different rooms 46, 48. The first person 26 in turn wears the first virtual reality glasses 12 and the first headphones 16. The second person 28 in turn wears the second virtual reality glasses 14 and the second headphones 18. In the case shown here, the persons 26, 28 each wear a microphone 50, 52 in front of their mouth to capture the respective speech sound 34, 36. Alternatively, it would also be possible for an artificial head 32 equipped with the binaural recording device to be set up in the respective rooms 46, 48 in order to capture the respective speech sound 34, 36 of the persons 26, 28.Although the two people 26, 28 are located in different rooms 46, 48, the virtual reality glasses 12, 14 display the virtual environment 38 including the virtual motor vehicle 40 simultaneously, so that, although the two people 26, 28 are not in the same room, they can nevertheless jointly examine the virtual motor vehicle 40 within the same virtual environment 38. By means of the microphones 50, 52 - or alternatively by means of respective artificial heads 32 arranged within the rooms 46, 48 and equipped with binaural recording devices - it is possible for the two people 26, 28 to also converse while jointly viewing the virtual motor vehicle 40 from a variety of perspectives within the virtual environment 38.The directional speech detection and speech reproduction are carried out analogously to the previously explained embodiment, in which the two persons 26, 28 were arranged in one and the same room 30.

[0036] By means of the explained virtual reality system 10 and the explained method for operating the virtual reality system 10, it is therefore possible in a simple and convenient manner for several people 26, 28 to simultaneously view one and the same virtual environment 38 with the same virtual motor vehicle 40 or any other virtual object and to be able to talk to each other at the same time.

Claims

[1] Method for operating a virtual reality system (10), comprising the steps: - specifying a first virtual observation position (P3) as a function of a detected position (P1) of a first person (26) and specifying a second virtual observation position (P4) as a function of a detected position (P2) of a second person (28); - displaying a virtual object (40) within a virtual environment (38) by means of first virtual reality glasses (12) worn by the first person (26) from the first virtual observation position (P3); - simultaneously displaying the same virtual object (40) within the same virtual environment (38) by means of a second virtual reality glasses (14) worn by the second person (28) from the second virtual observation position (P4); - detecting speech sound (34) from at least the first person (26) and converting the speech sound (34) into a first speech signal by means of a microphone device (22); - Determining the relative arrangement of the two virtual observation positions (P3, P4) to each other; - controlling a headset (18) worn by the second person (28) taking into account the determined relative arrangement of the virtual observation positions (P3, P4), so that the speech signal is reproduced by means of the headset (18) worn by the second person (28) as if the speech sound (34) were received at the second virtual observation position (P4) from the first virtual observation position (P3); - wherein the microphone device (22) has an artificial head (32) equipped with a binaural recording device and arranged in the same room (30) as the two persons (26, 28), by means of which the speech sound (34, 36) of the first person (26) and / or of the second person (28) is recorded. [2] Method according to claim 1, characterized by that speech sound (36) from the second person (28) is detected and converted into a second speech signal, wherein a headphone (16) worn by the first person (26) is controlled taking into account the determined relative arrangement of the virtual observation positions (P3, P4) in such a way that the second speech signal is reproduced by means of the headphone (16) worn by the first person (26) as if the speech sound (36) at the first virtual observation position (P3) were being received by the second virtual observation position (P4). [3] Method according to claim 1 or 2, characterized by that a respective head orientation of the two persons (26, 28) is detected and assigned to a respective virtual head orientation within the virtual environment (38), wherein the respective virtual head orientations of the persons (26, 28) are taken into account when controlling the headphones (16) worn by the first person (26) and / or when controlling the headphones (18) worn by the second person (28). [4] Method according to one of the preceding claims, characterized by that the first speech signal and / or the second speech signal is converted into a binaural speech signal. [5] Method according to one of the preceding claims, characterized by that detected ambient noises deviating from the speech sound (34, 36) of the first and / or the second person (26, 28) are filtered out by means of the microphone device (22). [6] Method according to one of the preceding claims, characterized by that a symbol (42) identifying the second person (28) is displayed at the second virtual observation position (P4) by means of the first virtual reality glasses (12) and a symbol (44) identifying the first person (26) is displayed at the first virtual observation position (P3) by means of the second virtual reality glasses (14). [7] Method according to one of the preceding claims, characterized by that a virtual motor vehicle is displayed as the virtual object (40).

Images