Method and system for presenting a shared virtual environment to users residing in different vehicles, and server unit, user device and vehicle

Abstract

The invention relates to a method for presenting a shared virtual environment (16) to a plurality of users (45) residing in respective vehicles (12) when different vehicles (12) are driving in a real environment (18). In at least one of the vehicles (12), a respective interactive user device (13) of the vehicle (12) receives definition data (47) from a respective user (45) residing in the vehicle (12), the definition data (47) describing a new virtual event (20) for the virtual environment (16), and at the same time as the user (45) generates the definition data (47), a position sensor of the vehicle (12) generates position data (34) of the vehicle (12), the position data (34) describing a current position of the vehicle (12) in the real environment (18). A stationary server unit (11) receives the definition data (47) and the corresponding position data (34) and generates corresponding event data (30) of the new virtual event (20), wherein, in accordance with the position data (34), a trigger area (19) is defined, which defines where in the real environment (18) the virtual event (20) should be triggered for the user (45) in his respective vehicle.

Description

Technical Field

[0001] This invention relates to a method for presenting a shared virtual environment to users residing in different vehicles while different vehicles are traveling in a real-world environment. The invention also provides a corresponding system and components of that system—a server unit for presenting an interactive user device for the virtual environment, and a vehicle including such an interactive user device. Background Technology

[0002] Virtual reality (VR) and augmented reality (AR) are technologies increasingly used in the gaming and production sectors to display virtual environments. In both VR and AR, the user wears what is known as a head-mounted display (HMD). In the following text, the more general term "interactive user equipment" is used to refer to an HMD or HMD that has at least one portable controller or HMD and additional control circuitry linked to the HMD (and may or may not have a controller).

[0003] Most use cases for VR and AR are limited to static domains, i.e., users wearing HMDs on stationary ground (e.g., in a room of a building). However, there is also interest in using HMDs in dynamic domains, such as during dynamic car rides. The implementation of these methods in production vehicles would offer significant advantages. In implementation, on the one hand, fixed VR content can be visualized for a fixed driving route, and on the other hand, VR content can be dynamically generated based on map information of a planned driving route.

[0004] This embodiment does not fully utilize the potential of VR users' location information in the vehicle and ignores the process by which VR users create and generate events / interactions.

[0005] Document US10,785,621B1 describes a virtual gaming system for vehicles. This system displays game characters in a virtual environment as the vehicle drives through a predefined area in the real-world environment. Users in the vehicle can interact with the game characters, for example, by shooting one of them, thus altering the state of the virtual game environment. When a vehicle passes another vehicle, the current state of the game environment can be transmitted to that other vehicle, so that users in that other vehicle can see this changed state of the game environment as it passes through the area and interacts with it. The virtual gaming system requires vehicles to meet each other to transmit the current state of the game environment. Furthermore, any changes that may occur in the game environment need to be pre-programmed by the game provider.

[0006] Document US2019 / 0101976A1 describes a system for displaying a virtual environment to a user in a vehicle. The displayed virtual content depends on the vehicle's current driving state; for example, if the vehicle is rapidly approaching a curve, a warning can be displayed in the virtual environment. Therefore, the user is restricted to consuming predefined virtual content within the virtual environment.

[0007] Document US2018 / 0040163A1 describes a system for displaying a virtual environment that depends on the vehicle's current driving route (as described above) and anticipated stimulus events (e.g., centrifugal force caused when the vehicle travels through a curve in the road). The user only experiences virtual content that needs to be prepared by the provider of the virtual environment. Summary of the Invention

[0008] The purpose of this invention is to provide a system for displaying a virtual environment to users in different vehicles, wherein the virtual environment also reflects the individual circumstances of the users.

[0009] This objective is achieved through the subject matter of the independent claims. Advantageous developments of other, less significant embodiments of the invention are specified in the following description, dependent claims, and drawings.

[0010] As a solution to achieve this objective, the present invention provides a method for presenting a shared virtual environment to a user residing in a corresponding vehicle when different vehicles are driving in a real-world environment. Users can reside in the vehicles simultaneously or at different overlapping and / or non-overlapping time intervals. When a corresponding user is in one of the vehicles and using an interactive user device, the user can create new virtual content by defining or editing definition data describing corresponding new virtual events. To this end, in at least one vehicle, the corresponding interactive user device of that vehicle receives definition data from the corresponding user residing in that vehicle and / or the user device generates definition data (automatically) based on the current driving situation of the vehicle. Therefore, the definition data describes user-defined and / or situation-based new virtual events for the virtual environment. This virtual event is linked to a specific triggering area in the real-world environment where the vehicle is being driven. To this end, when the user generates the definition data, the vehicle's position sensors generate vehicle position data. This position data describes the vehicle's current position in the real-world environment.

[0011] The user who creates the defining data is referred to herein as the "creating user". Additionally or alternatively, the user equipment may automatically generate this defining data of new virtual events based on sensor data describing the current driving situation. A location sensor, such as a receiver of location signals for GNSS (Global Navigation Satellite System) or GPS (Global Positioning System), indicates where the new virtual event is generated in the real-world environment. The real-world environment may include the road network the vehicle is driving on. The virtual event may be, for example, a message or map that the creating user can leave for other users in the virtual environment. The user who consumes or pays attention to the virtual events is also referred to below as the "consuming user".

[0012] To update the virtual environment, the static server unit receives definition data and corresponding location data from the vehicle and generates event data for new virtual events. The trigger area is defined based on the vehicle's location data, specifying where the virtual event should be triggered or located in the real-world environment. The static server then collects all new virtual events generated by the corresponding user creator. The virtual environment is then populated or enriched by creating user-generated virtual events. Each virtual event is associated with a trigger location or trigger area in the real-world environment. When a user accesses the virtual environment while residing in a real-world trigger area, the corresponding virtual event becomes accessible or visible to that user at that trigger location or trigger area. Therefore, a user needs to travel through or enter the corresponding trigger area while in a vehicle to detect, see, or access the corresponding virtual event in the virtual environment.

[0013] To access virtual events, the following logic is provided. The server unit receives corresponding request data from the corresponding user equipment of at least one vehicle. The request data indicates that the corresponding vehicle is approaching, entering, or located in the corresponding trigger area of ​​one of the virtual events. The server unit then sends the corresponding event data of the virtual event to the interactive user equipment of the vehicle, and the corresponding virtual event is displayed to the user in the vehicle as part of the virtual environment. The corresponding virtual event is thus automatically triggered or initiated. Virtual events created by the user remaining in the corresponding trigger area are thus presented to the corresponding consuming user in the corresponding trigger area. The virtual environment thus becomes a common virtual environment because the creating user can generate virtual events that can also be experienced or consumed by other consuming users. This allows the creating user to communicate in real time with at least one consuming user (i.e., the creating user and at least one consuming user use their respective interactive user equipment simultaneously or concurrently), or, when the creating user may have even left the trigger area, the creating user can leave the virtual event for the consuming user to consume later.

[0014] The advantage of this invention is that the virtual environment can be filled or enriched by virtual events created or added by the user, enabling communication mechanisms to be implemented within the virtual environment. If the corresponding virtual event is automatically created by the user device itself (e.g., caused by a specific driving situation), this allows signaling to at least one consumer user how the creating user perceives or experiences the driving situation.

[0015] The present invention also includes embodiments that provide features with additional technical advantages.

[0016] One embodiment may include: when a corresponding user device in at least one of the vehicles receives corresponding defined data for one of the virtual events, the user device also receives conditional data describing the current environmental conditions and / or current state of the vehicle, such as describing a driving situation. The conditional data may be user-defined, i.e., the creator user can set or describe the conditional data (e.g., by providing corresponding user input via a keyboard, a portable controller, or voice). Additionally or alternatively, at least a portion of the conditional data may be measured by the user device using at least one sensor of the vehicle, such that the conditional data describes the current driving situation and / or environmental conditions of the real-world environment. Correspondingly, the server unit receives this conditional data and generates corresponding triggering conditions for new virtual events. In addition to the triggering area, there are then triggering conditions. The virtual event is only displayed to the user if the user is residing in a vehicle that also satisfies the triggering conditions. In other words, the consumer user experiences or is presented with a virtual event only if the consumer user is in the same situation as the creator user. This prevents virtual events that may not be suitable or correspond to the consumer user's current situation from being presented.

[0017] One embodiment may specify at least one triggering condition including:

[0018] - The vehicle's speed has a value that falls within a predetermined speed range.

[0019] Traffic lights in real-world environments have predetermined states.

[0020] - Traffic is in a predefined state (e.g., traffic congestion).

[0021] -Weather in the real world environment

[0022] -The vehicle's specific direction of travel or direction of movement (e.g., northeast).

[0023] - The driving destination notified by the navigation system.

[0024] Speed ​​ranges can define a stationary vehicle or a vehicle traveling at low speeds, i.e., speeds below a predetermined threshold (e.g., the threshold can be greater than 0 and within a range from 5 km / h to 50 km / h). Speed ​​ranges can also define high speeds, where the lower limit of the speed range can be between 150 km / h and 180 km / h, and the upper limit can be between 185 km / h and 300 km / h. For example, a user in a vehicle traveling at high speed can leave a message such as "I'm here!" in a virtual event, and this message is only visible to or experienced by the user if the user is in a vehicle traveling at a speed within the described high-speed range. For traffic lights, virtual events can be generated for the traffic light's status, where the traffic light is red or green. For example, a user can leave a message such as "This traffic light was red when I arrived," and this virtual event is only visible to the user if the user's vehicle arrives at the traffic light when it is red. The speed and / or status of traffic lights can be signaled by the electronic control unit of the corresponding vehicle monitoring the speed or real-world environment. Traffic conditions can be described using traffic data received from traffic monitoring systems connected to vehicles. Traffic conditions could be, for example, traffic congestion, allowing users to leave comments such as "There's always traffic congestion in the morning," and the virtual event would only be presented to the consumer if they were also experiencing traffic congestion in the trigger area. Weather can be derived from weather data received from weather stations. Users can leave comments such as "This place looks beautiful when it's sunny," and users can define that the virtual event should be displayed whenever the consumer is in the trigger area without sunlight (e.g., at night or on a cloudy day). This informs the consumer that the real-world environment in the trigger area looks beautiful when it's sunny.

[0025] Because event data can be very large, user devices typically cannot store all available event data. One embodiment may include a server unit generating map data from the stored event data of virtual events and providing the map data to the vehicle, wherein the map data describes the corresponding triggering area of ​​the virtual event. The data volume or amount of map data (e.g., measured in megabytes) is less than or less than the amount of event data because the details or content of the virtual event does not need to be included in the map data. In the corresponding vehicle, based on the map data, the corresponding user device monitors the vehicle's current vehicle position and generates request data if the current vehicle position indicates that the vehicle is approaching or entering or is located within the triggering area. Thus, in each vehicle, the corresponding interactive user device can manage or observe where the boundaries or limits of the corresponding triggering area are located. Such a triggering area can be, for example, a single point or area defined by a polygon or geometry or a segment of a driving route. This allows the user device to monitor for itself whether event data should be requested from the server for a particular virtual event. This reduces the amount of data that needs to be transmitted from the server to the corresponding vehicle, and only map data is needed to determine whether the virtual event needs to be presented in the corresponding user device.

[0026] One embodiment may include defining data describing at least one of the following virtual events:

[0027] -Messages displayed in a virtual environment

[0028] - Camera images of real-world environments,

[0029] - Screenshot of the virtual environment

[0030] - Define the user's image for the data,

[0031] -Video stream,

[0032] -Attached image

[0033] -Audio messages,

[0034] -Tactile signals,

[0035] - Link to the song's media file.

[0036] Therefore, when at least one consumer passes through the trigger area, the creator can use virtual events to present graphical and / or auditory and / or tactile messages or experiences to the consumer.

[0037] One embodiment may include defining data describing interactive virtual events in which at least two users interact in real time via their respective user devices, i.e., simultaneously accessing the virtual environment. The creating user and the consuming user can thus have a real-time conversation. The virtual event used for such a conversation can be, for example, video conferencing and / or user interaction or other tasks. The creating user can create virtual events, but once the virtual event is accessed or activated by the consuming user, both users can participate or provide signals, such as video streams and / or text messages and / or audio messages, which can be exchanged between user devices to enable the conversation.

[0038] One embodiment may include defining data that also includes a geometric position set by the user, where the geometric position describes the geometric offset of the virtual event relative to the vehicle's position or the absolute position of the virtual event in the real-world environment. The user can thus move the position of the virtual event, for example, to display a message board. The corresponding user device displaying the virtual event projects the virtual event onto the geometric position. The user can input the geometric position by manually defining coordinates using, for example, a portable controller on the user device and / or by moving new events in the virtual environment. If a relative position is defined using a geometric offset, the virtual event will follow the vehicle as it moves in the real-world environment. If an absolute position is defined, the virtual event can be connected to and / or attached to objects in the real-world environment (in the case of augmented reality) or objects within the virtual event (in the case of virtual reality). This allows the virtual event to be associated with objects visible to the consuming user when experiencing or accessing the virtual event. For example, the user can generate virtual events that include comments about the corresponding object (e.g., "What a nice building!").

[0039] One embodiment may include: registering users in at least one user group, and associating at least one virtual event with a corresponding user group, and the server unit providing event data for the virtual event only to the corresponding user devices of users also registered in the user group associated with the virtual event. This allows access to the corresponding virtual environment to be delimited or restricted to a limited set of viewers or users. For example, a creating user can invite a group of friends (where each friend is a consuming user) to experience or access a new event. Unknown consuming users that the creating user wants to exclude from the new virtual event will not be presented with the virtual event created by the creating user.

[0040] One embodiment may include: monitoring user interaction with the virtual event when it is displayed to one of the users in the corresponding vehicle, describing the interaction through interaction data provided to a server unit, and adapting / adjusting event data of the virtual event, wherein the adapted / adjusted event data describes the result of the interaction. For example, a consumer user can become a creator user by leaving a reply or comment on a consumer virtual event created by another creator user. This allows users to interact in a non-real-time manner. For example, a consumer user can leave a symbol indicating that the consumer user likes the virtual event (as an interaction). This can signal the creator user who created or generated the virtual event.

[0041] One embodiment may include: via a corresponding user device, a corresponding virtual event is displayed in a shared virtual environment context overlaid on a real-world environment. Such a virtual environment context can be a virtual scene where the user can leave or locate their virtual events as objects or content. By providing a virtual environment context, the user device generates a virtual reality scene of the real-world environment that is not visible to the user. As known from the prior art, the virtual environment context can represent objects from the real-world environment, such as the road a vehicle is currently traveling on.

[0042] One embodiment may include projecting a corresponding virtual event into the field of vision of a corresponding user in a real-world environment via a corresponding user device. This is an augmented reality where at least one virtual event, such as an object, is presented to the user while the user is paying attention to or viewing the real-world environment.

[0043] As a solution for achieving the above objectives, the present invention provides a system comprising a server unit for receiving, storing, and transmitting event data, and at least one interactive user device for a corresponding vehicle, wherein the interactive user device is designed to display virtual events to a user in the vehicle and receive definition data of new virtual events from the user, wherein the server unit and the processing unit of the at least one interactive user device are designed to interact according to one of the embodiments of the described method. In other words, the system provides hardware or components for performing the method of the present invention.

[0044] As part of the solution for achieving the above-described objectives, the present invention provides a server unit for the system. The server unit can operate as described in conjunction with the method of the invention. In other words, the server unit includes a processing unit designed to store event data describing at least one virtual event in a virtual environment, wherein the virtual event is associated with a corresponding real-world triggering area in the real-world environment. If corresponding request data indicating that the vehicle is approaching, entering, or located in a corresponding triggering area of ​​one of the virtual events is received from the vehicle's interactive user equipment, the event data of the corresponding virtual event is sent to the vehicle's user equipment for displaying the virtual event as part of the virtual environment to the user in the vehicle. To generate the event data for the corresponding virtual event, the processing unit is designed to receive both event data and the vehicle's location data from at least one interactive user equipment in the vehicle when the user generates definition data, wherein the event data describes the user-defined new virtual event, and the vehicle's location data describes the vehicle's location in the real-world environment. The server unit combines the event data and the location data to create the event data for the corresponding virtual event and the triggering area in the real-world environment where the virtual event resides. The server unit may include a computer or a network of computers, such as a cloud computer. The server unit can be connected to the Internet, and the communication link between the server unit and the corresponding vehicle can be based on an Internet connection and / or a wireless connection. To establish a communication link between the vehicle and a fixed network, WiFi technology and / or mobile phone technology (e.g., 5G) can be used.

[0045] As part of the solution for achieving the above objectives, the present invention provides an interactive user equipment for a vehicle, wherein the user equipment includes a processing unit designed to:

[0046] - Receive definition data from the user residing in the vehicle, whereby the definition data describes new virtual events in the user-defined or automatically generated virtual environment.

[0047] - When the definition data is generated, vehicle position data is received from the vehicle's position sensors, wherein the position data describes the vehicle's current position in the real-world environment.

[0048] - Provide definition data and location data to the described server unit embodiment.

[0049] - Send request data to the server unit to receive event data, and

[0050] - Display the received event data to the user as part of the virtual environment.

[0051] The user equipment may be designed to include an HMD for displaying the VR and / or AR environment as described above. To receive user input, the user equipment may include at least one portable menu controller and / or keyboard and / or virtual keyboard and / or gesture recognition system, which may detect the user's hand, for example, based on camera data, and may recognize gestures performed by the user, for example, based on artificial neural networks and / or hidden Markov models. The user equipment may monitor the current vehicle position based on position data generated by a receiver of GNSS position signals. The user equipment may compare the current position with map data describing a trigger area for a virtual event. If approaching or entering the trigger area, or if the vehicle is already within the trigger area, request data for requesting event data may be sent to a server unit. To perform the method steps, the user equipment includes a processing unit, which may include at least one microprocessor and / or at least one ASIC (Application-Specific Integrated Circuit). To perform the method steps, software or program code may be stored in the data memory of the processing unit.

[0052] As part of the solution for achieving the above objectives, the present invention provides a vehicle including at least one user device, said user device being an embodiment of the aforementioned interactive user device. The vehicle of the present invention is preferably designed as a motor vehicle, particularly a passenger vehicle or truck, or as a passenger bus.

[0053] The present invention also includes combinations of features from different embodiments. Attached Figure Description

[0054] Exemplary embodiments of the present invention are described below. The accompanying drawings illustrate:

[0055] Figure 1 This is a schematic diagram of an embodiment of the system of the present invention; and

[0056] Figure 2 This is a diagram illustrating the driving situation. Detailed Implementation

[0057] The embodiments described below are preferred embodiments of the present invention. However, in these embodiments, each component represents a separate feature of the invention, which is considered independent of each other and each independently develops the invention, and is thus considered a component of the invention in a separate manner or in a manner different from the combinations shown. Furthermore, the embodiments may also be supplemented by other features of the invention already described.

[0058] In the accompanying drawings, the same reference numerals indicate elements that provide the same function.

[0059] Figure 1A system 10 is illustrated, which may include a back-end server unit or (for short) server unit 11 and multiple vehicles 12. In each vehicle 12, at least one interactive user device 13 may be provided, which may be designed as a head-mounted device (HMD) or at least may include an HMD. The communication link 14 between the respective user device 13 and the server unit 11 may operate based on, for example, an internet connection, which may be IP-based (Internet Protocol) and / or based on a wireless connection, such as Wi-Fi and / or mobile communication.

[0060] The corresponding user device 13 can be based on, for example, a VR engine 15 to provide information to the user. Figure 1 (Not shown) A virtual environment, particularly VRA or VR, is presented. The VR engine 15 can present virtual content of a virtual environment 16 that the user can focus on using an HMD. The user can also interact with the virtual environment 16. To dynamically generate the virtual environment 16 with content from other user creators, the corresponding user device 13 can operate an event map 17, which can indicate where the corresponding vehicle 12 is entering in the real-world environment 18. A corresponding trigger area 19 can be arranged or positioned to be associated with a corresponding virtual event 20. For example, the user of user device 13 can see a message 21 indicating whether the vehicle 12 being described by the user is in the trigger area 19, as a possible additional trigger condition 22, such as a traffic light 23 showing a red light 24. The virtual event 20, i.e., in this example, the message 21 related to, for example, the red light 23, can then be presented to the user in the virtual environment 16.

[0061] To monitor different trigger areas 19, the processing unit 25 of user equipment 13 can send map request data 26 to server unit 11. Server unit 11 can operate event provider 27, i.e., a software module or an interface that can provide an API (Application Programming Interface) for receiving and processing map request data 26. Server unit 11 may include data storage 28, where event data 30 for different virtual events 20 can be stored, for example, in event database 29. Based on event data 30, server unit 11 can generate or provide map data 31 to populate or generate an event map, for example, by means of the event provider. User equipment 13 can monitor the current position 32 of vehicle 12 and optionally the current orientation / direction of travel 33 of vehicle 12. For example, for GNSS position signal 36, position data 34 can be received from receiver 35. Based on position data 34, user equipment 13 can locate vehicle 12 or the current position 32 of vehicle 12 with respect to event map 17. If proximity and / or entry or location is detected within the trigger area 19 as indicated by event map 17 based on location data 34, request data 37 can be sent to server unit 11 to request corresponding event data 30, which can be sent by, for example, event generator 38 of server unit 11. Event generator 38 can be, for example, software program code or an interface, such as an API. Event reconstructor module 39, or generally the software module of processing unit 25, can generate a corresponding event 20 based on the received event data 30 (i.e., graphics and / or audio and / or top content that should be dynamically integrated into virtual environment 16). Once virtual event 20 is integrated into virtual environment 16, it becomes accessible, or it can be experienced by a user of user device 13. The user can interact with virtual event 20 using user device 13. This interaction can generate interaction data 40 that can be sent to server unit 11. For example, server unit 11 can operate the corresponding interface or API as state event generator 41. Interaction data 40 can be stored individually for each user in the user database 42 of the data storage 28, where different users and their user interaction data 40 can be grouped, for example, into user groups 43. Based on the interaction data 40, adapted user events 44 can be generated that are only valid for members of user group 43. Similarly, event data 30 for different virtual events 20 can also be grouped into user groups 43, so that only members of user group 43 or users registered to user group 43 can experience or access the corresponding virtual event 20.

[0062] Therefore, the user's vehicle driving experience is enhanced through geographic, dynamic, and interactive events influenced by other users. This enables gamification approaches (e.g., high scores in arcade games at red lights) and other interactions with other users (e.g., sharing news, videos, etc. at geographic locations).

[0063] Figure 2 This illustrates how event data 30 corresponding to a specific user event 20 can be generated by the corresponding user 45, who is the creator of the user. The user can use and operate the user device 13 while the user 45 is in the vehicle 12. The user device 13 may include a controller 46 for providing input data or defining data 47. For example, the user 45 can access or focus on a virtual environment 16, which... Figure 2 The present invention is augmented reality (AR), which allows objects 48 in a building or general real-world environment 18 to be associated with or augmented with virtual events 20—such as messages left by user 45 for at least one other user using the corresponding user device 13. For example, using controller 46, user 45 can set a relative or absolute geometric position 49 relative to object 48, or as an absolute position in the real-world environment 18. For example, geographic coordinates and optional height values ​​can be defined. By selecting the type and geometric position 49 of virtual event 20, definition data 47 is defined, which can be sent to server unit 11 along with current position data 34 of current vehicle position 32. In server unit 11, definition data 47 and position data 34 can be used to generate event data 30 for virtual event 20 and map data 31 for defining the corresponding trigger area 19. Then, as already based on... Figure 1 As explained, event data 30 and map data 31 are available for use by other user devices 13.

[0064] System 10 can generate and display user-triggered and / or automatically generated geographic-based interactive virtual events, which are stored in a server unit serving as the backend. These virtual events can then be signaled to different VR users / vehicles to create an event geographic map. When a user enters one or more stored event / interaction corresponding geographic areas (triggered areas), it can be dynamically integrated into the VR experience. Events can be interacted with between VR users.

[0065] In summary, within the virtual environment system 10, navigation and vehicle data will be exchanged between the vehicle and the HMD (i.e., the interactive user device) to create a VR environment and visualize vehicle activity in VR. In addition to this data, dynamic, interactive, and geo-based creation information for events / interactions can now be exchanged. To facilitate information exchange between the vehicle and the HMD, the interface can be extended (e.g., via Bluetooth Low Energy or WiFi), but it is also possible for the HMD to communicate directly with the infrastructure of the server unit via mobile communication standards (e.g., 5G).

[0066] In virtual environment applications, information is encoded into the "real-world" environment in the form of the geometry of trigger areas on a road network related to GPS coordinates and, preferably, the vehicle's direction of travel. Furthermore, events may include metadata, comprising values ​​from environmental factors, time factors, and / or random parameters. The process of creating events can be triggered on the system side via (partially) automatic or manual methods, and through user input or interaction. A triggered event signifies the download of user event data, which can be executed, displayed, or otherwise utilized on the user side, i.e., in the vehicle where the user resides on the user's device. User event data can include both user-specific data (e.g., user group information or personal data) and cross-user data.

[0067] The purpose of the system is to trigger a virtual event when the area marked by the user's (i.e., the user device used by the user) geographic location and event data is within or on the stored geometry of the virtual event, preferably also depending on the direction of travel and / or other predefined factors.

[0068] This first requires iteratively checking the geographic location of the user domain (i.e., the user device in the corresponding vehicle). Based on this, for example with the help of an event provider, nearby available virtual events and the specific conditions for triggering them are retrieved from the event database as map data in the backend and stored in the event map on the user device. All stored events in the VR environment generated by the user device are continuously checked to see if they will be triggered. Once the user device detects a trigger for an event in the currently displayed VR environment, it signals a request for corresponding event data to be sent to an event generator, for example, in the backend server unit, which then initiates the download of user event data, which is processed for the VR environment in the user domain portion (i.e., the user device), for example in an event reconstructor within the user device. When the event data is executed, displayed, or otherwise used, some state information of the user can also be stored as interactive data in the backend server unit and taken into account when the same virtual event is triggered again or when other related virtual events are triggered.

[0069] In summary, this example demonstrates how to deliver real-world location-based events within a dynamic virtual environment.

Claims

1. A method for presenting a shared virtual environment (16) to a user (45) residing in a respective vehicle (12) while different vehicles (12) are traveling in a real-world environment (18), wherein in at least one of the vehicles (12), - The corresponding interactive user equipment (13) of the vehicle (12) receives definition data (47) from the corresponding user (45) residing in the vehicle (12) and / or generates definition data (47) based on the current driving status of the vehicle (12), wherein the definition data (47) describes new virtual events (20) for the virtual environment (16); and - When the user (45) generates definition data (47), the vehicle's (12) position sensors generate vehicle (12) position data (34), wherein the position data (34) describes the current position of the vehicle (12) in the real-world environment (18), and The stationary server unit (11) receives definition data (47) and corresponding location data (34) from the vehicle (12), and generates event data (30) corresponding to the new virtual event (20), wherein a triggering area (19) is defined based on the location data (34), which defines where the virtual event (20) should be triggered in the real-world environment (18). The server unit (11) receives corresponding request data (37) from the corresponding user equipment (13) of at least one other vehicle in the vehicle (12), wherein the request data (37) indicates that the vehicle (12) is approaching or entering or is located in the corresponding triggering area (19) of a virtual event (20), and the server unit (11) sends the corresponding event data (30) of the virtual event (20) to the interactive user equipment (13) of the vehicle (12), and the corresponding virtual event (20) is displayed to the user (45) in the vehicle (12) as part of the virtual environment (16).

2. The method according to claim 1, wherein, When the corresponding user equipment (13) of at least one of the vehicles (12) receives the corresponding definition data (47) of a new virtual event (20), the user equipment (13) also receives condition (22) data, wherein the condition (22) data describes the current environmental conditions (22) and / or the current state of the vehicle (12), and The server unit (11) receives condition (22) data and generates at least one corresponding trigger condition (22) for a new virtual event (20), wherein the virtual event (20) is only displayed to the user (45) residing in a vehicle (12) that also satisfies the at least one trigger condition (22).

3. The method according to claim 2, wherein, The at least one triggering condition (22) includes: - The speed of vehicle (12) has a value within a predetermined range. -Traffic lights (23) in the real-world environment (18) have predefined states, - Traffic is in a predefined state. -Weather in real-world environments (18), -The specific direction of travel of the vehicle (12) (33), - The driving destination is notified by the navigation system (10) using signals.

4. The method according to any one of the preceding claims, wherein, The server unit (11) generates map data (31) based on the event data (30) of the stored virtual event (20), and provides the map data (31) to the vehicle (12), wherein the map data (31) describes the corresponding triggering area (19) of the virtual event (20), and In the corresponding vehicle (12), the corresponding user equipment (13) monitors the current vehicle (12) position of the vehicle (12) according to map data (31), and generates request data (37) if the current vehicle (12) position indicates that the vehicle (12) is approaching or entering or located in the trigger area (19).

5. The method according to any one of the preceding claims, wherein, Define data (47) to describe at least one of the following virtual events (20): - The message (21) displayed in the virtual environment (16), - Camera images of the real-world environment (18), - Screenshot of the virtual environment (16), - Defines the image of the user (45) who defines the data (47), -Video stream, -Attached image -Audio message (21), -Tactile signals, -A link to the song's media file.

6. The method according to any one of the preceding claims, wherein, Define data (47) to describe an interactive virtual event (20) in which at least two users (45) interact in real time via their respective user devices (13).

7. The method according to any one of the preceding claims, wherein, The definition data (47) also includes a geometric position (49) set by the user (45), wherein the geometric position (49) describes the geometric offset of the virtual event (20) relative to the vehicle (12) or the absolute position of the virtual event (20) in the real-world environment (18); and The corresponding user device (13) that is displaying the virtual event (20) projects the virtual event (20) onto the corresponding geometric position (49) in the virtual environment (16).

8. The method according to any one of the preceding claims, wherein, The user (45) is registered in at least one user group (43), and at least one virtual event (20) is associated with the corresponding user group (43), and the server unit (11) provides the event data (30) of the virtual event (20) only to the corresponding user device (13) of such user (45), that is, the user (45) is also registered in the same user group (43) associated with the virtual event (20).

9. The method according to any one of the preceding claims, wherein, When a virtual event (20) is displayed to one of the users (45) in the corresponding vehicle (12), the interaction between that user (45) and the virtual event (20) is monitored, and the interaction is described by interaction data (40) provided to the server unit (11), and the server unit (11) adapts the event data (30) of the virtual event (20), wherein the adapted event data (30) describes the result of the interaction.

10. The method according to any one of the preceding claims, wherein, The corresponding virtual events (20) are transmitted through the corresponding user equipment (13). -As a virtual reality (VR) display in the background of a public virtual environment (16) superimposed on the real-world environment (18), or - As augmented reality (AR) is projected onto the real-world environment (18) in the field of vision of the corresponding user (45).

11. A system (10) for a corresponding vehicle (12), comprising a server unit (11) for receiving, storing, and transmitting event data (30) and at least one interactive user device (13), wherein, The interactive user equipment (13) is designed to display virtual events (20) to a user (45) in a corresponding vehicle (12) and receive definition data (47) of new virtual events (20) from the user (45), wherein the server unit (11) and the corresponding processing unit (25) of the at least one interactive user equipment (13) are designed to interact using the method according to any one of the preceding claims.

12. A server unit (11) for use in the system (10) according to claim 11, wherein, The server unit (11) includes a processing unit (25) designed to: store event data (30) describing at least one virtual event (20) of a virtual environment (16), wherein the virtual event (20) is associated with a corresponding real-world triggering region (19) in a real-world environment (18); and If a request data (37) is received from the interactive user equipment (13) of the vehicle (12) indicating that the vehicle (12) is approaching or entering or located in the corresponding trigger area (19) of a virtual event (20), then the event data (30) of the corresponding virtual event (20) is sent to the user equipment (13) of the vehicle (12) for displaying the virtual event (20) as part of the virtual environment (16) to the user (45) in the vehicle (12), wherein In order to generate event data (30) for the corresponding virtual event (20), the processing unit (25) is designed to receive event data (30) from the interactive user equipment (13) of at least one of the vehicles (12), wherein the event data (30) describes the new virtual event (20), and - When definition data (47) is generated in the vehicle (12), the location data (34) of the vehicle (12) is received, which describes the location of the vehicle (12) in the real-world environment (18), and The server unit (11) combines event data (30) and location data (34) to create event data (30) of the corresponding virtual event (20) and the definition of the triggering area (19) in the real-world environment (18) where the virtual event (20) resides.

13. An interactive user equipment (13) for a vehicle (12), wherein, The user equipment (13) includes a processing unit (25), which is designed to: - Receive definition data (47) from the user (45) residing in the vehicle (12), wherein the definition data (47) describes user-defined new virtual events (20) for the virtual environment (16); and - When the user (45) generates definition data (47), position data (34) of the vehicle (12) is received from the vehicle's (12) position sensor, wherein the position data (34) describes the current position of the vehicle (12) in the real-world environment (18); and - Provide definition data (47) and location data (34) to the server unit (11) according to claim 12; and - Send request data (37) to the server unit (11) so that it can receive event data (30); and - Display the received event data (30) to the user (45) as part of the virtual environment (16).

14. A vehicle (12) comprising at least one interactive user equipment (13) according to claim 13.

Images