Method for operating a vehicle and system for carrying out the method

Abstract

To make a digital space tangible for vehicle occupants, data (O1, O2, O3, O4, O) are extracted from the provided digital space to control actuators (14) of a vehicle interior of the vehicle (1).

Description

[0001] The invention relates to a method for operating a vehicle, primarily focusing on the aspect of how actuators of a vehicle interior are controlled. The invention also relates to a corresponding system.

[0002] The invention also relates in particular to an interaction between a digital space and a vehicle. The digital space can be provided as a two-dimensional or three-dimensional space, in particular as a so-called metaverse. According to Wikipedia, the free encyclopedia, in the version of June 22, 2023, under the keyword "metaverse," a metaverse is a concept in which a digital space is created through the interaction of virtual, augmented, and physical reality.

[0003] The digital space can also be provided using computer program elements.

[0004] The interaction of virtual and physical reality in the manner of a metaverse is known from US 2022 / 024 395 A1 with the physical reality of a racing car: A digital twin of the racing car is created in a virtual world, and this is displayed there in real time with a physically real driving environment and physically real vehicle components.

[0005] US 2022 / 242450 A1 discloses a system for merging a real and virtual environment of a vehicle using a metaverse platform. The virtual environment is adapted to the real environment based on data for a test vehicle; furthermore, objects that do not exist in the real environment are displayed in the virtual world.

[0006] When accessing media content today, the experience has become increasingly intense and thus more authentic thanks to ever-improving output devices (displays, speakers). For vehicle occupants who are not driving the vehicle (front passenger, person in the back seat, or the actual driver, who is able to devote themselves to other activities due to semi-autonomous or fully autonomous driving of the vehicle), there is the option to engage in activities such as accessing media content, playing computer games, or the like.

[0007] CN 1 14 931 749 A shows that a computer game is played in a vehicle, whereby information about sound or noises is taken from the game and a steering wheel of the vehicle is subjected to vibrations that depend on the sound.

[0008] CN 1 12 569 589 A discloses controlling a video game via motor vehicle units and providing a feedback signal from the game. The feedback signal can control the air conditioning, lights, etc.

[0009] The publication US 2017 / 0 278 303 A1 generally concerns the field of obtaining virtual sensor data using a mobile virtual object, i.e. a virtual vehicle.

[0010] The object of the present invention is to allow vehicle occupants to participate in events in the digital space in a better way.

[0011] This object is achieved by a method for operating a vehicle according to patent claim 1 and a system for carrying out this method according to patent claim 9.

[0012] The method according to the invention for operating a vehicle comprises providing a digital space (2D space / 3D space; metaverse; using computer program elements, see above), extracting data from the digital space and using the extracted data to control actuators of a vehicle interior of the vehicle.

[0013] The invention thus introduces the concept of translating data content from the digital space into control commands for actuators. Vehicle occupants can sense / perceive the effects of the actuators and thus better participate in what's happening in the digital space.

[0014] In the invention, color values ​​are extracted from the digital space. Since the digital space can be represented at least by an optical output device, this preferred embodiment can be implemented practically any time, if desired.

[0015] The color values ​​are extracted using a virtual digital camera. Such a virtual digital camera can be moved, aligned, and so on in digital space (here: specifically conceived as a metaverse, in which the virtual camera performs tasks that a real camera would otherwise perform in a real space) and can capture images / photographs. For example, the virtual digital camera can be panned. In one example, a forest is shown on one side of the digital space and a sea on the other. If the digital camera is then panned from the forest to the sea, the color value changes from green to blue.

[0016] The digital space is typically defined at least visually, i.e., by means of data that can be displayed on a screen or other optical output device; optionally, acoustic outputs and haptic outputs (vibration of hand-held elements, etc.) are also possible.

[0017] Extracting data from digital space can involve reading data from a data set that defines the digital space or generating new data from that data. Data can also be captured based on a representation of the digital space (e.g., filming a display, recording acoustic playback with a microphone).

[0018] According to a preferred embodiment of the invention, additional grayscale values ​​and / or brightness values ​​are extracted from the digital space. Since the digital space can be represented at least by an optical output device, this preferred embodiment can be implemented practically any time, if desired.

[0019] In the invention, it is advantageous if the vehicle is also provided virtually in the digital space, and the camera captures the digital space from a perspective taken from the viewpoint of the virtually displayed vehicle and / or the vehicle's surroundings. Thus, the vehicle can move through a landscape in a computer game and has a camera on the roof that can capture the color value of the changing landscape.

[0020] In particular in connection with the recording of color values, but theoretically also possible when capturing (extracting) data with a modality different from optical (acoustic, haptic detectable), vehicle interior lighting can be controlled.

[0021] In the above example of panning the camera from a forest to the sea, the vehicle's interior lighting can change from green to blue accordingly. In this way, the vehicle's occupants are immersed in the digital space (metaverse, etc.) and experience it directly ("immersion").

[0022] Alternatively or additionally, an air conditioning system, a seat heater, a seat cooler, a seat massage device, and the like can be controlled depending on the extracted data. In this way, the vehicle interior experience becomes physically tangible for the vehicle occupant; this is particularly useful in addition to the embodiment in which the vehicle interior lighting is controlled.

[0023] According to a preferred embodiment of the invention, the digital space is determined (i.e. provided) by a media content and / or a computer game.

[0024] In principle, the computer game can be played using input devices on the vehicle itself, and the media content can be accessed using output devices on the vehicle.

[0025] Preferably, however, there is a connection between the vehicle and a device different from the vehicle, with the device different from the vehicle being used to access the media content and / or play the computer game. The device can be located in the vehicle or even entirely external. In the latter case, the driver's spouse or child, or other vehicle occupant, can play the computer game and, by controlling the vehicle actuators, allow the respective vehicle occupant to participate in the digital space defined by the computer game or media content ("Blue Level reached" or similar).

[0026] According to a preferred embodiment of the invention, data from or in the vehicle is also captured (using sensors) and used to modify the digital space. Thus, in a sense, capture in the reverse direction is also possible.

[0027] The system according to the invention for carrying out the method of the type described above comprises: - a vehicle with a vehicle interior and actuators associated with it; - a data processing device for providing the digital space; an interface for exchanging data between the vehicle and the data processing device, wherein the data processing device comprises means for extracting data from the digital space and is designed to transmit it to the vehicle via the interface, wherein the means for extracting are designed to extract the color values ​​from the digital space and wherein the means for extracting comprise the virtual digital camera, - wherein the vehicle has a control unit designed to control the actuators according to the extracted data.

[0028] For use cases or application situations that may arise during the method and which are not explicitly described here, it may be provided that, in accordance with the method, an error message and / or a request to enter user feedback is issued and / or a default setting and / or a predetermined initial state is set.

[0029] The invention also includes the control device for the motor vehicle. The control device can have a data processing device or a processor device that is configured to carry out an embodiment of the method according to the invention. For this purpose, the processor device can have at least one microprocessor and / or at least one microcontroller and / or at least one FPGA (Field Programmable Gate Array) and / or at least one DSP (Digital Signal Processor). In particular, a CPU (Central Processing Unit), a GPU (Graphical Processing Unit), or an NPU (Neural Processing Unit) can be used as the microprocessor. Furthermore, the processor device can have program code that is configured to carry out the embodiment of the method according to the invention when executed by the processor device. The program code can be stored in a data memory of the processor device.The processor device can be based, for example, on at least one circuit board and / or on at least one SoC (System on Chip).

[0030] The invention also includes further developments of the system according to the invention, in particular of the associated motor vehicle, which have features already described in connection with the further developments of the method according to the invention. For this reason, the corresponding further developments of the system / motor vehicle according to the invention are not described again here.

[0031] The motor vehicle used according to the invention is preferably designed as a motor vehicle, in particular as a passenger car or truck, or as a passenger bus or motorcycle.

[0032] As a further solution, the invention also encompasses a computer-readable storage medium comprising program code which, when executed by a computer or computer network, causes the computer to carry out an embodiment of the method according to the invention. The storage medium can be provided at least partially as a non-volatile data memory (e.g. as a flash memory and / or as an SSD - solid state drive) and / or at least partially as a volatile data memory (e.g. as a RAM - random access memory). The storage medium can be arranged in the computer or computer network. However, the storage medium can also be operated on the Internet, for example, as a so-called app store server and / or cloud server. The computer or computer network can provide a processor circuit with, for example, at least one microprocessor.The program code may be provided as binary code and / or as assembly code and / or as source code of a programming language (e.g. C) and / or as a program script (e.g. Python).

[0033] The invention also encompasses combinations of the features of the described embodiments. The invention therefore also encompasses implementations that each comprise a combination of the features of several of the described embodiments, unless the embodiments are described as mutually exclusive.

[0034] Exemplary embodiments of the invention are described below. Shown are: the single figure schematically illustrates the structure of a system for carrying out the method according to an embodiment.

[0035] The exemplary embodiments explained below are preferred embodiments of the invention. In the exemplary embodiments, the described components of the embodiments each represent individual features of the invention that can be considered independently of one another, each of which also develops the invention independently of one another. Therefore, the disclosure is intended to encompass combinations of the features of the embodiments other than those shown. Furthermore, the described embodiments can also be supplemented by further features of the invention already described.

[0036] In the figures, the same reference symbols designate elements with the same function.

[0037] A motor vehicle, designated as a whole by 1, comprises a central control unit 10, which receives data from sensor devices 12 and transmits control commands to actuators 14. Of interest here are actuators for the vehicle interior, such as, in particular, actuators for the vehicle interior lighting ("ambient lighting"), the vehicle air conditioning (blowers and the like), and other peripheral devices such as devices related to the vehicle seat.

[0038] A data processing device, designated as a whole by 2 in the figure, serves to provide a digital space, in particular a metaverse. It comprises a memory 20 for a computer program. Alternatively or additionally, media content (video, film, etc.) can be stored in the memory. The computer program can be run, or the media content retrieved, by being called in or by a processor 22, which uses a working memory 24 and writes data therein. Data is read from the working memory 24 and sent to an output device 30, which can be imagined, among other things, as a display. (Alternatively, according to the dashed arrow, it is possible for the output device to be controlled directly by the processor 22, which then accesses the data from the working memory 24.)) A computer game also includes an input device, which is designated 26 in the present case. In a computer game, a user can, for example, use the output device 30 to orient themselves as to where an avatar / virtual vehicle or the like is located. They can then make inputs which are fed to the processor 22 as input data I1 (I for “input”). The motor vehicle has a communications interface 18, which in turn is connected to a communications interface 28 of the data processing device 2. Optionally, data from the motor vehicle can also be fed to the processor 22 as input I2. For example, such data can indicate where the vehicle is located (with location measurement by the sensor device 12. For example, “GPS” (Global Positioning System), a vehicle speed can be recorded, and the like.The use of second input data I2 in addition to the first input data I1 can be useful if the vehicle plays a role in the digital space (metaverse) generated by the processor 22 with the data supplied to the main memory 24, for example, if it is shown itself. (Alternatively, only the data I2 are supplied to the processor 22 and no data of the type of input data I1. Input data I1 or I2 are not mandatory.)

[0039] In the present case, actuators 14 in the vehicle 1, which are related to the vehicle interior, are to be controlled in such a way that this corresponds in some way to the digital space. Accordingly, data must be extracted from the digital space. This can be done using output data O1 (where O stands for "output"), which originates directly from the processor 22; output data O2 can be tapped, which corresponds to the control commands that correspond to the transmission of the data from the main memory 24 to the output device ("RGB" values ​​and similar color, gray, and brightness codes), or they can be provided by the output device as output data O3. Furthermore, passive sensors (in the case of an optical output device, for example a camera, symbolized here as 32) can optionally provide output data O4.Regardless of whether O1 and / or O2 and / or O3 and / or O4 are detected, output data O are ultimately sent via the communication interface 28 of the data processing device 2 to the communication interface 18 of the vehicle 1 and are processed there by the control unit 10 and translated into control commands for the actuators of the vehicle interior.

[0040] As an alternative to the embodiment shown in the single figure, external data sources can also be used; for example, the data processing device 2 can access a server 3. Furthermore, the computer game can run on an external processor of an external server. This can be particularly the case for computer games in which multiple participants interact with each other, with the participants being represented by avatars and / or virtual representations of their vehicles and moving within one and the same metaverse.

[0041] The function of using the communication interface 28 in communication with a vehicle 1 can be accessed, for example, through an application on the data processing device ("app"). The data processing device 2 can be designed, in particular, as a tablet computer, smartphone, smartwatch, or other wearable device, and the like. The data processing device 2 can also theoretically be integrated into the vehicle 1 or, conversely, be located entirely outside the vehicle.

[0042] Overall, the examples demonstrate how a modular software component, which can be called a metaverse-to-vehicle cabin bridge, can be deployed, coupling the digital world with a vehicle cabin.

Claims

[1] Method for operating a vehicle (1), comprising: - Providing a digital space; - Extracting data from the digital space, wherein color values ​​are extracted from the digital space and wherein the color values ​​are extracted by means of a virtual digital camera; - Using the extracted data to control actuators (14) of a vehicle interior of the vehicle (1). [2] Method according to claim 1, in which grey values ​​and / or brightness values ​​are also extracted from the digital space. [3] Method according to claim 1 or 2, wherein the vehicle (1) is provided virtually in the digital space and the camera captures the digital space taking a perspective from the viewpoint of the virtually represented vehicle (1) and / or the surroundings of this vehicle (1). [4] Method according to one of claims 1 to 3, in which a vehicle interior lighting is controlled. [5] Method according to one of claims 1 to 4, in which an air conditioning system, a seat heating system, a seat cooling system and / or a seat massage device is controlled. [6] Method according to one of claims 1 to 5, wherein the digital space is determined by a media content and / or a computer game. [7] Method according to claim 6, wherein the media content is retrieved and / or the computer game is played using a device different from the vehicle (1). [8] Method according to one of claims 1 to 7, in which data from or in the vehicle (1) are also recorded and used to change the digital space. [9] System for carrying out the method according to any one of claims 1 to 8, comprising: - a vehicle (1) with a vehicle interior and actuators (14) associated therewith; - a data processing device (2) for providing the digital space; - an interface (18, 28) for exchanging data between the vehicle (1) and the data processing device (2), wherein the data processing device comprises means for extracting data (O1, O2, O3, O4; O) from the digital space and is designed to transmit these to the vehicle (1) via the interface (28), wherein the means for extracting are designed to extract the color values ​​from the digital space and wherein the means for extracting comprise the virtual digital camera, wherein the vehicle (1) has a control unit (10) which is designed to control the actuators (14) in accordance with the extracted data.

Images