Method and device for synchronous visualization

Abstract

The invention relates to a method for synchronously visualizing at least one parameter (FP1 to FPn) of a vehicle and / or at least one parameter of at least one vehicle function (FFP1 to FFPm) and / or at least one parameter (IP1 to IPx) of a vehicle occupant, wherein - the at least one parameter (FP1 to FPn, FFP1 to FFPm, IP1 to IPx) and a voice input (SE) of the vehicle occupant are fed to a control unit (4), - the at least one parameter (FP1 to FPn, FFP1 to FFPm, IP1 to IPx) and the voice input (SE) are combined by means of the control unit (4) using a generative artificial intelligence (KI), - depending on the combination, contents of an image sequence (BF) correlating with the at least one parameter (FP1 to FPn, FFP1 to FFPm, IP1 to IPx) and the voice input (SE) are generated by means of the generative artificial intelligence (KI), and - the image sequence (BF) is output on at least one display unit (5) visible to the vehicle occupant synchronously with a progression of the at least one parameter (FP1 to FPn, FFP1 to FFPm, IP1 to IPx). The invention further relates to a device (1) for synchronously visualizing at least one parameter (FP1 to FPn) of a vehicle and / or at least one parameter (FFP1 to FFPm) of at least one vehicle function and / or at least one parameter (IP1 to IPx) of a vehicle occupant.

Description

[0001] Mercedes-Benz Group AG

[0002] Method and device for synchronous visualization

[0003] The invention relates to a method for synchronous visualization.

[0004] The invention further relates to a device for synchronous visualization.

[0005] From KR 2016 0072990 A, a feedback system and a method for providing tailored information to a vehicle user through interaction between a vehicle and a portable device are known. The feedback system comprises a vehicle display unit that uses an electronic control unit to acquire vehicle status information, and a portable device that displays feedback information on the vehicle status on a screen, derived from the acquired vehicle status information. The portable device also displays biofeedback information and lifelog feedback information on the screen by acquiring bioinformation via a health sensor and lifelog information via a detection sensor.By integrating the driver's bioinformation and lifestyle patterns into the vehicle status during the journey, an optimal operating state can be confirmed in a portable device, thereby enabling control of the vehicle's driving behavior.

[0006] Furthermore, a method for image generation is known from CN 117930968 A, comprising the following steps:

[0007] - Detecting an action performed by a user after an initial image has been generated based on descriptive information of an image to be generated provided by the user;

[0008] - Determining an image generation requirement according to the action performed by the user, wherein the image generation requirement is a requirement that should be met when a new image is generated based on the descriptive information of the image to be generated; and

[0009] - Generating a second image according to the description information of the image to be generated according to the image generation requirement.

[0010] German patent DE 102023 101 187 A1 describes a method for operating an entertainment system in a motor vehicle, wherein the entertainment system outputs an individually constructed, artificial illustration for each passenger via an output unit of the entertainment system in the motor vehicle. Parameters comprising input data from a passenger are acquired by a processor via a communication system. The acquired parameters are mapped into the illustration by means of an imaging module comprising a neural network.

[0011] The present invention is based on the objective of providing a novel method and a novel device for the synchronous visualization of at least one parameter of a vehicle and / or at least one parameter of at least one vehicle function and / or at least one parameter of a vehicle occupant.

[0012] The problem is solved according to the invention by a method which has the features specified in claim 1 and by a device which has the features specified in claim 8.

[0013] Advantageous embodiments of the invention are the subject of the dependent claims.

[0014] In the inventive method for the synchronous visualization of at least one parameter of a vehicle and / or at least one parameter of at least one vehicle function and / or at least one parameter of a vehicle occupant

[0015] - at least one parameter and a voice input from the vehicle occupant are fed to a control unit,

[0016] - the control unit uses a generative artificial intelligence to combine at least one parameter and the voice input,

[0017] - Depending on the combination, generative artificial intelligence generates content for an image sequence that correlates with at least one parameter and the voice input, - the image sequence is displayed on at least one display unit visible to the vehicle occupant synchronously with a progression of at least one parameter and

[0018] - at least a diffusion model is used by the generative artificial intelligence to generate the content of the image sequence.

[0019] The present method, through the synchronized output of visual data to the vehicle occupants, enables improved perception of these parameters. In particular, the synchronized output of the image sequence amplifies the effect of each parameter on the vehicle occupant. For example, this can significantly improve the occupant's perception of comfort features in the vehicle, such as massage sequences, as well as bodily actions and reactions, such as breathing and / or heart rate. The synchronous output of the image sequence, achieved through generative artificial intelligence, enhances comfort for the vehicle occupant and prevents discomfort resulting from asynchronicity.This allows for an optimized user experience for vehicle occupants. Furthermore, it can enhance the so-called digital luxury for vehicle occupants, as it enables relatively simple personalization.

[0020] In this context, a diffusion model is understood to be a type of generative model designed to generate images that closely resemble the training data. During training, the input data is distorted, for example, with Gaussian noise. The diffusion model learns how to suppress this noise to reconstruct the original data. Using diffusion models trained in this way, it is possible to generate a wide variety of images based on text specifications. Thus, for example, dynamic content can be output based on so-called "stable diffusion."

[0021] In one possible embodiment of the procedure, the vehicle parameters will be at least an acceleration value of the vehicle and / or

[0022] - a vehicle speed and / or

[0023] - the vehicle's driving dynamics and / or

[0024] - a driving program selection that influences the vehicle's driving dynamics and / or noise generation. Such vehicle parameters are particularly well-suited to being perceived more effectively by the vehicle occupants through the synchronous output of the image sequence.

[0025] In another possible embodiment of the procedure, the parameters of at least one vehicle function are / will be used as parameters of at least one vehicle function.

[0026] - an activation state and / or movement parameter and / or a position of at least one actuator located within the vehicle and designed to generate signals perceptible by a vehicle occupant via haptic and / or mechanosensory and / or vibratory and / or kinesthetic means and / or

[0027] - an activation sequence of actuators arranged in or on a vehicle seat for generating haptic signals and their positions in or on the vehicle seat and / or

[0028] - a sequence of tones from an emitted acoustic signal and / or

[0029] - the volume and / or volume profile of an emitted acoustic signal and / or

[0030] - a sequence of sound signals generated by means of at least one structure-borne sound transducer and / or

[0031] - the strength of at least one sound signal generated by means of at least one structure-borne sound transducer and / or

[0032] - a position and / or an activation state and / or a heat output power of at least one heat source located inside the vehicle and / or

[0033] - a position and / or an activation state and / or a performance of at least one fan located inside the vehicle and / or

[0034] - an activation state and / or temperature control parameter of an air conditioning system intended for climate control of a vehicle interior and / or

[0035] - an activation state and / or ionization parameter of an ionizer designed to ionize air within a vehicle interior. Such vehicle function parameters are particularly well-suited to be perceived more easily by the vehicle occupant through the synchronous output of the image sequence.

[0036] In another possible embodiment of the procedure, the vehicle occupant is used as a parameter.

[0037] - Vital data of the vehicle occupant and / or

[0038] - Position data of body parts of the vehicle occupant and / or

[0039] - Movement data of body parts of the vehicle occupant is used. These parameters of the vehicle occupant are also particularly suitable for being better perceived by the vehicle occupant through the synchronous output of the image sequence.

[0040] In another possible embodiment of the process, generative artificial intelligence uses voice input to recognize the emotions of the vehicle occupant and adapts the content of the image sequence to those emotions. This allows for a simple way of taking the current emotions of the vehicle occupant into account when generating the image sequence.

[0041] In another possible embodiment of the method, the vehicle occupant's emotions are recognized from at least one parameter of the vehicle occupant, and the content of the image sequence is adapted to the occupant's emotions. This also allows for a simple way of taking the vehicle occupant's current emotions into account when generating the image sequence.

[0042] In another possible design of the procedure

[0043] - The generative artificial intelligence generates text from at least one parameter and the speech input to create the content of the image sequence and / or

[0044] - The generative artificial intelligence generates a prompt from at least one parameter and the speech input to create the content of the image sequence, converts the prompt into text and the text into an image of the image sequence.

[0045] Using such a trained generative artificial intelligence or several such trained generative artificial intelligences, the content of the images in the image sequence can be adapted to at least one parameter and the speech input in a simple and reliable manner.

[0046] The device according to the invention for the synchronous visualization of at least one parameter of a vehicle and / or at least one parameter of at least one vehicle function and / or at least one parameter of a vehicle occupant has the following features:

[0047] - at least one acquisition unit for recording at least one parameter,

[0048] - at least one microphone to capture voice input from a vehicle occupant,

[0049] - a control unit which is trained to execute a generative artificial intelligence which combines the at least one parameter and the speech input and, depending on the combination, generates content of an image sequence correlating with the at least one parameter and the speech input using at least one diffusion model, and

[0050] - at least one display unit, linked to the control unit in terms of data technology and visible to the vehicle occupant, which is designed to output the image sequence transmitted by the control unit synchronously to a progression of at least one parameter.

[0051] The device enables improved perception of vehicle occupants by synchronizing the output of visual data to each parameter. In particular, the synchronized output of the image sequence amplifies the effect of each parameter on the occupant. For example, this can significantly improve the occupant's perception of comfort features in the vehicle, such as massage sequences, as well as bodily actions and reactions, such as breathing and / or heart rate. Thanks to the synchronous output of the image sequence to each parameter, achieved through generative artificial intelligence, occupant comfort is enhanced, and discomfort resulting from asynchronicity is avoided.This allows for an optimized user experience for vehicle occupants. Furthermore, it enhances the so-called digital luxury for vehicle occupants, as it enables simple personalization. Exemplary embodiments of the invention are explained in more detail below with reference to the drawings.

[0052] This shows:

[0053] Fig. 1 schematically shows three images generated by a generative artificial intelligence depending on the breathing rate and heart rate of a vehicle occupant.

[0054] Fig. 2 schematically shows four images generated by means of a generative artificial intelligence depending on a voice input from a vehicle occupant.

[0055] Fig. 3 schematically shows a block diagram of a device for the synchronous visualization of at least one parameter of a vehicle and / or at least one parameter of at least one vehicle function and / or at least one parameter of a vehicle occupant and

[0056] Fig. 4 schematically shows a block diagram of a device for the synchronous visualization of parameters of a vehicle function.

[0057] Corresponding parts are marked with the same reference symbols in all figures.

[0058] Figure 1 shows three images B1 to B3 generated by a generative artificial intelligence Kl, which is shown in more detail in Figures 3 and 4, depending on the breathing rate and heart rate of a vehicle occupant.

[0059] In this context, generative artificial intelligence (AI), also referred to as Generative Artificial Intelligence, is understood to be an artificial intelligence trained to generate text, images (B1 to B3), or other media using generative models. Such generative models learn patterns and structures from their input training data and then generate new data with similar properties.

[0060] The vehicle has several vehicle components by means of which various vehicle functions, such as comfort functions, can be generated.

[0061] The vehicle components include, for example, as shown in more detail in Figure 4, massage systems integrated into a vehicle seat 6 with controllable actuators 7.1 to 7.20. Such actuators 7.1 to 7.20 are, for example, air-fillable bladders and / or vibration motors, which can be controlled in specific sequences depending on their position in the vehicle seat 6. So-called static massage bladders, which provide lateral support or lumbar support for the vehicle occupant, can also be included or can be individually inflated.

[0062] Furthermore, the vehicle components include, for example, a sound system in which tones and / or sequences of tones are emitted via loudspeakers and / or sound signals are generated by means of so-called structure-borne sound transducers.

[0063] The vehicle components may also include drives for moving individual elements of a vehicle seat 6, for example a backrest 6.1 or a seat cushion 6.2, which is also shown in more detail in Figure 4.

[0064] Furthermore, the vehicle components may include heat sources located on or around a vehicle seat, for example, so-called heating fields. Examples of such heating fields are neck pillow heating fields, hot stone back heating fields, and heating fields located in cavities around or on the vehicle seat, in which, for example, objects can be heated.

[0065] Alternatively or additionally, the vehicle components may include at least one fan located inside the vehicle for ventilating the vehicle interior and / or at least one ionizer for ionizing air located inside the vehicle interior and / or an air conditioning system intended for climate control of the vehicle interior.

[0066] All these vehicle components generate a haptic, mechanosensory, kinesthetic, vibratory and / or thermally perceptible effect on the body surface of a vehicle occupant and, by means of the corresponding effect, influence the posture and body condition of the vehicle occupant.

[0067] The physical condition of the vehicle occupant is determined, for example, using biometric data or vital signs and bodily data. This determination can be made, for instance, using detection units located within the vehicle, such as a camera and / or steering wheel sensors, and / or wearable devices worn by the occupant, such as a smartwatch. Vital signs recorded include, for example, respiratory rate, heart rate, blood pressure, body temperature, and / or the occupant's stress level.

[0068] Body posture can also be determined using detection units located inside the vehicle, such as a camera. These units capture information such as head position, head movement, body position, and / or body movement to determine posture.

[0069] In addition to visually detectable body posture and detectable vital data, the vehicle occupant can express their physical state caused by the effects, for example through emotions and / or words, which can also be visually and / or acoustically detected by means of a suitable detection unit.

[0070] The posture and physical condition of the vehicle occupant are further influenced by vehicle parameters, in particular the vehicle's driving behavior. Driving behavior is characterized, for example, by at least one acceleration value of the vehicle, such as positive or negative longitudinal acceleration or lateral acceleration, the vehicle's speed, its driving dynamics, and / or a selected driving program that influences the vehicle's driving dynamics and / or noise generation.

[0071] It is generally known that synchronized visualization of effects perceptible to a person—for example, the synchronous output of images B1 to B3 to actions performed by the vehicle components, or the image sequences BF shown in more detail in Figure 4 with content corresponding to the actions—is perceived positively by the person and can also enhance the effect produced by the respective vehicle component. A time lag in the perception of such effects across different sensory channels can cause discomfort and unease in the person.

[0072] To avoid this during operation for a vehicle occupant and simultaneously enhance the occupant's perception of such effects, it is planned to adapt visual content on at least one display unit 5, shown in more detail in Figures 3 and 4, in or around the vehicle to the aforementioned effects and to output the visual content synchronized with the effects. It is therefore planned to convert the current activation state of a aforementioned vehicle component, in particular a vehicle component for generating a comfort function, and / or the current status of bio- and / or body data and / or vehicle parameters into a generated image B1 to B3 or an image sequence BF by assigning a text module to the respective activation state of the vehicle component, in order to provide the vehicle occupant with synchronized information across all sensory channels. The image B1 to B7 or...The image sequence BF is displayed on at least one display unit 5.

[0073] The generation of image B1 to B3 or image sequence BF and the synchronous output on the at least one display unit 5 are carried out automatically by means of the generative artificial intelligence Kl. In particular, a synchronous visualization of at least one parameter of the vehicle and / or at least one parameter of at least one of the aforementioned vehicle functions and / or at least one parameter of the vehicle occupant is carried out depending on at least one speech input SE of the vehicle occupant supplied to the generative artificial intelligence Kl and shown in more detail in Figures 3 and 4.

[0074] In one possible configuration, the generative artificial intelligence Kl for generating the content of images B1 to B3 or image sequence BF includes at least a diffusion model, for example a so-called stable diffusion model.

[0075] Images B1 to B3 are generated by the generative artificial intelligence (AI) Kl based on the respiratory and heart rates of the vehicle occupants. This is achieved, for example, using a so-called default prompt, which defines the basic aesthetic to be generated for images B1 to B3 by AI Kl, such as in a stable diffusion process. The default prompt is characterized by a basic style, which may be chosen based on the vehicle manufacturer, a futuristic look, high-contrast lighting, and a 16:9 aspect ratio. Depending on the captured parameters of the vehicle occupants, particularly the vital signs of respiratory and heart rates, the default prompt can be supplemented with keywords.For example, in cases of high respiratory rate and high heart rate, low-contrast lighting and a smooth progression of images B1 to B3 can be chosen instead of high-contrast lighting.

[0076] For example, if the vehicle occupant parameters "low breathing rate" and "low heart rate" are recorded, the default prompt is supplemented with keywords such as "high-contrast lighting" and "expressive," so that image B1 is generated using the generative artificial intelligence Kl. If, for example, the vehicle occupant parameters "low breathing rate" and "medium heart rate" are recorded, the default prompt is supplemented with keywords such as "high-contrast lighting" and "flowing," and an unspecified and appropriately adapted image is generated using the generative artificial intelligence Kl.For example, if parameters such as "low breathing rate" and "high heart rate" are recorded for the vehicle occupant, the default prompt is supplemented with keywords such as "high-contrast lighting" and "rest," and an unspecified and appropriately adapted image is generated using the generative artificial intelligence Kl.

[0077] For example, if the vehicle occupant's parameters "medium breathing rate" and "low heart rate" are recorded, the default prompt is supplemented with keywords such as "medium contrast lighting" and "expressive," so that image B2 is generated using the generative artificial intelligence Kl. If, for example, the vehicle occupant's parameters "medium breathing rate" and "medium heart rate" are recorded, the default prompt is supplemented with keywords such as "medium contrast lighting" and "flowing," and an unspecified and appropriately adapted image is generated using the generative artificial intelligence Kl.For example, if the vehicle occupant's parameters are "medium breathing rate" and "high heart rate," the default prompt is supplemented with keywords such as "medium contrast lighting" and "calm," and the generative artificial intelligence (AI) generates an image that is not shown in detail and is adjusted accordingly. If, for example, the vehicle occupant's parameters are "high breathing rate" and "low heart rate," the default prompt is supplemented with keywords such as "low contrast lighting" and "expressive," so that the generative artificial intelligence (AI) generates image B2.For example, if the vehicle occupant's parameters "high breathing rate" and "medium heart rate" are recorded, the default prompt is supplemented with keywords such as "low contrast lighting" and "flowing," and a corresponding, unspecified image is generated using generative artificial intelligence (AI). Similarly, if the vehicle occupant's parameters "high breathing rate" and "high heart rate" are recorded, the default prompt is supplemented with keywords such as "low contrast lighting" and "rest," and a corresponding, unspecified image is generated using generative artificial intelligence (AI).

[0078] Figure 2 shows four images B4 to B7 generated by a generative artificial intelligence Kl depending on a speech input SE of a vehicle occupant.

[0079] The speech input SE, which serves as the basis for generating images B4 to B7, is, for example, "Sit quietly, do nothing, spring comes, the grass will grow by itself" or "Sitz ruhig, tue nichts, der Frühling kommt, das Gras wächst von selbst".

[0080] The prompt is specified based on the voice input (SE). This determines whether a person or a seat should be visualized. The prompt can be further customized via voice input (SE), for example, by adding and / or replacing keywords. This can be done, for instance, depending on the vital signs of the vehicle occupant.

[0081] As a result, image B4, for example, shows a person sitting relaxed, surrounded by plants. Image B5 shows a person sitting relaxed in a meadow. Image B6 shows a chair located in a meadow. Image B7 shows a person sitting relaxed in a meadow.

[0082] Figure 3 shows a block diagram of a device 1 for the synchronous visualization of at least one parameter FP1 to FPn of a vehicle and / or at least one parameter FFP1 to FFPm of at least one vehicle function and / or at least one parameter IP1 to IPx of a vehicle occupant.

[0083] The device 1 comprises a detection unit 2 for detecting at least one parameter FP1 to FPn, FFP1 to FFPm, IP1 to IPx, a microphone 3 for detecting a voice input SE of a vehicle occupant, a control unit 4 and at least one display unit 5 which is data-linked to the control unit 4 and visible to the vehicle occupant.

[0084] The control unit 4 is designed to execute the generative artificial intelligence Kl, which combines at least one parameter FP1 to FPn, FFP1 to FFPm, IP1 to IPx and the speech input SE and, depending on the combination, generates content of an image sequence BF that correlates with at least one parameter FP1 to FPn, FFP1 to FFPm, IP1 to IPx and the speech input SE.

[0085] The display unit 5 is configured to synchronize the image sequence BF transmitted by the control unit 4 with a progression of at least one parameter FP1 to FPn.

[0086] Display units 5 can be FFP1 to FFPm and IP1 to IPx. The display unit 5 can be a screen or projection surface located inside the vehicle. Alternatively, the display unit 5 can also be located outside the vehicle and be configured as a screen or projection surface. The display unit 5 can, for example, also be part of infrastructure located in the vicinity of the vehicle or other road users.

[0087] The vehicle's parameters FP1 to FPn include, for example,

[0088] - at least one acceleration value of the vehicle, for example, a positive or negative longitudinal acceleration and / or a lateral acceleration, and / or

[0089] - a vehicle speed and / or

[0090] - the vehicle's driving dynamics and / or

[0091] - a driving program selection that influences the vehicle's driving dynamics and / or noise generation.

[0092] The parameters FFP1 to FFPm of at least one vehicle function include, for example, an activation state and / or movement parameters and / or a position of at least one actuator located inside the vehicle and intended to generate signals perceptible by a vehicle occupant via haptic and / or mechanosensory and / or vibratory and / or kinesthetic means (7.1 to 7.20) and / or

[0093] - an activation sequence of actuators 7.1 to 7.20 arranged in or on a vehicle seat 6 for generating haptic signals and their positions in or on the vehicle seat 6 and / or

[0094] - a sequence of tones from an emitted acoustic signal and / or

[0095] - the volume and / or volume profile of an emitted acoustic signal and / or

[0096] - a sequence of sound signals generated by means of at least one structure-borne sound transducer and / or

[0097] - the strength of at least one sound signal generated by means of at least one structure-borne sound transducer and / or

[0098] - a position and / or an activation state and / or a heat output power of at least one heat source located inside the vehicle and / or

[0099] - a position and / or an activation state and / or a performance of at least one fan located inside the vehicle and / or

[0100] - an activation state and / or temperature control parameter of an air conditioning system intended for climate control of a vehicle interior and / or

[0101] - an activation state and / or ionization parameter of an ionizer designed for the ionization of air within a vehicle interior.

[0102] For example, vehicle functions are recorded as parameters FFP1 to FFPm, including comfort functions such as...

[0103] - an activation status of a number of heating elements,

[0104] - an activation status of a number of massage bubbles located in a vehicle seat (6),

[0105] - an activation status of a number of multi-contour bubbles located in a vehicle seat 6,

[0106] - pressure measurement at a number of positions on a vehicle seat,

[0107] - a status of the air conditioning system and / or

[0108] - a status of the ionizer. The vehicle occupant parameters IP1 to IPx, which indicate reactions of their body to, for example, the vehicle functions and / or vehicle parameters FP1 to FPn, include, for example...

[0109] - Vital data of the vehicle occupant and / or

[0110] - Position data of body parts of the vehicle occupant and / or

[0111] - Movement data of body parts of the vehicle occupant

[0112] The body's reactions manifest themselves, for example, in the form of emotions and physiological responses, i.e., vital signs. Vital signs include, for example, respiratory rate, heart rate, blood pressure, body temperature, and / or the stress level of the vehicle occupant. Furthermore, the body's reactions are reflected in positional and movement data of body parts. For example, the three-dimensional position of the head in space, movement in space, or movement relative to the head position of defined body points, such as the nose, left shoulder, right shoulder, etc., are recorded.

[0113] Additional sources of text that can influence the visual content generated by generative artificial intelligence (AI), such as stable diffusion, include, for example, the following:

[0114] For example, a vehicle occupant can specify a desired location using voice input (SE). The voice input could be something like "I'd like to be at the beach" or include vacation memories such as "mountains and water".

[0115] SE can also use voice input to suggest sayings and / or stories, such as "Sit quietly, do nothing, spring comes, the grass will grow by itself" or "Sitz ruhig, tue nichts, der Frühling kommt, das Gras wächst von selbst".

[0116] Furthermore, a sketch or a real photograph, for example of a body posture, which is then converted into a photorealistic image B1 to B7 or an animation, can be used as the basis for the model of the generative artificial intelligence Kl. In a method carried out using the device 1 for the synchronous visualization of at least one parameter FP1 to FPn of the vehicle and / or at least one parameter FFP1 to FFPm of at least one vehicle function and / or at least one parameter IP1 to IPx of a vehicle occupant, the at least one parameter FP1 to FPn, FFP1 to FFPm, IP1 to IPx is first acquired by means of the acquisition unit 2 and supplied to the control unit 4. Furthermore, the voice input SE of the vehicle occupant is acquired by means of the microphone 3 and also supplied to the control unit 4.

[0117] Using control unit 4, the generative artificial intelligence Kl combines at least one parameter (FP1 to FPn, FFP1 to FFPm, IP1 to IPx) and the voice input SE. Depending on the combination, the generative artificial intelligence Kl generates the content of an image sequence (BF) that correlates with the at least one parameter (FP1 to FPn, FFP1 to FFPm, IP1 to IPx) and the voice input SE. Control unit 4 can be, for example, a central control unit in the vehicle or in a backend system. At least parts of control unit 4 can also be implemented in both the vehicle and the backend. For example, the generative artificial intelligence Kl is located in a cloud and can be addressed "over the air" by the vehicle via an API interface.Alternatively or additionally, a simplified generative artificial intelligence (AI) is present in the vehicle, while more complex requests are processed by the generative artificial intelligence (AI) located in the cloud, and results are transferred to the vehicle at defined time intervals.

[0118] The image sequence BF is then transmitted by the control unit 4 to at least one display unit 5 visible to the vehicle occupant. This display unit outputs the image sequence BF synchronously with the progression of at least one parameter FP1 to FPn, FFP1 to FFPm, IP1 to IPx. The display unit 5 is designed as a screen or projection surface, for example, a screen, windscreen display, etc. Projection and display of the image sequence BF on any conceivable surface inside or outside the vehicle, or by transferring image information via wireless communication (e.g., radio or other technologies) to infrastructure in the vehicle's vicinity or to other road users, is possible. For example, the image sequence BF can be displayed in the vicinity of a charging station where the vehicle's electrical energy storage device is being charged.

[0119] In one possible embodiment, parameters IP1 to IPx are used to record the breathing rate of the vehicle occupant, which represents ten inhalations and exhalations per minute.

[0120] The breathing rate is broken down into, for example, an inhalation and an exhalation phase. A text module is then assigned to the breathing rate itself, the inhalation phase, and the exhalation phase, or possibly even more finely granularly. This text module defines what is displayed on the display unit 5, for example, a central display in the vehicle, and also which movement is visualized in which direction. This content and movement can optionally be extended to other technologies and implemented there.

[0121] For example, a signal indicating the activation of a vehicle function, and thus an effect on the vehicle occupants, is transmitted to control unit 4 and converted there into a visual signal and a visual effect, which are output via the image sequence BF. Visual effects can be displayed on screen surfaces, interior surfaces, side windows, windshields, rear windows, and on the vehicle roof. This can be achieved, for example, through appropriately positioned screens and / or a projection. This advantageously allows for the continuous creation of new, individualized, interactive, and innovative visual effects.

[0122] This means that, using device 1, an adaptive and interactive method for the generative generation of visual effects based on generative artificial intelligence (AI), in particular using diffusion models, can be executed. The synchronous output of the image sequence BF allows the occupant to experience synchronized perception of the respective parameters FP1 to FPn, FFP1 to FFPm, and IP1 to IPx across all sensory channels.

[0123] Figure 5 shows a block diagram of a possible embodiment of the device 1 according to Figure 4 for the synchronous visualization of parameters FFP1 to FFPm of a vehicle function. In the illustrated embodiment, the parameters FFP1 to FFPm relate to an activation sequence of actuators 7.1 to 7.20 arranged in a vehicle seat 6 for generating haptic signals and their positions in or on the vehicle seat 6. The actuators 7.1 to 7.20 are designed to generate a massage function for a vehicle occupant located in the vehicle seat 6 and are configured as so-called massage bubbles or other actuators 7.1 to 7.20 designed to exert mechanical pressure on the vehicle occupant. Actuators 7.1 to 7.14 are arranged in a backrest 6.1 of the vehicle seat 6, and actuators 7.15 to 7.20 are arranged in a seat cushion 6.2 of the vehicle seat 6. Furthermore, vibration motors can also be arranged in the vehicle seat 6.

[0124] The massage function, performed by actuators 7.1 to 7.20, is achieved through a directed and timed activation of these actuators. As indicated by the left arrow, actuators 7.1 to 7.5 in the seat backrest 6.1 are activated sequentially, followed by actuators 7.15 and 7.16 in the seat cushion 6.2. Simultaneously, as indicated by the right arrow, actuators 7.10 to 7.14 in the seat backrest 6.1 are activated sequentially, followed by actuators 7.19 and 7.20 in the seat cushion 6.2. The activation of the aforementioned actuators 7.1 to 7.5, 7.15, and 7.16, as well as actuators 7.10 to 7.14, 7.19, and 7.20, occurs, for example, within ten seconds. After six seconds, two vibration motors can also be activated.

[0125] The aforementioned activation sequence and positions of actuators 7.1 to 7.5, 7.15, 7.16, actuators 7.10 to 7.14, 7.19, 7.20 and vibration motors, parameters FFP1 to FFPm, are transmitted from the detection unit 2, for example a seat control unit, to the control unit 4.

[0126] Furthermore, the vehicle occupant provides user-specific input via voice command SE, which can include, for example, a description of a destination, a desired atmosphere, and / or an emotion. For example, the vehicle occupant might provide the voice command SE "Moving water and nature." This voice command SE is captured by microphone 3 and also transmitted to control unit 4. Using the generative artificial intelligence Kl implemented in control unit 4, the input data is processed, for example, via stable diffusion.

[0127] The parameters FFP1 to FFPm and the speech input SE are processed in such a way that they are combined and, depending on the combination, the generative artificial intelligence Kl generates the contents of an image sequence BF that correlates with the parameters FFP1 to FFPm and the speech input SE.

[0128] The image sequence BF is then output via display unit 5 and shows a first waterfall WF1. The direction of movement of the water of this waterfall WF1 and the duration of this movement are defined by the activation direction of actuators 7.1 to 7.5, 7.15, 7.16 and actuators 7.10 to 7.14, 7.19, 7.20 from top to bottom within ten seconds.

[0129] Synchronously with the activation of a vibration motor in the rear area of ​​the seat cushion 6.2 and, for example, one second later with the activation of a vibration motor in the front area of ​​the seat cushion 6.2, a second waterfall WF2 is shown in the image sequence BF.

[0130] This synchronous visual display allows the effects that are haptically perceptible to the vehicle occupant via actuators 7.1 to 7.5, 7.15, 7.16, actuators 7.10 to 7.14, 7.19, 7.20, and the vibration motors to be visually represented for the vehicle occupant. The visual effects displayed change depending on which actuator 7.1 to 7.20 and / or vibration motor and / or other vehicle components are currently activated.

Claims

Mercedes-Benz Group AG Patent claims 1. Method for the synchronous visualization of at least one Parameters (FP1 to FPn) of a vehicle and / or at least one parameter of at least one vehicle function (FFP1 to FFPm) and / or at least one parameter (IP1 to IPx) of a vehicle occupant, wherein - which at least one parameter (FP1 to FPn, FFP1 to FFPm, IP1 to IPx) and a voice input (SE) of the vehicle occupant are supplied to a control unit (4), - based on the control unit (4) by means of a generative artificial intelligence (Kl) which combines at least one parameter (FP1 to FPn, FFP1 to FFPm, IP1 to IPx) and the speech input (SE), - depending on the combination, the generative artificial intelligence (Kl) generates content of an image sequence (BF) that correlates with at least one parameter (FP1 to FPn, FFP1 to FFPm, IP1 to IPx) and the speech input (SE), - the image sequence (BF) is displayed on at least one display unit (5) visible to the vehicle occupant synchronously to a progression of at least one parameter (FP1 to FPn, FFP1 to FFPm, IP1 to IPx). - at least a diffusion model is used by the generative artificial intelligence (AI) to generate the content of the image sequence (ES).

2. Method according to claim 1, characterized in that the parameters (FP1 to FPn) of the vehicle are - at least one acceleration value of the vehicle and / or - a vehicle speed and / or - the vehicle's driving dynamics and / or - a driving program selection that influences the vehicle's driving dynamics and / or noise generation is used / will be used.

3. Method according to claim 1 or 2, characterized in that the parameters of the at least one vehicle function (FFP1 to FFPm) are - an activation state and / or movement parameter and / or a position of at least one actuator (7.1 to 7.20) arranged within the vehicle and designed to generate signals perceptible by a vehicle occupant haptically and / or mechanosensorily and / or vibratorily and / or kinesthetically and / or - an activation sequence of actuators (7.1 to 7.20) arranged in or on a vehicle seat (6) for generating haptic signals and their positions in or on the vehicle seat (6) and / or - a sequence of tones from an emitted acoustic signal and / or - the volume and / or volume profile of an emitted acoustic signal and / or - a sequence of sound signals generated by means of at least one structure-borne sound transducer and / or - the strength of at least one sound signal generated by means of at least one structure-borne sound transducer and / or - a position and / or an activation state and / or a heat output power of at least one heat source located inside the vehicle and / or - a position and / or an activation state and / or a performance of at least one fan located inside the vehicle and / or - an activation state and / or temperature parameter of an air conditioning system intended for climate control of a vehicle interior and / or - an activation state and / or ionization parameter of an ionizer intended for the ionization of air within a vehicle interior is used / will be used.

4. Method according to one of the preceding claims, characterized in that as parameters of the vehicle occupant (IP1 to IPx) - Vital data of the vehicle occupant and / or - Position data of body parts of the vehicle occupant and / or - Movement data from body parts of the vehicle occupant is used.

5. Method according to one of the preceding claims, characterized in that, by means of generative artificial intelligence (Kl), emotions of the vehicle occupant are recognized based on the speech input (SE) and the contents of the image sequence (BF) are adapted to the emotions of the vehicle occupant.

6. Method according to one of the preceding claims, characterized in that emotions of the vehicle occupant are recognized from at least one parameter (IP1 to IPx) of the vehicle occupant and the contents of the image sequence (BF) are adapted to the emotions of the vehicle occupant.

7. Method according to one of the preceding claims, characterized in that - generative artificial intelligence (AI) for generating the content of the Image sequence (BF) from which at least one Parameters (FP1 to FPn, FFP1 to FFPm, IP1 to IPx) and speech input (SE) generate a text and convert the text into an image (B1 to B7) of the image sequence (BF) and / or - generative artificial intelligence (AI) for generating the content of the Image sequence (BF) from which at least one Parameters (FP1 to FPn, FFP1 to FFPm, IP1 to IPx) and the speech input (SE) generate a prompt, convert the prompt into text and convert the text into an image (B1 to B7) of the image sequence (BF).

8. Device (1) for synchronous visualization of at least one Parameters (FP1 to FPn) of a vehicle and / or at least one Parameters (FFP1 to FFPm) of at least one vehicle function and / or at least one parameter (IP1 to IPx) of a vehicle occupant, comprising - at least one acquisition unit (2) for acquiring at least one parameter (FP1 to FPn, FFP1 to FFPm, IP1 to IPx), - at least one microphone (3) for capturing a voice input (SE) from a vehicle occupant, - a control unit (4) which is designed to execute a generative artificial intelligence (Kl) which combines at least one parameter (FP1 to FPn, FFP1 to FFPm, IP1 to IPx) and speech input (SE) and, depending on the combination, generates content from a system with at least one parameter (FP1 to FPn, FFP1 to FFPm, IP1 to IPx) The image sequence (BF) is generated using at least one diffusion model and parameters (FP1 to FPn, FFP1 to FFPm, IP1 to IPx) and the speech input (SE). - at least one display unit (5) that is data-linked to the control unit (4) and visible to the vehicle occupant, which is designed to output the image sequence (BF) transmitted by the control unit (4) synchronously to a progression of at least one parameter (FP1 to FPn, FFP1 to FFPm, IP1 to IPx).

Images