Entertainment system and method

Abstract

An entertainment system 1 for a vehicle 10 comprises an apparatus configured to analyse sound from an audio source, render visual effects 12 based on features of the sound, and to display the visual e

Description

FIELD OF THE INVENTION

[0001] The invention relates to an entertainment system according to Claim 1. Furthermore, the present invention relates to a corresponding method, a corresponding computer program product, and a corresponding non-transitory computer-readable storage medium. BACKGROUND INFORMATION

[0002] In the state of the art, entertainment systems and / or infotainment systems for vehicles are known, offering only static displays or simple visualizations on screens inside the vehicle. These systems are often limited to playing videos or displaying light effects without considering the interaction between music and visual representation.

[0003] With the automation of many or most driving tasks through autonomous driving, additional entertainment possibilities arise during the ride, previously considered distracting for the driver.

[0004] The objective of the present invention is to provide an entertainment system that dynamically adapts visual effects to music features, offering an immersive experience without compromising safety or driving comfort.

[0005] This objective is achieved through an entertainment system with the features of Patent Claim 1 and by the inventive method. Advantageous designs of the inventive entertainment system are considered advantageous embodiments of the inventive method, wherein the means of the entertainment system are used to execute the method steps. Further advantageous developments of the invention are described in the dependent patent claims, the following description, and the figures. SUMMARY OF THE INVENTION

[0006] A first aspect of the invention relates to an entertainment system for a vehicle, in particular for a passenger vehicle, especially for an at least partially autonomous motor vehicle, comprising an electronic computing device for analyzing sound from a source. The source can be vehicle-internal or vehicle-external, such as data sources, streams, or databases. For example, music may also be transmitted to the car via a third device. The computing device is configured to capture the sound from the source and create visual effects based on sound features such as beats, frequencies, and / or patterns using a rendering system. These visual effects are transmitted to a device, e.g. display device designed to present the visual effects synchronized with the played sound. Preferably, the display device is provided as smart glasses. The electronic computing device may be part of the display device, a component of the smart glasses, or external, e.g., mounted on the vehicle, or alternatively on a third device, such as a mobile device like a smartphone connected to the smart glasses and / or to the vehicle.

[0007] The electronic computing device continuously analyzes the features of the played music, such as beats per minute, spectral properties, and / or recurring patterns, processing this data to create precise visual effects. The rendering system generates the effects in real time, utilizing a predefined library or dynamically created content. The display device projects the effects onto screens or through smart glasses, creating the impression that the effects appear in the vehicle's surroundings. An advantage lies in the seamless integration of audio and visual experiences, entertaining users without diverting their attention from the surroundings.

[0008] Users perceive these effects as "synchronized to the music," comparable to a beat indicator and / or spectrum analyzer on a conventional stereo system. Similar entertainment systems are used in dance clubs, where displays or light / laser systems provide visual effects. This invention transfers this concept to vehicle applications, particularly in at least partially autonomous vehicles.

[0009] The visualization processing may be carried out by the entertainment system, either integrated with the vehicle's infotainment system or acting as such. Alternatively, an app on the smart glasses, their computing unit (e.g., a connected smartphone), or a combination of both may be utilized. Users may use this setup to switch visual effects on and off, adjust them, and make music changes.

[0010] Therefore, components of the entertainment system may include a rendering system for creating visual effects, a library of effects that may be parameterized and / or combined, and a music analysis unit, such as an algorithm for frequency analysis using e.g. Fourier transformation and / or beat detection.

[0011] Unlike visualizations on vehicle displays with limited size or ambient light installations with limited precision, this solution combines an audio and visual experience directly in front of the driver or passenger. By overlaying visual information onto the road ahead or within a 360-degree field of view, users of such an entertainment system may utilize the windshield or the entire vehicle interior to enjoy the experience without focusing on a screen or relying on ambient lighting.

[0012] In other words, smart glasses, i.e., glasses with integrated displays or visual projection systems, are proposed to enhance the vehicle user's view—such as the driver and / or passenger—by dynamically superimposing generated visualizations. These visualizations are synchronized with the music being played and the current driving context and may be adjusted accordingly. Users may thus look through the vehicle's front windshield while being entertained by these visual additions to the music. It is also possible to display visual additions on side windows, other windows like roof windows, or within the interior, for example, to create a 3D sensation. Additionally, videos on an interior display may be enhanced with visual additions for an immersive experience, e.g., a 360-degree visualization involving the entire environment.

[0013] The visualizations displayed in the vehicle user's view using smart glasses take into account the current music and driving context. For music, the entertainment system analyzes the currently played music for features such as beats per minute, notes / frequencies (chromagram, spectral analysis), recurring patterns, and song lyrics using speech recognition, text analysis, or external sources like vehicle-external data sources. The entertainment system selects visual effects based on a predefined library, downloads new ones, adjusts them, or dynamically creates them according to specifications and / or environmental conditions. For example, colors in the vehicle's environment may be detected to use contrasting colors for the visual addition, or interior colors and / or lighting may be considered, similar to a backlit TV.

[0014] The entertainment system may also take metadata, such as track information (e.g., genre or artist), into account to select or combine effects.

[0015] In an advantageous embodiment of the invention, the visual effects are designed to appear as if displayed in the immediate environment of the vehicle. The projection takes the vehicle's position and movement into account, enabling an immersive experience for users. It is particularly preferred that the visual effects are displayed in an immediate area, preferably ahead of the vehicle in the direction of travel. Alternatively, they may also be displayed 360 degrees around the vehicle. In any case, they should be shown on the road or an area above it, providing both visual and auditory entertainment. In another embodiment, tactile signals, adjustable to bass sounds, may be integrated.

[0016] In another advantageous embodiment of the invention, the actual vehicle surroundings are captured by sensors to adapt the effects accordingly. This integration enhances the safety measures of the visual effects. For example, weather conditions such as rain and / or darkness may be detected, with the visual effects adapted accordingly by using complementary colors or adjusting to brightness conditions. Reflections on the road may also be considered, as well as tunnel driving. For instance, a larger surrounding area may be utilized for the visual effects on open roads, whereas a smaller surrounding area may be used in traffic or at lower speeds.

[0017] In another advantageous embodiment of the invention, surrounding traffic is detected and considered when creating the effects to provide a realistic and safety-oriented visual experience. For the driver's safety in hazardous situations—such as in urban traffic, approaching vehicles, traffic jams, or highway driving—the entertainment system adapts the visual effects. Additionally, vehicle sensors like LIDAR, optical detection devices (e.g., cameras), and external sources such as navigation systems or V2V systems use data to detect hazards early and control the visual effects accordingly. This allows visual effects to be projected onto the road surface without visually cutting through vehicles ahead. Lane markings detected by the cameras may be used to represent the road course or limit the effects to the vehicle's lane. For instance, at night, the driving lanes may be highlighted using visual effects. Steering angles, vehicle acceleration, and braking may also be used to parameterize the effects.

[0018] In another advantageous embodiment of the invention, vehicle sensor data such as speed, steering angle, and braking operations are used to dynamically adapt the visual effects to the driving context. This enables synchronization between visual effects and the driving situation. For example, sensor data may be used to limit the visual effects to a predefined area for safety in the surroundings.

[0019] In another advantageous embodiment of the invention, navigation instructions are integrated into the visual effects to provide users with additional information such as turn instructions or lane changes. This improves the system's functionality, for instance, navigation instructions may be integrated into the visual effects, such as during a lane change for an exit. Visual effects may be used for navigation, for example, through turn-by-turn visualizations. Technically, this is based on the vehicle's existing navigation system, with directional instructions serving as input for the visual effects.

[0020] In other words, the inventive entertainment system offers a combination of audio analysis, rendering, and visual display synchronized with the played music. The entertainment system uses advanced sensor data processing to create visual effects in real time, dynamically adapting them to the environment and driving situation.

[0021] In other words, the system provides an immersive entertainment solution integrating music, visual effects, and driving data. This allows the vehicle environment to be perceived as part of the entertainment experience without compromising user safety or comfort.

[0022] Another aspect of the invention relates to a method for operating an entertainment system for a vehicle. This method describes how the system interactively and dynamically generates and displays visual effects based on the characteristics of the sound being played by the user. The method involves several steps: First, the sound is captured from a source, which may be either vehicle-internal, such as an infotainment system, or vehicle-external, such as a smartphone or streaming service. Next, an electronic computing device analyzes the captured sound to identify features like beats, frequencies, or other audio patterns. Based on these analyzed features, a rendering system dynamically generates matching visual effects. These effects are transmitted to a display device, preferred to smart glasses, which presents them in synchronization with the music being played, creating an immersive, sound-aligned visual experience for the user. This method enables an integration of music and visual representation by generating effects in real time and adapting them to the specific sound characteristics. The synchronized display ensures that the effects not only accompany the music but also enhance it, thereby improving the in-vehicle entertainment experience.

[0023] Another aspect of the invention relates to a computer program product containing program code for executing the method to run the communication system.

[0024] Additionally, the invention includes a non-transitory computer-readable storage medium that securely stores this program.

[0025] Further advantages, features, and details of the invention derive from the following description of preferred embodiment as well as from the drawing. The features and feature combinations previously mentioned in the description as well as the features and feature combinations mentioned in the following description of the figure and / or shown in the figure alone can be employed not only in the respectively indicated combination but also in any other combination or taken alone without leaving the scope of the invention. BRIEF DESCRIPTION OF THE DRAWING

[0026] The novel features and characteristic of the disclosure are set forth in the appended claims. The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and together with the description, serve to explain the disclosed principles. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the figures to reference like features and components. Some embodiments of system and / or methods in accordance with embodiments of the present subject matter are now described below, by way of example only, and with reference to the accompanying figures.

[0027] The drawing shows in:

[0028] Fig 1 a vehicle as seen from the perspective of a user wearing smart glasses to show visual effects on the road.

[0029] In the figure the same elements or elements having the same function are indicated by the same reference signs. DETAILED DESCRIPTION

[0030] In the present document, the word "exemplary" is used herein to mean "serving as an example, instance, or illustration". Any embodiment or implementation of the present subject matter described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

[0031] While the disclosure is susceptible to various modifications and alternative forms, specific embodiments thereof have been shown by way of example in the drawing and will be described in detail below. It should be understood, however, that it is not intended to limit the disclosure to the particular forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternatives falling within the scope of the disclosure.

[0032] The terms “comprises”, “comprising”, or any other variations thereof, are intended to cover a non-exclusive inclusion so that a setup, device or method that comprises a list of components or steps does not include only those components or steps but may include other components or steps not expressly listed or inherent to such setup or device or method. In other words, one or more elements in a system or apparatus preceded by “comprises” or “comprise” does not or do not, without more constraints, preclude the existence of other elements or additional elements in the system or method.

[0033] In the following detailed description of the embodiment of the disclosure, reference is made to the accompanying drawing that forms part hereof, and in which is shown by way of illustration a specific embodiment in which the disclosure may be practiced. This embodiment is described in sufficient detail to enable those skilled in the art to practice the disclosure, and it is to be understood that other embodiments may be utilized and that changes may be made without departing from the scope of the present disclosure. The following description is, therefore, not to be taken in a limiting sense.

[0034] Fig. 1 shows a vehicle 10 as seen from the perspective of a user wearing smart glasses. Through the windshield 11, the road 14 is visible, enhanced by visual effects 12 dynamically projected / superimposed onto the user’s view. These visual effects 12 are synchronized with the music being played and adapt to the driving context and environment of the vehicle, offering an immersive experience while ensuring that the road 14 remains visible for safe driving.

[0035] The visual effects 12 are generated by an entertainment system 1 integrated into the vehicle 10. This entertainment system 1 includes an electronic computing device that analyzes sound features such as beats, frequencies, and patterns from an audio source, which may be vehicle-internal, like the infotainment system, or vehicle-external, such as a smartphone or cloud-based music service. Using this data, the computing device processes and creates real-time visual effects through a rendering system. These effects are projected onto the user’s view via the smart glasses, which may use augmented reality technology to overlay the effects onto the road 14 and the vehicle's surroundings. This setup makes the visual effects appear as though they are floating within the user’s immediate environment, enhancing the entertainment experience without obstructing critical views.

[0036] Safety is a core consideration in the design of the entertainment system 1. First, the visual effects 12 are contextually adapted based on real-time driving data, such as speed, steering angle, and braking operations. For instance, during high-speed travel or complex traffic situations, the entertainment systems 1 may reduce the intensity of the effects or limits their projection to peripheral areas, ensuring that the driver's focus remains on the road 14. Second, sensors integrated into the vehicle, such as cameras and LIDAR, monitor environmental conditions, allowing the system to adapt the brightness and contrast of the effects in poor visibility situations like rain, fog, or darkness. For example, complementary colors may be used to improve visibility without creating glare.

[0037] The entertainment system 1 also integrates data from surrounding traffic 18. It detects nearby vehicles, pedestrians, and lane markings, ensuring that the visual effects 12 do not obscure critical road elements. For example, lane boundaries 14a may be highlighted, or upcoming turns may be indicated without interfering with the perception of potential hazards. Navigation assistance is another important feature, with the entertainment system 1 incorporating directional cues, such as arrows, into the visual effects to guide the driver safely through lane changes or exits. These cues are directly displayed on the road 14, blending entertainment with practical functionality.

[0038] Finally, a projection area 16 of the visual effects 12 may be dynamically adjusted based on the driving environment. On open roads 14, a broader projection area can be used to enhance the experience, while in urban settings or dense traffic 18, the effects are confined to a smaller, less intrusive area. This ensures that the entertainment system 1 remains functional and non-distracting in diverse driving conditions.

[0039] Overall, the entertainment system 1 combines immersive visual effects 12 with advanced safety features. By leveraging augmented reality and real-time sensor data, it provides a seamless integration of entertainment and driving, enhancing the user experience while maintaining focus on safety-critical elements of the road 14. Reference sign's list 1 entertainment system 10 vehicle 12 visual effect 14 road 14a lane boundaries 16 projection area 18 traffic

Claims

1. Entertainment system (1) for a vehicle (10), comprising an electronic computing device for analyzing sound from a source, characterized in thatthe electronic computing device is configured to capture the sound from the source, create visual effects (12) based on sound features of the sound using a rendering system, and transmit them to a display device for presentation to a user, wherein the display device is configured to display the visual effects (12) synchronized with the sound played by a sound system for the user.

2. Entertainment system (1) according to Claim 1, characterized in thatthe visual effects (12) are generated in such a way that they create the impression of being displayed in the immediate surroundings of the vehicle (10).

3. Entertainment system (1) according to Claim 1 or 2, characterized in thatthe actual surroundings of the vehicle (10) are detected and taken into account when generating the visual effects (12).

4. Entertainment system (1) according to Claim 2 or 3, characterized in thattraffic in the actual surroundings of the vehicle (10) is detectable and incorporated into the presentation of the visual effects.

5. Entertainment system (1) according to any of the preceding claims, characterized in thatvehicle sensor data are used to dynamically adapt the visual effects (12).

6. Entertainment system (1) according to any of the preceding claims, characterized in thatnavigation instructions are integrated into the visual effects (12).

7. Entertainment system (1) according to any of the preceding claims, characterized in thatthe display device is embodied as smart glasses.

8. Method for operating an entertainment system (1) for a vehicle (10), the entertainment system (1) comprising an electronic computing device for analyzing sound from a source, characterized by the steps of:- Capturing the sound from the source using the electronic computing device;- Analyzing the sound to identify sound features;- Generating visual effects (12) based on the identified sound features using a rendering system;- Transmitting the visual effects (12) to a display device; and- Displaying the visual effects (12) on the display device, synchronized with the sound played by a sound system for the user.

9. A computer program product comprising program code means for performing a method according to claim 8.

10. A non-transitory computer-readable storage medium comprising at least the computer program product according to claim 9.IntellectualPropertyOfficeApplication GB2501623.9Search report under Section 17 of the Patents Act 1977Date search completed: 16 July 2025Claims searched: 1-10International classificationSubclass and subgroup Valid from B60K35 / 23 01 / 01 / 2024 B60K35 / 28 01 / 01 / 2024 B60R1 / 00 01 / 01 / 2022 G02B27 / 01 01 / 01 / 2006Field of searchWorldwide search of patent documents classified in the following areas of the IPC:B60K, B60R, G02B, H04SDatabases used in the preparation of this search report:SEARCH-PATENTDocuments considered to be relevantPatent literatureRelevant Document of relevanceCategory claimsX 1-10 GB 2616648 A (WAYRAY AG), see particularly figures 4-9, pages 11-20 disclosing augmented reality system for a vehicle HUD X 1-10 US 2019 / 0369391 A1 (CORDESSES et al.), See particularly paragraphs 0030-0075 disclosing features of augmented reality apparatus displaying images relating to currently playing music X 1-6, 8-10 WO 2023 / 202384 A1 (GREAT WALL MOTOR CO LTD), See figures 5, 6, detailed description of invention disclosing augmented reality device with music mode for displaying song information to driver A - US 2021 / 0166659 A1 (TAKATA et al.), See entire document disclosing HUD with ambient sound detection and alert display means A - CN 112185415 A (GREE ELECTRIC APPLIANCES INC ZHUHAI), See entire document disclosing system to analyse sounds and generate visual representations thereofCategoriesLetter or symbol Description X Document indicating lack of novelty or inventive step. Y Document indicating lack of inventive step, if combined with another document of the same category. & Member of the same patent family. A Document indicating technological background. P Document published on or after the priority date but before the fling date of the present application. E Earlier application published on or after the filing date of the present application.

Images