Method for generating display content for a display device
By automatically creating display content through generative machine learning models, the problem of inflexible adaptation of display content on vehicle display devices has been solved, enabling personalized and real-time generation of display content and improving user experience and display effects.
Patent Information
- Application Number
- CN202580028481.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-07-08
- Filing Date
- 2025-05-14
- Publication Date
- 2026-11-17
AI Technical Summary
In the prior art, the display content of vehicle display devices cannot be flexibly adapted to displays of different shapes and sizes, resulting in truncated or distorted edges. Furthermore, updates are costly and not real-time, and the content cannot be displayed synchronously according to driver preferences and driving conditions.
Using a generative machine learning model, display content is automatically created based on vehicle parameters, including shape factor, content parameters, driving parameters, and user parameters, to generate display content adapted to the display device.
It enables flexible adaptation of displayed content, avoids edge truncation or distortion, provides a personalized user experience, improves the real-time performance and adaptability of displayed content, and reduces update costs.
Smart Images

Figure CN123125576A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for generating display content for display devices, particularly in vehicles. In particular, the display content should be flexibly generated based on original content for use with a variety of different display devices. Background Technology
[0002] Most vehicles are standardly equipped with at least one display device, such as a monitor, to display various content, such as information about the vehicle or its functions, navigation information, and similar content, but increasingly also for passenger entertainment. This could be, for example, a central display on the dashboard, one or more displays for rear passengers, or a display set up for the front passenger, the central display being visible to both the driver and the front passenger.
[0003] Such displays are often designed differently across different vehicle models or product lines; that is, the displays can have different sizes and geometries, and can also have different resolutions. For example, in addition to purely rectangular displays, there are increasingly more displays with other shape factors, such as circular displays, or even displays with other non-rectangular geometries. The aforementioned in-vehicle displays are also typically different. In order to fully display the content, especially the background, this content must be specifically created for each type of display. For example, display content created for a rectangular display of a certain size cannot be easily displayed on a smaller or larger, or even circular, display, because the edges will be truncated, black borders will appear, or the content will be distorted.
[0004] Therefore, the creation of display content has been limited to date. While the content can all be based on the same original material, it is adapted to different display types. This is costly. Furthermore, such content is typically created only once, for example, before vehicle delivery, and is therefore fixed to a specific vehicle and display type and cannot be changed. With such a static set of display content, there is no flexibility to adapt to current or even external application content, no synchronization with new or personal content, and no consideration of driver preferences, driving conditions, or similar situations. Although display content can be updated via online software updates when necessary, this is costly and not real-time, ultimately only providing a static set of display content again. Summary of the Invention
[0005] The objective of this invention is to generate display content for display devices, particularly in vehicles, that can be flexibly adapted to the display device.
[0006] The solution to this task is achieved based on the teachings of the independent claims. Different embodiments and modifications of the invention are the subject of the dependent claims.
[0007] A first aspect of the invention relates to a method, particularly computer-implemented, for generating display content for a display device in a vehicle. In this method, original content is provided as the basis for display content to be created for the display device, and at least one vehicle parameter is detected. Display content for the display device is created from the original content, wherein the creation is performed with the aid of a generative machine learning model, taking into account the at least one vehicle parameter, such that the resulting display content represents the original content and is adapted to the at least one vehicle parameter. The resulting display content can then be displayed on the display device.
[0008] Therefore, the method described above according to the first aspect is particularly based on: considering at least one vehicle parameter when creating display content in order to adapt the original content accordingly. Generative machine learning models are used for this purpose. This allows for the flexible generation of suitable display content for the display device from any original content, such as current multimedia content or external content, without the need for pre-made display content.
[0009] In general, the approach based on the first aspect is grounded in the fundamental idea of providing automated creation, such as the automated creation of background images. By using generative artificial intelligence, flexible and automated generation can be achieved without prior user or developer intervention. As will be described in more detail below, by considering one or more vehicle parameters, the approach based on the first aspect opens up the possibility of adapting the displayed content to the current given conditions in any situation, incorporating changes, and thereby also achieving an improved user experience.
[0010] The concept of "vehicle" used here specifically refers to passenger vehicles, including all types of motor vehicles, hybrid and battery-powered electric vehicles, and vehicles such as sedans, vans, buses, trucks, and delivery vehicles.
[0011] As used herein, the concept of "display device" should be understood in particular as a device for displaying images or videos, i.e., content that a user can visually perceive. This may include monitors, screens, and similar devices. In particular, display devices can be located in vehicles. Other terminal devices, such as mobile terminal devices, such as smartphone displays, and similar devices may also be referred to as "display devices."
[0012] As used herein, the concept of "shape factor" should be understood in particular as the physical size, shape, and general dimensions of a display device. This can include various aspects such as the aspect ratio and diagonal of the display. "Shape factor" is also referred to herein as the "geometry" of the display device. Furthermore, a display device can be characterized by its resolution or number of pixels.
[0013] As used herein, the concept of "display content" should be understood in particular as content that can be visually displayed on a display device so that it can be visually perceived by a user. This can be images, graphics, animations, and the like. Display content can be static or dynamic, and in particular, it can be video or a video sequence.
[0014] As used herein, the concept of “original content” should be understood in particular as data that may not be directly displayed as “display content” or should not be directly displayed as “display content” (even though this is possible in principle). More precisely, the original content serves as a basis for generating, in particular by means of generative artificial intelligence and taking into account other parameters, the actual display content, which can then be displayed on a display device.
[0015] The concept of "vehicle parameters" as used herein specifically refers to parameters relating to the vehicle itself and its operation, including the users within the vehicle. These can be physical parameters of the vehicle, such as the dimensions and layout of the vehicle or its components, including one or more display devices within the vehicle. However, vehicle parameters can also include (driving) parameters (that can be measured, if necessary, by means of appropriate sensor mechanisms), such as vehicle speed, driving conditions, ADAS level (the degree of autonomous vehicle control), etc. User parameters can be detected as vehicle parameters, for example, via a driver monitoring system, which may have at least one camera within the vehicle's interior space. This allows for the detection of user, particularly driver, characteristics such as emotional or mood state, especially attention state, such as alertness or fatigue, direction of observation, identity, and similar characteristics.
[0016] The concept of "generative artificial intelligence" or "generative machine learning model" used herein specifically refers to an artificial intelligence (i.e., a machine learning method) capable of generating text, images, videos, or other data using a generative model. Consider input data, such as raw content (images, videos), and at least one vehicle parameter. The generative AI model learns the patterns and structure of the input training data and then generates new data with similar characteristics. In the current context, generative artificial intelligence is used to generate display content for a display device.
[0017] The terms “comprising,” “including,” “including,” “having,” “having,” “with,” or any other variation thereof, used herein where necessary, shall cover non-exclusive inclusion. Thus, a method or apparatus that includes or has a list of elements, for example, is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to the method or apparatus.
[0018] Furthermore, unless explicitly stated otherwise, "or" refers to an inclusive "or" rather than an exclusive "or". For example, condition A or B is satisfied by one of the following conditions: A is true (or exists) and B is false (or does not exist), A is false (or does not exist) and B is true (or exists), and both A and B are true (or exist).
[0019] As used herein, the term "an" or "a kind" is defined as "one or more". The terms "another" and "another" and any other variations thereof shall be understood as "at least one other".
[0020] As it is used here, “multiple” should be understood as “two or more”.
[0021] The terms "configuration" or "design" used to fulfill a specific function (and its variations) should be understood in the context of this invention as meaning that the corresponding device already exists in a design or configuration in which the device can perform the function, or that the device is at least configurable, i.e., settable, so that the device can perform the function after a corresponding setting. This configuration can be performed, for example, by setting parameters of a process flow or switch, to activate or deactivate the function or setting. In particular, the device can have multiple predetermined configurations or operating modes, such that the configuration can be performed by selecting one of these configurations or operating modes.
[0022] The preferred embodiments of the method are described below. Unless explicitly excluded or technically impossible, these preferred embodiments can be combined with each other and with other aspects of the invention as described herein.
[0023] In some embodiments, at least one shape factor of the display device is detected as a vehicle parameter, the shape factor defining the size and / or shape of the display device, and the original content is display content having a format unsuitable for the shape factor of the display device. The display content is then created such that the format of the created display content matches the shape factor of the display device. In this way, any original content can be adapted to the display device, especially enabling the display content to be fully displayed on the display device, i.e., without truncating edges or supplementing with black (unused) edges, or even displaying the content in a distorted manner. Of course, the resolution or number of pixels of the display device can also be advantageously considered together with the shape factor.
[0024] In some implementations, detecting the original content includes analyzing the original content to detect at least one content parameter, which represents at least one important feature of the original content, wherein the content parameter is considered when creating the display content so that the display content includes the important feature. This can advantageously also be done using machine learning models. In particular, this ensures that the display content not only looks attractive but also that relevant parts of the content are not lost. Content parameters may include at least one of the following: the type of the original content, at least one important image component, at least one important text component, and metadata of the original content (especially when the original content is multimedia content). For example, in rectangular original content, relevant text may be located at the edges, and the relevant text may then be appropriately integrated into circular display content. Other parameters, such as the genre of the musical work, the current volume, etc., may also be taken into account in the construction of the display content to further improve the user experience.
[0025] In some implementations, at least one driving parameter is detected as a vehicle parameter, representing at least one characteristic of the vehicle's current driving, i.e., especially the driving during which display content is to be created (and displayed). The driving parameter may include at least one of the following: driving conditions (e.g., whether the vehicle is moving or stationary), navigation information (e.g., route information, destination information), vehicle speed, driving mode, driving duration, current time (time or especially daytime), and weather information (weather forecast or current weather). To detect the driving parameter, and especially to detect the surrounding environmental conditions that can be taken into account in the creation of display content, external cameras or other external or internal sensors of the vehicle may be used. With the driving parameter taken into account, the display content can be adapted more precisely. For example, it may be specified that the display content only includes motion components when the vehicle is stationary or the corresponding autonomous driving mode is activated, so as not to distract the driver.
[0026] In some implementations, at least one user parameter is detected as a vehicle parameter, representing at least one current characteristic of the current user. The user parameter may include at least one of the following: the user's identity, the user's emotional or mood state, the user's direction of observation, or other parameters obtained by the driver monitoring system, particularly via cameras or other (interior space) sensors, as well as the user's seating position or, generally, seat occupancy status. The displayed content can be tailored to individual needs, which can improve both the user experience and driving safety, for example, when the driver's direction of observation and level of attention are taken into account.
[0027] In some embodiments, at least one type and / or location of a display device in the vehicle is detected as a vehicle parameter. This allows, for example, differentiation of whether the display device is visible to the driver or whether it is solely for the entertainment of rear passengers. The displayed content can then be adapted accordingly, particularly regarding driving safety.
[0028] In some implementations, the displayed content is background content for the display device, wherein foreground content to be displayed is further analyzed, and the foreground content is to be displayed together with the background content, wherein the foreground content is considered when creating the displayed content. This can be avoided, for example, by preventing relevant portions of the background from being obscured by the foreground, or by preventing portions of the foreground and background from being difficult to identify due to unfavorable overlap.
[0029] A second aspect of the invention relates to a system for data processing, the system having at least one processor configured to perform a method according to a first aspect of the invention.
[0030] A third aspect of the invention relates to a computer program comprising instructions that, when executed on a system according to a second aspect, cause the system to perform the method according to a first aspect.
[0031] The computer program can be stored, in particular, on a non-volatile data carrier. Preferably, this is a data carrier in the form of an optical data carrier or a flash memory module. Advantageously, the computer program should be transacted independently of the processor platform on which one or more programs are to be implemented. In another implementation, the computer program can exist as a file on a data processing unit, particularly a server, and can be downloaded via a data connection, such as the Internet, or a dedicated data connection, such as a private network or local network. Furthermore, the computer program can have multiple separate program modules that work together.
[0032] The system according to the second aspect may accordingly have a program memory in which a computer program is stored. Alternatively, the system may also be designed to access, via a communication connection, a computer program available externally, for example, on one or more servers or other data processing units, particularly for exchanging data with that computer program, which is used or constitutes the output of the computer program during the operation of the method or the computer program.
[0033] The features and advantages explained with respect to the first aspect of the invention also apply accordingly to the other aspects of the invention. Attached Figure Description
[0034] Other advantages, features and applicability of the present invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings.
[0035] Here: Figure 1 A flowchart illustrating a method according to one embodiment is shown; and Figure 2 A view showing the interior space of the vehicle; Figure 3 Different examples of display shape factors are shown; and Figure 4 This shows an example of creating display content for a circular display. Detailed Implementation
[0036] In all figures, the same reference numerals are used for the same or corresponding elements of the invention.
[0037] exist Figure 1 The diagram shows a flowchart of one embodiment of a method 100 for generating display content. Figure 2 The diagram shows a view of the vehicle interior space 1 with three different display devices 2, 3, and 4. On one hand, a central display device 2 is located in the dashboard, which typically constitutes the vehicle's main display and can display various content, such as vehicle data and settings, navigation data, and multimedia data. Additionally, a display device 3 is located on the passenger side, primarily for the passenger's entertainment. Finally, a circular display device 4 is located in the dashboard, which can be used to display various information about the vehicle, but can also display current multimedia content. Furthermore, a rear-seat display device (not shown) for passenger entertainment may also be provided. A camera 5 is used as part of a driver monitoring system to monitor the driver.
[0038] Depend on Figure 2 The different shape factors of display devices 2, 3, and 4 are clearly visible inside the vehicle. Figure 3The illustration exemplifies the different display sizes and shape factors 10 that may occur within a fleet of vehicles from a single vehicle manufacturer. It is clear that significant resources must be invested until the vehicles are released to the market to adapt the display content to each individual display.
[0039] The basic concept of this invention is to generate display content, such as background images, based on original content for a display device. This display content automatically adapts to the shape factor of the display and is simultaneously based on multimedia content that is being played on a central display, a rear-seat display, or for individual passengers, such as the front passenger, or other entertainment devices in the vehicle. Furthermore, context-dependent information, based on data from interior space cameras, vehicle sensor arrays, route planning, and real-time vehicle information, is considered when creating these background images. Based on the driver's personalized information, these images can also be more effectively tailored to the driver's preferences.
[0040] By adapting the displayed content to the corresponding display device in the vehicle, black edges can be avoided, for example. Because the displays for in-vehicle entertainment systems are relatively small (compared to home entertainment electronics), the available space should be used optimally. Entertainment content should not be cropped but rather adapted to the shape factor of the car's displays. Many users always expect fresh and current content, not just a collection of standard images available at the point of sale.
[0041] Furthermore, a strong experience is often desired when reproducing multimedia content. The user experience can be further improved by tailoring it to the current situation and driving context, emphasizing the user's personality, expectations, and current emotions (e.g., through in-car camera recognition). In other words, the right content should be displayed to the right person in the right situation.
[0042] Furthermore, drivers should not be distracted while driving. It is expected that displayed content will be adapted to ADAS levels. Some users also expect that in-vehicle functions and displayed content will be seamlessly accessible on their mobile devices, such as smartphones, smartwatches, tablets, or other devices. Finally, there may be legal regulations, such as restrictions on the censorship of certain entertainment content due to Digital Rights Management (DRM).
[0043] To meet all the above needs and requirements, display content is created based on the original content using method 100. For this purpose, a series of vehicle parameters are considered (step S2), including the geometry or shape factor of the display device, as well as content parameters, driving parameters, and user parameters. While these are in... Figure 1The parameters are detected in separate steps, but it should be understood that the order and composition are arbitrary. Alternatively, all vehicle parameters can be detected simultaneously and then processed.
[0044] In step S1, the original content is first provided. This could be, for example, the current multimedia content (audio, video, game). Content parameters are then derived in step S2a. These content parameters include, for example, the type of multimedia content, cover art (as an image), genre, current volume, and the frequency or tempo of the song. If these (content) parameters are not readily available, an AI algorithm can be used to predict them.
[0045] In step S2b, driving parameters are detected, such as driving status, time (day or night), speed, information about route guidance, driving conditions / criticalities (e.g., autonomous mode), etc. The location of the display device within the vehicle (driver, front passenger, front, or rear) can also be determined. External cameras or sensors can be used to detect other driving parameters, such as the vehicle's surrounding environmental conditions.
[0046] In step S2c, user parameters or personal parameters are detected, such as the driver's / co-driver's emotional state, the driver's line of sight, the driving environment, and the passenger's personal preferences, as detected by the interior space camera.
[0047] In step S3, these vehicle parameters are now fed into a generative AI algorithm that generates images (or, in another embodiment, motion graphics or full video) precisely adapted to the situation, display device, and content. The resulting display content can then be displayed on the display device (step S4).
[0048] Figure 4 The following example illustrates how display content 9 is generated for display device 6 for a played song. The original content 7 does not fill the surface of display device 6, resulting in an unused or black surface 7'. If the portion is supplemented to fill the surface, original content 8 can be generated; however, this original content must be truncated at the edge 8' because it has a different shape factor (here, a rectangle rather than a circle). Only display content 9 generated by this method, taking into account the aforementioned parameters, accurately fits for display device 6. It is also conceivable that the outline graphic could be a moving element that changes its speed and color according to the intensity of the music.
[0049] The method 100 can also be used to adapt external content, such as third-party applications, to display devices, which typically exist in a format and design determined by the third-party provider. For example, third-party applications often exist in a square design and can only be adapted with difficulty to other display shapes in vehicles. Using the method 100, such external content can now be flexibly adapted to different display devices with different shape factors, so that the external content appears, for example, in its entirety without being cropped or distorted. Here, the free surface surrounding the initial design of the application can be filled, for example, simply by continuing the pattern using generative artificial intelligence, or it can be automatically filled with the content of the application.
[0050] In particular, the requirements explained above are met by method 100. Display content is generated that adapts to the corresponding monitor size, thereby optimizing the full and unique use of the monitor size. Content is not unintentionally cropped; the monitor format is taken into account in the generated display content. The display content is always fresh and current through the use of generative AI algorithms that “quasi-automatically” create images based on current media content. There is no need to create predefined images. User experience is improved by adapting the color design of the created display content based on the original content, but also on other parameters such as daytime, driving conditions, and music parameters such as volume and genre. Personal experience can also be improved by utilizing data collected by interior space cameras, sensor information, route information, and personal preferences to enhance contextual relevance for the user at any given moment.
[0051] When considering the display's position within the vehicle, driving safety can be ensured. For example, images / videos can be adapted to be non-distracting (e.g., through less detail, blurry content, no motion, etc.) as long as they are visible to the driver. When considering ADAS status, the displayed content can also be adapted accordingly. For example, when Autonomous Driving Level 3+ is activated, moving content may be permitted; otherwise, it may not.
[0052] Finally, it can be envisioned that the method considers not only the devices integrated in the vehicle, but also the personal terminal devices of the corresponding users, such as their smartphones. For this, necessary information must be synchronized between the vehicle and the personal terminal device. This could be achieved, for example, by wirelessly transmitting metadata about the content being played in the vehicle to the smartphone. Accordingly, the vehicle application on the mobile terminal device can adapt to its context based on this metadata and according to the size and shape factors of the terminal device (e.g., a smartphone with angular features, a smartwatch with round features, AR glasses with augmented reality, etc.).
[0053] While at least one exemplary embodiment has been described above, it should be noted that numerous variations exist. It should also be noted that the described exemplary embodiments are merely non-limiting examples and are not intended to limit the scope, applicability, or configuration of the apparatus and methods described herein. Rather, the foregoing description will provide guidance to those skilled in the art for implementing at least one exemplary embodiment, wherein it should be understood that various changes can be made to the functionality and arrangement of the elements described in the exemplary embodiments without departing from the technical solutions and their legal equivalents as defined in the appended claims.
[0054] List of reference numerals
[0055] 100 Methods for generating display content
[0056] 1. Vehicle interior space
[0057] 2. Display device (central display)
[0058] 3. Display device (passenger's display)
[0059] 4. Display device (instrument display)
[0060] 5 cameras
[0061] 6 display devices
[0062] 7 Original Content
[0063] 7' Unused surface
[0064] 8 Original Content
[0065] 8' protruding edge
[0066] 9. Display content generated
[0067] 10 shape factor
Claims
1. A method (100) for generating display content (9) for a display device (6) in a vehicle, wherein, The method includes: - Provide the original content (7, 8) as the basis for the display content (9) to be created for the display device (6); - Detect at least one vehicle parameter; and - Display content (9) for the display device (6) is created from the original content (7, 8), wherein the creation is performed with the aid of a generative machine learning model taking into account the at least one vehicle parameter, such that the resulting display content (9) represents the original content (7, 8) and is adapted to the at least one vehicle parameter.
2. The method according to claim 1, wherein, At least one shape factor (10) of the display device (6) is detected as a vehicle parameter, the shape factor defining the size and / or shape of the display device (6), and the original content (7, 8) is display content having a format that is not suitable for the shape factor of the display device (6), and the display content (9) is created such that the format of the created display content (9) matches the shape factor of the display device (6).
3. The method according to claim 1 or 2, wherein, Detecting the original content (7, 8) includes analyzing the original content (7, 8) to detect at least one content parameter, the content parameter representing at least one important feature of the original content (7, 8), the content parameter being considered when creating the display content (9) so that the display content (9) includes the important feature.
4. The method according to claim 3, wherein, The content parameters include at least one of the following: the type of the original content, at least one important image component, at least one important text component, and the metadata of the original content.
5. The method according to any one of the preceding claims, wherein, At least one driving parameter is detected as a vehicle parameter, the driving parameter representing at least one characteristic of the vehicle's current driving.
6. The method according to claim 5, wherein, The driving parameters include at least one of the following: driving conditions, navigation information, vehicle speed, driving mode, driving duration, current time, and weather information.
7. The method according to any one of the preceding claims, wherein, Detect at least one user parameter as a vehicle parameter, wherein the user parameter represents at least one current feature of the current user.
8. The method according to claim 7, wherein, The user parameters include at least one of the following: user identity, user emotion, user's observation direction, user's seat position, and seat occupancy status.
9. The method according to any one of the preceding claims, wherein, Detect at least one type and / or location of the display device in the vehicle as a vehicle parameter.
10. The method according to any one of the preceding claims, wherein, The displayed content is background content for the display device, and the method further includes: - Analyze the foreground content to be displayed, which is to be displayed together with the background content, taking the foreground content into account when creating the display content.
11. A system for data processing, the system having at least one processor configured to perform the method according to any one of the preceding claims.
12. A computer program comprising instructions that, when executed on a system according to claim 11, cause the system to perform the method according to any one of claims 1 to 10.
