Operation of an operating device

Abstract

The present invention relates to a method, a computer program with instructions and a device for operating an operating device, as well as to a corresponding operating device. In a first step, a user input can be recorded (10) in order to determine a predetermined thematic design. After dynamic content has been read in (11), the read-in content is processed by an artificial intelligence algorithm (12) in order to generate content adapted to the predetermined thematic design. At least one adapted content is reproduced by the operating device (13). The invention also relates to a means of transport and a user device with an operating device which use a method or a device according to the invention.

Description

[0001] The present invention relates to a method, a computer program with instructions, and a device for operating an operating device, as well as to a corresponding operating device. The invention also relates to a means of transportation and a user device with an operating device that utilize a method or device according to the invention.

[0002] The number and size of display devices in vehicles are constantly increasing. Display devices can be found on the market, for example, as instrument clusters for the driver, as central displays, and even as passenger displays. These display devices often also serve as control devices, often referred to as human-machine interfaces (HMI).

[0003] While the design of the displayed content was initially fixed, with more modern operating devices the user could influence the design of the displayed content to a certain extent.

[0004] For example, CN 105117225 A describes a graphical user interface for a vehicle infotainment system in which a user can change a color scheme and a background image.

[0005] A modern control panel allows the use of different themes or switching between them. Themes, also known as themes, are designs that represent a specific topic and contain special design elements. For example, a theme might incorporate visual and auditory design elements from a popular film or display appropriate content related to relaxation and meditation.

[0006] Against this background, CN 116302272 A describes a graphical user interface for a vehicle infotainment system that allows switching between different thematic designs.

[0007] The use of motifs is not limited to control devices installed in vehicles. Motifs can also be used on user devices, such as mobile communication devices or computers.

[0008] For example, US 2004 / 0216054 A1 describes a mobile communications device with a graphical user interface. The graphical user interface allows the user to choose between various design themes.

[0009] Creating themes is a challenge for the user interface designer and a complex process, as several elements of the user interface must be adapted accordingly. Especially with dynamic and database-based elements, such as profile pictures, it is impossible for the designer to create theme adjustments in advance.

[0010] It is an object of the invention to provide improved solutions for operating an operating device.

[0011] This object is achieved by a method having the features of claim 1, by a computer program with instructions according to claim 8, by a device having the features of claim 9, by an operating device according to claim 10, by a means of transport according to claim 11, and by a user device according to claim 12. Preferred embodiments of the invention are the subject of the dependent claims.

[0012] According to a first aspect of the invention, a method for operating an operating device comprises the steps: - Reading dynamic content; - Processing the read content using an artificial intelligence algorithm to generate content adapted to a given thematic design; and - Playback of at least one adapted content by the operating device.

[0013] According to a further aspect of the invention, a computer program contains instructions which, when executed by a computer, cause the computer to perform the following steps for operating an operating device: - Reading dynamic content; - Processing the read content using an artificial intelligence algorithm to generate content adapted to a given thematic design; and - Playback of at least one adapted content by the operating device.

[0014] The term "computer" should be understood broadly. In particular, it also includes control units, embedded systems, and other processor-based data processing devices. The execution of the aforementioned steps can be performed directly by the computer or involve the computer controlling a component intended to execute a step, such as a projection unit or a sensor.

[0015] The computer program may, for example, be made available for electronic retrieval or stored on a computer-readable storage medium.

[0016] According to a further aspect of the invention, a device for operating an operating device comprises: - a reading module for reading dynamic content; - a processing module for processing the read content using an artificial intelligence algorithm to generate content adapted to a given thematic design; and - a playback module for playing at least one adapted content by the operating device.

[0017] In the solution according to the invention, the generation of different motifs and the required modification of dynamic or database-based HMI elements are carried out by an artificial intelligence algorithm. For this purpose, the system can predefine which HMI elements can be replaced depending on the motif. The algorithm then processes the original HMI elements in such a way that they fit into the predefined thematic design. However, the content remains fundamentally recognizable. Dynamic content can, for example, be provided by a user, e.g., as content of a contact list. Predefined thematic designs can be films or painting styles, for example. The algorithm can be implemented in the operating device itself or in the device that uses the operating device. Alternatively, the algorithm can also be implemented in connected hardware or a cloud solution.

[0018] According to one aspect of the invention, user input is captured to determine the predefined thematic design. This allows a user to choose between different motifs and thus adapt the design to personal preferences or a current mood.

[0019] According to one aspect of the invention, the user input comprises a selection of a predetermined thematic design or a request to provide a new thematic design. The available thematic designs can be provided by the manufacturer of the operating device or the associated software. To select a predetermined thematic design, the user can, for example, select from a list. Alternatively or additionally, it can be provided that the user is given the opportunity to request a new thematic design, e.g., by means of a voice or text input. In response to a corresponding input, a new thematic design is then generated.

[0020] According to one aspect of the invention, the artificial intelligence algorithm searches for training content or content templates based on the request. It can then be trained with the retrieved training content or use a retrieved content template to generate a thematic design. If, for example, the user desires a thematic design inspired by Van Gogh's paintings, the system can perform a corresponding image search in available databases or on the Internet and train the algorithm with the retrieved images. The algorithm is then able to process the dynamic content appropriately. An algorithm that only requires a sample image, however, can work directly using an image template without training.In both cases, the user can make a selection from the training content or content templates found in order to influence the thematic design.

[0021] According to one aspect of the invention, the dynamic content is image content or audio content. The image content can be, for example, the profile pictures in a telephone contact list, and the audio content can be ringtones assigned to the contacts. Image content can also be the faces of vehicle occupants recorded by an interior camera, modified by the algorithm to suit the theme, and displayed on a screen. The audio content can also be sound elements of the system that are linked, for example, to certain actions. The profile pictures can be modified by the algorithm so that they are adapted to the motif of a movie, so that they resemble actors in the film or generally reflect the look of the film. Accordingly, the audio content can be modified to reflect the style of the film's music or the film's sound effects.If necessary, additional HMI elements can be modified to create a consistent theme. For example, background images can reference movie scenes, or the voice of a character can be used for voice output. Dynamic content can also include lighting effects. For example, the color, gradient, and brightness of direct or ambient lighting in the (interior) room can be influenced to simulate, for example, a space atmosphere or the effect of a sunset, in keeping with the theme.

[0022] According to one aspect of the invention, the clothing, accessories, hairstyles or appearance of depicted persons or an image background are modified by the artificial intelligence algorithm. For example, if a western motif is chosen as the theme, the depicted persons can be shown with appropriate clothing or contemporary hairstyles. The appearance can also be changed, e.g. if smiling faces do not fit with a selected dark theme or if rather petite persons do not fit with a selected martial theme. The adaptation of the appearance can also include the body, movement, voice, manner of speaking and character traits of depicted persons. The depicted persons can be real persons or fictional characters.

[0023] According to one aspect of the invention, the artificial intelligence algorithm is formed by a generative adversarial network or a variational autoencoder. Suitable algorithms are described, for example, in X. Han et al.: "A Method for Style Transfer from Artistic Images Based on Depth Extraction Generative Adversarial Network," Applied Sciences. 2023; 13(2):867, in Z.-S. Liu et al.: "Multiple Style Transfer via Variational AutoEncoder," arXiv:2110.07375, or S. Yan et al.: "Combining GAN with reverse correlation to construct personalized facial expressions," PLoS ONE 18(8): e0290612. T. Walczyna et al.: "Overview of Voice Conversion Methods Based on Deep Learning," Applied Sciences. 2023; 13(5):3100.

[0024] According to one aspect of the invention, an operating device comprises a device according to the invention or is configured to carry out a method according to the invention.

[0025] Advantageously, a means of transport with an operating device or a user device with an operating device comprises a device according to the invention or is configured to carry out a method according to the invention for operating the operating device. The means of transport can be, in particular, a passenger car, but also a bus or a commercial vehicle. The user device can be, for example, a smartphone, a tablet, or a computer.

[0026] Further features of the present invention will become apparent from the following description and the appended claims taken in conjunction with the figures. Fig. 1 schematically shows a method for operating an operating device; Fig. 2 shows a first embodiment of a device for operating an operating device; Fig. 3 shows a second embodiment of a device for operating an operating device; Fig. Figure 4 schematically represents a means of transport in which a solution according to the invention is implemented; Fig. 5 shows schematically a vehicle interior with an operating device according to the invention; Fig. Figure 6 schematically shows a user device in which a solution according to the invention is implemented; and Fig. Figure 7 shows examples of adapting content to a given thematic design.

[0027] To better understand the principles of the present invention, embodiments of the invention are explained in more detail below with reference to the figures. It is understood that the invention is not limited to these embodiments and that the described features may also be combined or modified without departing from the scope of the invention as defined in the appended claims.

[0028] Fig. 1 schematically shows a method for operating an operating device. In a first step, a user input can be recorded 10 in order to determine a predetermined thematic design. For example, the user input can comprise a selection of a predetermined thematic design or a request to provide a new thematic design. After reading in 11 dynamic content, the read-in content is processed 12 by an artificial intelligence algorithm in order to generate content adapted to the predetermined thematic design. The artificial intelligence algorithm can, for example, be formed by a generative adversarial network or a variational autoencoder. In the event that a request to provide a new thematic design has been recorded, the artificial intelligence algorithm can, for example,Conduct a search for training content or content templates and be trained with the retrieved training content, or use a retrieved content template to generate a thematic design. The dynamic content is preferably image content, audio content, or lighting effects. In the case of image content, for example, clothing, accessories, hairstyles, or appearance of depicted people or an image background can be modified. In the case of audio content, for example, voices or speech patterns of people or sound effects can be modified. At least one adapted content is reproduced by the operating device 13.

[0029] Fig. 2 shows a simplified schematic representation of a first embodiment of a device 20 for operating an operating device. The device 20 has an interface 21, via which a user input E can be recorded, which a user makes by means of an operating device 1. The user input E can be used to determine a predetermined thematic design. For example, the user input E can comprise a selection of a predetermined thematic design or a request to provide a new thematic design. A reading module 22 is configured to read in dynamic content I. A processing module 23 is configured to process the read-in content I using an artificial intelligence algorithm in order to generate content I adapted to the predetermined thematic design. ato generate. The artificial intelligence algorithm can, for example, be formed by a generative adversarial network or a variational autoencoder. In the event that a request to provide a new thematic design has been recorded, the artificial intelligence algorithm can, for example, search for training content or content templates on the basis of the request and be trained with the training content found or use a found content template to generate a thematic design. The dynamic content is preferably image content or audio content or lighting effects. In the case of image content, for example, clothing, accessories, hairstyles or appearance of depicted people or an image background can be modified. In the case of audio content, for example, voices or manner of speaking of people or sound effects can be modified.A playback module 24 is configured to play at least one adapted content I. a by the operating device 1. For this purpose, the playback module 24 can, for example, display the adapted content I a output to the operating device 1 via the interface 21.

[0030] The read-in module 22, the processing module 23, and the playback module 24 can be controlled by a control module 25. Settings of the read-in module 22, the processing module 23, the playback module 24, or the control module 25 can be changed via the operating device 1 or another user interface 27. The data generated in the device 20 can be stored in a memory 26 of the device 20 if necessary, for example, for later evaluation or for use by the components of the device 20. The read-in module 22, the processing module 23, the playback module 24, and the control module 25 can be implemented as dedicated hardware, for example, as integrated circuits. Of course, they can also be partially or completely combined or implemented as software running on a suitable processor, for example, a CPU or a GPU.Alternatively, the interface 21 can also be divided into an input and an output. In the example shown, the device 20 is implemented as a standalone component. Alternatively, it can also be part of the control device 1.

[0031] Fig. 3 shows a simplified schematic representation of a second embodiment of a device 30 for operating an operating device. The device 30 has a processor 32 and a memory 31. For example, the device 30 is a computer, a control unit, or an embedded system. Instructions are stored in the memory 31 which, when executed by the processor 32, cause the device 30 to perform the steps according to one of the described methods. The instructions stored in the memory 31 thus embody a program executable by the processor 32 which implements the method according to the invention. The device 30 has an input 33 for receiving data. Data generated by the processor 32 are provided via an output 34. In addition, data can be stored in the memory 31. The input 33 and the output 34 can be combined to form a bidirectional interface.

[0032] The processor 32 may include one or more processor units, such as microprocessors, digital signal processors, or combinations thereof.

[0033] The memories 26, 31 of the described embodiments can have both volatile and non-volatile memory areas and can comprise a wide variety of storage devices and storage media, for example hard disks, optical storage media or semiconductor memories.

[0034] Fig. 4 schematically illustrates a means of transport 40 in which a solution according to the invention is implemented. In this example, the means of transport 40 is a motor vehicle. The motor vehicle has a sensor system 41, e.g., one or more video cameras. It also has an operating device 1, which can be implemented, for example, as a display device 42. A device 20 according to the invention, which can, for example, be a component of a computer 43, serves to operate the operating device 1. For example, the display device 42 can display a telephone contact list in which the profile pictures displayed are adapted to a thematic design desired by the user. The device 20 generates the correspondingly adapted content for this purpose. A connection to a backend can be established by means of a data transmission unit 44, e.g.for retrieving training content or for retrieving updated software for the vehicle's components. A memory 45 is provided for storing data. Data exchange between the various components of the vehicle takes place via a network 46.

[0035] Fig. 5 schematically shows a vehicle interior with an operating device 1 according to the invention. In this example, the operating device 1 is integrated into a dashboard of the vehicle. It can of course also be installed in a different position, for example in the center console or free-standing on the dashboard. A device 20 according to the invention, which is integrated into the operating device 1 or arranged elsewhere in the vehicle, is used to operate the operating device 1. In the example shown, the operating device 1 has a row of additional operating buttons 3 on the side next to a touch-sensitive display surface 2 and below the touch-sensitive display surface 2, for example for quickly selecting certain functional areas. In addition, there are two rotary controls 4, such as those typically used for volume control and scrolling within lists.Voice input can be captured via a microphone installed in the interior. This can be used to display, for example, a telephone contact list with thematically adapted profile pictures. Likewise, the visual appearance of displayed menus can be customized to a theme desired by the user.

[0036] Fig. Figure 6 schematically illustrates a user device 50 in which a solution according to the invention is implemented. In this example, the user device 50 is a smartphone. The smartphone has a touchscreen 51, which serves as an operating device 1 with a touch-sensitive input surface. A device 20 according to the invention integrated into the smartphone serves to operate the operating device 1. This device enables, for example, a telephone contact list with thematically adapted profile pictures to be displayed on the touchscreen 51.

[0037] Fig. Figure 7 shows examples of adapting Content I to a given thematic design. In this example, Content I is image content, specifically profile pictures. Fig. 7a) The profile picture shows a smiling man. The user has chosen a Western theme for the design. The artificial intelligence algorithm modifies the profile picture so that the resulting customized content I a shows a rather grim-looking man with a cowboy hat. Nevertheless, the original content remains fundamentally recognizable. Fig. 7b) The profile picture shows a smiling woman. The user has chosen Picasso as the theme for the design. The artificial intelligence algorithm modifies the profile picture so that the resulting customized content I a shows an abstractly drawn woman in the style of Picasso's works. Here, too, the original content remains fundamentally recognizable. List of reference symbols 1 control device 2 input area 3 control buttons 4 rotary controls 10 Capturing user input 11 Reading dynamic content 12 Processing the read content 13 Play at least one adapted content 20 Device 21 Interface 22 Reading module 23 Processing module 24 Playback module 25 Control module 26 storage 27 User interface 30 Device 31 storage 32 processor 33 Entrance 34 Exit 40 means of transport 41 Sensor technology 42 computers 43 Display device 44 Data transmission unit 45 storage 46 Network 50 user devices 51 touchscreen 52 processor E User input I Contents I a Adapted content QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] CN 105117225 A

[0004] CN 116302272 A

[0006] US 2004 / 0216054 A1

[0008] Cited non-patent literature

[0000] X. Han et al.: “A Method for Style Transfer from Artistic Images Based on Depth Extraction Generative Adversarial Network,” Applied Sciences. 2023; 13(2):867

[0023] Liu et al.: „Multiple Style Transfer via Variational AutoEncoder“, arXiv:2110.07375, oder S. Yan et al.: „Combining GAN with reverse correlation to construct personalized facial expressions“, PLoS ONE 18(8): e0290612

[0023] T. Walczyna et al.: „Overview of Voice Conversion Methods Based on Deep Learning“, Applied Sciences. 2023; 13(5):3100

[0023]

Claims

[1] Method for operating an operating device (1), comprising the steps: - Reading (11) of dynamic content (I); - Processing (12) of the read-in content (I) by an artificial intelligence algorithm in order to generate content (I) adapted to a given thematic design a ) and - Reproducing (13) at least one adapted content (I a ) by the operating device (1). [2] Method according to claim 1, wherein a user input (E) is recorded (10) for determining the predetermined thematic design. [3] Method according to claim 2, wherein the user input (E) comprises a selection of a predetermined thematic design or a request to provide a new thematic design. [4] The method of claim 3, wherein the artificial intelligence algorithm performs a search for training content or content templates based on the request. [5] Method according to one of the preceding claims, wherein the dynamic contents (I) are image contents or audio contents or lighting effects. [6] Method according to claim 5, wherein clothing, accessories, hairstyles or appearance of depicted persons or an image background are modified by the artificial intelligence algorithm. [7] Method according to one of the preceding claims, wherein the artificial intelligence algorithm is formed by a generative adversarial network or a variational autoencoder. [8] Computer program with instructions which, when executed by a computer, cause the computer to carry out the steps of a method according to one of the preceding claims for operating an operating device (1). [9] Device (20) for operating an operating device (1), comprising: - a reading module (22) for reading (11) dynamic contents (I); - a processing module (23) for processing (12) the read-in contents (I) by an artificial intelligence algorithm in order to generate contents (I) adapted to a given thematic design a ) and - a playback module (24) for playing (13) at least one adapted content (I a ) by the operating device (1). [10] Operating device (1), wherein the operating device (1) comprises a device (20) according to claim 9 or is configured to carry out a method according to one of claims 1 to 7. [11] Means of transport (40) with an operating device (1), wherein the means of transport (40) has a device (20) according to claim 9 or is designed to carry out a method according to one of claims 1 to 7 for operating the operating device (1). [12] User device (50) with an operating device (1), wherein the user device (50) has a device (20) according to claim 9 or is configured to carry out a method according to one of claims 1 to 7 for operating the operating device (1).

Images