Method for operating application software, proxy software for controlling the application software, processor system, and motor vehicle having processor system

Abstract

The invention relates to a method for operating application software (AS) in a motor vehicle (KFZ), the motor vehicle (KFZ) having a processor system and a display device (CID) and a combined display and operating device (ABE). The method is characterised in that the processor system executes the application software (AS) and, during execution, the application software (AS) generates output data (AD) which are to be displayed in a display region, and the processor system operates proxy software (PS) different from the application software (AS), which proxy software receives the output data (AD) from the application software (AS) and, depending on user input data (NED) generated by the display and operating device (ABE) in response to a user input, selects a region between the display device (CID) and the display and operating device (ABE) as the display region and forwards the output data (AD) to the selected display region.

Description

[0001] Method for operating an application software, proxy software for controlling the application software, processor system and motor vehicle with processor system

[0002] DESCRIPTION:

[0003] The invention relates to a method for operating application software in a motor vehicle, for which the motor vehicle has a processor system and a display device and a combined display and operating device.

[0004] A motor vehicle can contain multiple display devices, i.e., screens or displays. Particularly in the context of autonomous driving, where the driver requires minimal concentration on the road, it may be desirable to run or display application software on a specific screen available, such as for video playback or image viewing, e.g., vacation photos. It may also be desirable to switch the display of the application software between the available screens in the vehicle. Providing such functionality, especially for third-party application software, can be cumbersome, as the application software would have to be adapted to the software and hardware environment of the vehicle.Third-party application software refers to application software that is not developed or provided by the vehicle manufacturer. It could, for example, be streaming software that allows video playback. In the field of home entertainment, so-called screen mirroring is known in this context. This is a technology that allows the screen of a mobile device to be transferred to a television screen, for example, and thus the image output of the application or application software running on the mobile device to be viewed on the large television screen.Likewise, with screen casting, an application software that is running on a mobile phone, for example, can be played on an external screen. In contrast to screen mirroring, the playback then only takes place on the external screen and not additionally on the screen of the mobile phone.

[0005] From US 2011 / 0185390 A1, a system for the interaction of a user of a motor vehicle with third-party application software on a mobile terminal is known, wherein the mobile terminal can be operated via motor vehicle-side input means when contents of the third-party application software are displayed on a motor vehicle screen, and input interfaces of the mobile terminal are converted to the motor vehicle-side input means.

[0006] From DE 10 2014204 747 A1, a motor vehicle computer system with a motor vehicle-side display arrangement is known, which can be used to play back media content from a mobile terminal coupled to the motor vehicle computer system, wherein the mobile terminal can be used by a user or motor vehicle occupant as a display and operating device for creating motor vehicle configurations and for moving media content to the motor vehicle-side display arrangement.

[0007] US 2019 / 0339918 A1 discloses a method for providing a user interface for third-party application software for display via a motor vehicle infotainment system, wherein the infotainment system adopts a user interface generated by a separate display and operating device of a user with operating options adapted to the third-party application software. A disadvantage of the described prior art is that for reproducing or displaying output data from application software or third-party application software and / or for controlling the application software in a motor vehicle, either an additional mobile device or at least a separate, i.e., non-motor vehicle-side display and operating device is required.Running the application directly on a vehicle's computer system and having it control the vehicle's various display devices is not possible without prior reprogramming.

[0008] US 2020 / 0387278 A1 describes techniques that facilitate remote access to a vehicle touchscreen.

[0009] US 2016 / 0321043 A1 shows a method for integrating an algorithm, which is coded in a programming language and compiled into algorithm software libraries, into a target information processing system comprising a set of target system algorithms associated with a process proxy, algorithm variables and variable data object types.

[0010] US 2013 / 0212237 A1 discloses a customizable configuration data structure containing information about the proxies to be deployed, wherein a proxy infrastructure has a controller for reading the customizable configuration data structure and creating the proxies and a management processor has firmware-based functions to which the proxies are to access via an interface between the proxy infrastructure and the management processor.

[0011] DE 102014 016 570 A1 shows a vehicle door with a control panel which is designed as a touchscreen and can be switched between at least two display modes, wherein in the first control display mode the touchscreen shows at least a first control operating element for controlling a first actuator of the vehicle and in the second control display mode a second control operating element for controlling a second actuator which is different from the first actuator.

[0012] The subsequently published DE 102021 214619 A1 shows an operating device for a motor vehicle, which has a particularly touch-sensitive surface element for detecting control commands from a user on the surface element, wherein the surface element is designed to be elastically deformable and is held on a support body, wherein it is provided that a layer of an elastically deformable soft material is arranged between the support body and the surface element.

[0013] The invention is based on the object of controlling several screens in a motor vehicle with as little effort as possible when using application software.

[0014] The problem is solved by the subject matter of the independent patent claims. Advantageous developments of the invention are described by the dependent patent claims, the following description, and the figures.

[0015] The invention provides a method for operating application software in a motor vehicle, wherein the motor vehicle has a processor system and a display device and a combined display and operating device. There is therefore at least one display and operating device in which both the display functionality and the operating functionality are used by the method (e.g., a touchscreen). In addition, there is at least one display device in which only the display functionality is to be used by the method (e.g., a screen). There are therefore several display areas. The method is characterized in that the processor system executes the application software, and the application software generates output data during execution, e.g., video stream data of a video film, which are to be displayed in a display area. The “display area” is, for example,to be understood as a pixel field or “window” in which the output data is to be displayed.

[0016] The method assumes that the application software itself is not programmed to switch between the multiple display areas. This is because the processor system runs proxy software that is separate from the application software. This proxy software receives the output data from the application software and, depending on user input data, selects between the display device and the display and control device as the display area and forwards the output data to the selected display area. The user input data can be generated by the display and control device in response to a user input.

[0017] In other words, a user in a motor vehicle should be able to select on which of the screens available in the motor vehicle and belonging to the motor vehicle output data of a currently executed application software should be reproduced or displayed.

[0018] This offers the advantage that the user can display or play the application software on a screen that is currently visible to them, for example, based on their seating position. Especially with regard to autonomous driving, a driver, who may be the user, can also change their seating position in such a way, for example, by moving the seat back or reclining a backrest, that the screens visible to them change compared to a seating position in which they are driving the vehicle themselves.

[0019] The processor system on which the application software is to be executed can be, for example, a computing device and / or a computer hardware environment of the motor vehicle. The display device on which the output data can be displayed or reproduced can be a TFT screen (TFT: Thin-Film Transistor), an OLED screen (OLED: Organic Light-Emitting Diode), or an LCD display (LCD: Liquid Crystal Display). The combined display and control device can comprise a screen like the display device, which additionally has a touch-sensitive surface configured to accept user inputs, for example, by touching a graphical control element on the screen with a user's finger.

[0020] The application software can be a computer program executable by the processor system, for example, a computer program for playing media content. The proxy software can be provided by a motor vehicle manufacturer and stored in an executable manner on the processor system. The application software can originate from a third-party provider, i.e., be downloaded to the processor system from a third-party server facility. The output data can comprise an image stream, for example, in the case of media application software for displaying media content, for example, moving image material such as a film. The proxy software represents a computer program or adapter software that is executed by the processor system but is different from the application software.The proxy software, as adapter software, can act as a link or mediator between the operating system contained within the processor system and the application software through interfaces for receiving and outputting data. The interfaces of the proxy software can interact or be connected to corresponding interfaces on the part of the application software and the operating system. An interface can refer to a program interface or software interface for interprocess communication (IPC), such as an API (Application Programming Interface). The output data of the application software should first be transferred to the proxy software.User input data generated by a user can be passed to the proxy software. The proxy software can determine the display area, i.e., the screen, on which the output data is displayed after the output data has been passed to the operating system. The combined display and control device can be a touchscreen located within the user's reach. The other display device can be located outside of the user's reach. In a given seating position, the range can be between 0.7 m and 1.1 m.

[0021] The invention also includes embodiments which provide additional advantages.

[0022] One embodiment provides that the proxy software has the following interfaces: a user interface, via which the user input data for operating the application software is received from the display and operating device by the proxy software, an input / output interface, via which a first part of the user input data, which goes beyond the selection of the display area, can be transferred from the proxy software to the application software (for example, operating inputs intended for the application software itself), an output data reception interface, via which the proxy software receives the output data (for example, video data) from the application software, wherein the output data is provided with a display instruction in an allocation unit comprised by the proxy software depending on a second part of the user input data,which identifies the display area (the user input data thus determines the screen for output), and an output data transfer interface, via which the output data together with the display instruction are transferred to an operating system included in the processor system, which transfers the output data for output depending on the display instruction to the display area described by the display instruction (for example, the operating system takes the video data and displays it on the screen,which is specified by the proxy software). This results in the aforementioned adapter function of the proxy software. In other words, the proxy software should be connected between the application software and the operating system. For this purpose, the proxy software can have interfaces. The output data of the application software can be received by the proxy software via the output data reception interface. Depending on the user input data, the proxy software should generate a display instruction. For receiving the user input data, the proxy software can have the user interface. The display instruction can specify,On which display area the output data should be output after being transferred from the proxy software to the operating system. For this purpose, the display instruction (e.g., a selection command for selecting the display device to be used) can be transferred to the operating system along with the output data (e.g., video data) via the output data transfer interface. The transfer of the output data to the respective display device or display and control device specified by the display instruction for output can be done by the,

[0023] Operating system in a manner known from the prior art. The user input data can consist of two parts: a first part that describes commands or operating instructions from the user for controlling the application software, and a second part that describes the selection of the display area by the user and is intended for the proxy software. For transferring the first part, the proxy software can have an input / output interface via which the commands are transferred to the application software. The selection of the display area or the generation of the display instruction can be carried out by the allocation unit, which can be comprised of the proxy software, depending on the second part of the user input data.This software architecture or software network architecture of the proxy software and the application software and the operating system in conjunction with the display device and the display and operating device provides the advantage that the user can control the application software via the display and operating device within his reach and can determine or select the display area.

[0024] One embodiment of the method provides that the application software to be executed is downloaded from a third-party server device onto the processor system and installed there.

[0025] In other words, the application software should not only originate from the vehicle manufacturer's application software offering. Rather, the proxy software should enable third-party application software that is not tailored to the software and / or hardware environment of the vehicle to be operated in the vehicle. "Operating" here means executing the application software on the processor system, defining the display area by a user, outputting application software output data to the display area, and controlling or operating the application software by the user. Such a third-party server facility could be, for example, the App Store™ from Apple™ or the Google Play Store™ from Google™.For example, the application software can be designed or programmed to operate on a hardware and / or software environment that is not typical for a motor vehicle, such as a mobile device, i.e., a mobile phone, a tablet PC, or a desktop PC. The proxy software then simulates the application software's "familiar" hardware and / or software environment by providing and / or receiving the data streams required to operate the application software or those that occur during operation, while simultaneously adapting them to the hardware and / or software environment present in the motor vehicle.

[0026] One embodiment provides that the proxy software queries at least one operating function for operating the application software from metadata of the application software and makes it available in a selectable manner on the display and operating device, wherein for this purpose graphical operating elements are provided by the proxy software independently of the application software.

[0027] In other words, the proxy software should determine, from at least one information file about the application software, for example, created by the operating system or supplied by the manufacturer of the application software, control commands or control functions with which the application software can be controlled, i.e. which can be received as input and understood or implemented by the application software. Depending on the determined control commands, the proxy software can provide graphical control elements on the display and control device, each of which describes the respective control command, for example using a symbol. When the user triggers or selects a control element, the control command assigned to the control element by the proxy software can be transferred to the application software via the proxy software.This results in the advantage that the proxy software provides the user with an operating or control option that is tailored to the operation of the application software via the display and operating device.

[0028] Thus, one embodiment provides in particular that the proxy software provides a playback control function and / or a volume control function and / or a voice control function and / or a cursor control function as the at least one operating function.

[0029] In other words, in addition to the application software-specific and determined control commands, the proxy software should provide generic or general control commands or operating functions for the user to control the application software, which can be selected or triggered. For example, a playback control function such as play / pause and / or a volume control function—i.e., control commands tailored to controlling media application software—can be provided on the display and operating device in the form of control elements. Additionally or alternatively, a control element can also be provided on the display and operating device. When selected by the user, this control element triggers a voice control function, allowing the user to control the application software via voice control.Additionally or alternatively, a cursor control function can be provided via the touch-sensitive display or screen of the display and operating device, with which functional elements can be selected. These functional elements can be displayed on a screen that is not touch-sensitive or is outside the reach of the user. Range here refers to the operating range within which the user can touch an operating device, for example, with their hand. The at least one operating function can additionally or alternatively be an operating function for controlling media content, such as a fast-forward function, a rewind function, and / or a progress bar. This offers the advantage that the user can control the application software according to their needs.

[0030] One embodiment provides that the display and operating device has a touch-sensitive operating surface for receiving a user input, wherein user input data is generated in response to the user input.

[0031] The display and control device can be designed as a touch display or touch screen, as is known from the prior art. This offers the advantage that the display and control device does not require any additional input device or control device, such as a computer mouse or keyboard, in addition to the display device, i.e., a screen, to receive user input.

[0032] One embodiment provides for the display and operating device to be provided on at least one armrest of a seat of the motor vehicle. This provides the advantage that the display and operating device is always within reach of the user, even if they have positioned their backrest horizontally or have moved the seat back from a driving position. Preferably, the display and operating device is arranged or attached to an armrest of the driver's seat. Additionally or alternatively, the display and operating device can be arranged on an armrest of any other seat in the motor vehicle.

[0033] For use cases or application situations that may arise during the method and which are not explicitly described here, it may be provided that, in accordance with the method, an error message and / or a request to enter user feedback or user input is output and / or a default setting and / or a predetermined initial state is set.

[0034] The invention comprises proxy software for coupling application software to an operating system comprised by a processor system, wherein the proxy software comprises instructions which, when executed by the processor system, are configured to cause the processor system to execute the embodiments or further developments relating to the proxy software.

[0035] The proxy software can be included in the operating system or executed as a standalone computer program by the processor system. If the proxy software is executed as a standalone computer program, the application software can be spawned by the proxy software. This means that the application software is executed by the proxy software or dependent on the proxy software as a so-called child process in order to link the application software to the operating system via the proxy software. For example, the output of the application software can be forwarded to the proxy software via operating system commands, such as strace, where it can then be redirected to the selected output area. This offers the advantage that the proxy software can be executed separately or in a separate

[0036] It can be installed or retrofitted in an installation step on the processor system, for example, if it was not already installed when the vehicle was delivered, to link application software with the operating system. If the proxy software is already included in the operating system upon delivery, the advantage is that the proxy software does not have to be installed in a separate step, thus saving the user time and effort. A child process can be initiated using operating system commands such as fork or Process / Task.

[0037] The invention comprises a processor system comprising a processor circuit for executing application software, a digital data memory for executably storing application software and an operating system and proxy software, a display device, a combined display and control device, and a transmission unit configured for wireless data transmission to a third-party server device for downloading application software. The data transmission can be performed via a computer network such as the Internet. The transmission unit can be a mobile radio module and / or a WLAN module (WLAN: Wireless Local Area Network).

[0038] For this purpose, the processor circuit can have at least one microprocessor and / or at least one microcontroller and / or at least one FPGA (Field Programmable Gate Array) and / or at least one DSP (Digital Signal Processor). In particular, a CPU (Central Processing Unit), a GPU (Graphical Processing Unit) or an NPU (Neural Processing Unit) can be used as the microprocessor. Furthermore, the processor circuit can have program code, in particular in the form of proxy software and / or at least one application software, which is configured to carry out an embodiment of the method according to the invention when executed by the processor circuit. The program code can be stored in a digital data memory of the processor circuit in addition to or alternatively to the digital data memory of the processor system. The processor circuit can, for example,based on at least one circuit board and / or on at least one SoC (System on Chip).

[0039] As a further solution, the invention also comprises the digital data memory of the processor circuit and / or the processor system, a computer-readable storage medium comprising program code, in particular in the form of proxy software and / or at least one application software, which, when executed by a computer or a computer network or the processor system and / or the processor circuit, causes them to carry out an embodiment of the method according to the invention. The storage medium can be provided at least partially as a non-volatile data memory (e.g., as a flash memory and / or as an SSD - solid state drive) and / or at least partially as a volatile data memory (e.g., as a RAM - random access memory). The storage medium can be arranged in the computer or computer network or the processor system and / or the processor circuit.The storage medium can also be operated, for example, as a so-called app store server and / or cloud server on the Internet. The computer or computer network can provide an additional processor circuit, for example, with at least one microprocessor. The program code can be provided as binary code and / or as assembly code and / or as source code of a programming language (e.g., C) and / or as a program script (e.g., Python).

[0040] The invention also includes further developments of the processor circuit according to the invention, which has features already described in connection with the further developments or embodiments of the method according to the invention. For this reason, the corresponding further developments of the processor circuit according to the invention are not described again here. Furthermore, the invention provides a motor vehicle comprising the processor circuit according to the invention.

[0041] The motor vehicle according to the invention is preferably designed as a motor vehicle, in particular as a passenger car or truck, or as a passenger bus or motorcycle.

[0042] The invention also encompasses combinations of the features of the described embodiments. The invention therefore also encompasses implementations that each comprise a combination of the features of several of the described embodiments, unless the embodiments are described as mutually exclusive.

[0043] Exemplary embodiments of the invention are described below. Shown are:

[0044] Fig. 1 is a schematic representation of a display and operating device with graphical operating elements for controlling an application software, the output data of which are displayed on a display device also shown; and

[0045] Fig. 2 is a schematic representation of a software network architecture of proxy software, application software and operating system in conjunction with a display device and a display and operating device for operating an application software.

[0046] The exemplary embodiments explained below are preferred embodiments of the invention. In the exemplary embodiments, the described components of the embodiments each represent individual features of the invention that can be considered independently of one another, each of which also develops the invention independently of one another. Therefore, the disclosure is intended to encompass combinations of the features of the embodiments other than those shown. Furthermore, the described embodiments can also be supplemented by further features of the invention already described.

[0047] In the figures, the same reference symbols designate elements with the same function.

[0048] Fig. 1 shows a schematic representation of an arrangement of a display and operating device ABE and a display device CID. The display and operating device ABE can be designed as a touch-sensitive display, as is known from the prior art. In a first display section 4 of the display and operating device ABE, graphical operating elements for controlling an application software AS can be provided. In a second display section 5 of the display and operating device ABE, further graphical operating elements can be provided for selection by a user, such as an application software selection element 1, a display change element 2, and / or a home element 3.

[0049] In one exemplary embodiment, a user can download application software AS from a server device of a third-party provider via a wireless communication unit or transmission unit via a computer network or data network such as the Internet, for example via a cellular network and / or a WLAN network. The transmission unit can be included in a processor system of the motor vehicle KFZ. After downloading, the application software AS can be installed in an executable manner, for example, on a digital data storage device included in the processor system. To execute the application software AS, the user can select the application software selection element 1 on the display and operating device ABE. A selection output can then open in which the user selects the application software AS from a possible plurality of already installed application software.In this example, the proxy software PS can be integrated into the operating system OS included in the processor system. In this case, the operating system OS can automatically detect that the application software AS selected for execution originates from a third-party provider and automatically connect the application software to interfaces of the proxy software. Alternatively, if the proxy software PS is installed as standalone application software, it can first be started by the user. A broad selection dialog can then provide the user with a selection of application software AS that should be started by the proxy software PS as a so-called child process. The application software AS to be executed can, for example, be media application software for playing films or music. It represents an example of the application software described here.It can be provided that the application software AS, after starting, initially outputs its output data, e.g. graphic data from an operating menu, also on the display and operating device ABE. When the application software AS is started, either directly or via the proxy software PS, it can be provided that the proxy software PS determines or queries which functions and / or control commands are stored in the application software AS from an information file contained in the operating system OS or a file collection of the application software, such as metadata and / or manifest files. Since this example may involve media application software, the operating or control functions that can be determined include, for example, the functions play, pause, skip forward, skip backward, a volume control, a mute function and / or a progress bar of a media file.After starting the application software AS on the display and control unit ABE, the proxy software PS can display the selection of the determined operating functions or control commands using corresponding graphical control elements. The control elements can be displayed in the first display section 4 of the display and control unit ABE, and the function executed there.

[0050] Application software AS can be faded transparently or semi-transparently. The user can now, for example, start a media file such as a film in the media application software. To do so, the user can select the corresponding control element, such as the play symbol, by touching it with their finger, whereby the display and control device ABE can be designed as a touch-sensitive display for this purpose. Once the film starts on the display and control device ABE, the user can then select display change element 2. The playback of the film or the display of output data AD from the application software AS can then be moved to a display device CID arranged in the motor vehicle KFZ.After moving the application software AS, for example, a message can be output to the user in the first display section 4 on the display and operating device ABE that the currently executing application software AS is being output on the display device CID or another screen that may be present in the motor vehicle KFZ and included in a processor circuit of the motor vehicle. If multiple display devices or screens are included in the processor circuit, an input prompt can be output to the user upon selection of the display change element 2, allowing them to select at least one of the screens. However, the operating elements in the second display section 5 can still be provided so that they can be selected when the application software is being output on the display device. The display device CID can be designed as a non-touch-sensitive screen.If the user now wishes to transmit a control command to the application software AS, for example, to pause the film, it can be provided that they first "return" the application software AS to the display and control device ABE (display change) by selecting the display change element 2. The control elements in the first display section 4 can now be made available and selected again. By selecting the home element 3, for example, the execution of the application software AS can be terminated and a start screen or exit screen can be displayed on the display and control device ABE and / or the display device CID, or an exit screen of the currently running application software.

[0051] The process underlying this embodiment will be explained with reference to Fig. 2.

[0052] A display system ASS arranged in a motor vehicle KFZ can comprise the display and operating device ABE, the display device CID and a processor system. The processor system can comprise the operating system OS, the proxy software PS and the application software AS stored on a digital data memory. When the user selects a control element on the display and operating device ABE, user input data NED can be generated and transferred to the proxy software PS via a user interface NS. The user input data NED can be divided into two parts: a first part NED1, which can comprise control commands to the app, and a second part NED2, which can comprise a selection of a display area.If, for example, the user starts the playback of a media file by selecting a control element provided for it, the user input data NED can be transmitted from the display and control device ABE via the user interface NS to the proxy software PS. The proxy software PS can transfer the first part of the user input data NED1, which describes the control command for playback, to the application software AS via an input / output interface IAS. The playback of the media file can then start on the display and control device ABE. If the user now wants to move the playback to the display device CID, they can select the display change element 2. This selection of the display area can be included in the second part of the user input data NED2 and transmitted in the proxy software PS from the user interface NS to an allocation unit ZE.In the allocation unit ZE, a display instruction AW can be generated depending on the second part of the user input data NED2. This means that the display instruction AW can be used to specify, based on a user input, on which screen the output data of the application software AS should be shown—i.e., on the display and control device ABE or on the display device CID. The output data AD, which in this example can correspond to an image stream or audio data, can be transferred in the application software AS to the proxy software PS. The proxy software PS can have an output data reception interface ADES for this purpose. After generating the display instruction AW, the proxy software PS can transfer the output data AD together with the display instruction AW to the operating system OS via an output data transfer interface ADUS.With the aid of the display instruction AW, the operating system OS can, as is known from the prior art, forward the output data AD to the screen selected by the user, in this case to the display device CID or to the display and control device ABE, wherein the display device CID and the display and control device ABE can be connected to the operating system OS via a respective data interface.

[0053] In a further embodiment, the application software AS can be executed directly after startup on the display and control device ABE. In this example, however, the control functions or control commands for the application software AS determined by the proxy software PS can continue to be available for selection on the display and control device ABE after a change in the display area. To do this, the user starts, for example, the playback of a media file such as a film, as in the first embodiment, and shifts the playback of the film or the output of the output data AD from the display and control device ABE to the display device CID by selecting the display change element 2.After switching the display area from the ABE display and control unit to the CID display unit, the control functions determined by the PS proxy software can continue to be provided in the first display section 4 in the form of control elements, making them selectable for the user. This offers the advantage that the user can continue to control the AS application software via the ABE display and control unit without having to switch the display area back to the ABE display and control unit by selecting the display switching element 2.

[0054] A particularly preferred embodiment is described below.

[0055] Current infotainment generations or display systems in motor vehicles can be characterized by the fact that both OEM-specific (OEM: Original Equipment Manufacturer) content such as application software AS and application software AS from third-party providers can be integrated into the system or processor system or the display system ASS.

[0056] Typical OEM application software can be vehicle application software or navigation application software. A typical third-party application software AS, which the OEM may not want to develop itself but would rather take "as is" from a digital ecosystem or app store, could be Netflix™, for example. Such application software typically does not have OEM-specific input and / or output mechanisms, but, analogous to the tablet and / or smartphone world, can expect a touchscreen for display and / or user interaction. At the same time, vehicle interiors can evolve due to the possibilities of highly automated driving.A driver or user no longer has to sit directly behind a steering wheel within easy reach of displays and switches or the ABE display and control unit and a CID display unit, but can use the space gained for secondary activities and move away from a control panel for this purpose.

[0057] This can create a need to run third-party application software AS on displays or screens in the vehicle that may be out of touch range or the driver's or user's operating range. One challenge, therefore, can be to run third-party application software that expects to be operated via touch on non-touchable or inaccessible displays.

[0058] The idea is to first start the third-party application software AS on a smaller control display (FCD) or the display and control unit ABE, which can be located within touch range or operating range of the driver or user. On this FCD or the display and control unit ABE, the user can perform all required operating steps (e.g. select and play a film) and then move (cast) the entire application software AS to a display or screen or the display unit CID outside of touch range or operating range and consume it on this display. If the user wishes to make further entries in the third-party application software AS, they move the application software AS back to the FCD or the display and control unit ABE.If the application software AS has been moved to the display device CID, the user may need a way to retrieve the application software AS on the FCD or the display and control device ABE. One option could be a so-called generic proxy app or proxy software PS, which takes the place of the moved third-party application software AS on the FCD or the display and control device ABE. This proxy app or proxy software PS can include generic control options (volume, play, pause, and / or skip) as well as other input methods or tools such as a touchpad and / or cursor control to enable simple operation of the third-party application software AS even without retrieving it in a "cast" state.

[0059] To clarify: This idea cannot be a "mirroring" approach, i.e., duplicating the app output data on two displays. Rather, unlike mirroring, the app or application software AS can be closed on a first display or the display and control device ABE and opened on a second display or display device CID. And if the app or application software AS is closed on the first display or the display and control device ABE, the proxy app or proxy software PS can appear.

[0060] The advantage of this mechanism is that it can enable vehicle concepts with small operating displays within the user's touch range or operating range, and possibly larger displays or the CID display device outside the user's touch range or operating range in the vehicle. This allows a customer or user to control and consume infotainment content even when outside of a driving or driver position – including third-party application software that may not be programmed for such a scenario.

[0061] Overall, the examples show how a method for operating application software, proxy software for controlling the application software, a processor system and a motor vehicle with a processor system can be provided.

Claims

PATENT CLAIMS: 1 . Method for operating application software (AS) in a motor vehicle (KFZ), wherein the motor vehicle (KFZ) has a processor system and a display device (CID) and a combined display and operating device (ABE), characterized in that the processor system executes the application software (AS) and the application software (AS) generates output data (AD) during execution, which are to be displayed in a display area, and the processor system operates proxy software (PS) which is different from the application software (AS), which receives the output data (AD) from the application software (AS) and, depending on user input data (NED) which are generated by the display and operating device (ABE) in response to a user input, selects between the display device (CID) and the display and operating device (ABE) as a display area and forwards the output data (AD) to the selected display area.

2. The method according to claim 1, wherein the proxy software has the following interfaces: • a user interface (NS), via which user input data (NED) for operating the application software (AS) is received from the proxy software PS from the display and operating device (ABE), • an input output interface (IAS), via which a first part of the user input data (NED1 ) that goes beyond the selection of the display area is transferred from the proxy software (PS) to the application software (AS), • an output data reception interface (ADES), via which the proxy software (PS) receives output data (AD) from the application software (AS), wherein the output data (AD) is transmitted in dependence on a second part of the user input data (NED2) in a format defined by the proxy software (PS) comprising an allocation unit (ZE) is provided with a display instruction (AW) which identifies the display area, and • an output data transfer interface (ADUS), via which the output data (AD) together with the display instruction (AW) are transferred to an operating system (OS) included in the processor system, which transfers the output data (AD) for output depending on the display instruction (AW) to the display area described by the display instruction (AW).

3. Method according to one of the preceding claims, wherein the application software (AS) to be executed is downloaded from a server device of a third-party provider to the processor system and installed there.

4. Method according to one of the preceding claims, wherein the proxy software (PS) queries at least one operating function for operating the application software (AS) from metadata of the application software (AS) and makes it available on the display and operating device (ABE) in a selectable manner, wherein for this purpose operating elements are provided by the proxy software (PS) independently of the application software (AS).

5. The method according to claim 4, wherein the proxy software (PS) provides as the at least one operating function a playback control function and / or a volume control function and / or a voice control function and / or a cursor control function.

6. Method according to one of the preceding claims, wherein the display and operating device (ABE) has a touch-sensitive operating surface for receiving a user input, wherein User Input Data (NED) is generated in response to the user input.

7. Method according to one of the preceding claims, wherein the display and operating device (ABE) is provided on at least one armrest of a seat of the motor vehicle (KFZ).

8. Proxy software (PS) for coupling an application software (AS) with an operating system (OS) comprised by a processor system, wherein the proxy software (PS) comprises instructions which, when executed by the processor system, are configured to cause the processor system to execute the method steps relating to the proxy software (PS) according to one of the preceding claims.

9. Processor system, comprising a processor circuit for executing application software (AS), a digital data memory for executable storage of application software (AS) and an operating system (OS) and proxy software (OS), a display device (CID) and a combined display and operating device (ABE) and a transmission unit, designed for wireless data transmission with a server device of a third-party provider for downloading application software (AS).

10. Motor vehicle (KFZ) comprising a processor system according to claim 9.

Images