Multimedia system with optimized performance

Abstract

The present invention relates to a multimedia system in a vehicle, the system being capable of running more than one user application at the same time, the system comprising: at least one processor; and a computer-readable storage medium, the computer-readable storage medium comprising a non-transitory computer-readable storage medium storing computer-readable instructions, the system being configured to execute the computer-readable instructions by the processor, wherein the computer-readable instructions comprise a detection module and a display controller, the detection module being configured to detect a condition of the system, and the display controller being configured to determine which user application is the foreground application being displayed on the display and present the display content, wherein if the detection module detects that the condition exceeds a threshold, the display controller is configured to present each frame of the display content at an attenuated rate, wherein if the detection module detects that the condition is within a threshold, the display controller is configured to present each frame of the display content without any attenuation. The present invention also relates to a related method for optimizing the performance of a multimedia system in a vehicle being capable of running more than one user application at the same time.

Description

Technical Field

[0001] The present invention relates generally to multimedia systems in vehicles, and in particular to optimizing the performance of multimedia systems. Background Art

[0002] With the widespread use of high-definition displays in in-vehicle multimedia entertainment systems, the bus bandwidth connected to the memory of such systems has become critical when considering performance. This is because high bandwidth is required to render content to a high-definition display when reading and writing to memory. In particular, the simultaneous use of applications that require such high memory bus bandwidth often results in a 100% bus load, causing slow system response, which is manifested in, for example, slow execution of user input, sharp drops in frame rate, and delays in rendering to the display. Bus load is the percentage of total bus traffic passing through the bus and includes the overhead of bus arbitration. Bus utilization is the percentage of total bus traffic that is useful - that is, useful data passing through the bus, such as data entering the application rather than unneeded data that will be refreshed. When the memory bus has a high bus load percentage, such as 90% or more, the bus utilization may be only a small fraction of the bus load percentage, such as less than 60%, depending on the system. In other words, although the memory bus is occupied, for example, 90% of the time, the actual useful data or throughput passing through the memory bus may be less than 60%. In addition, as systems become more complex, the number of bus masters increases, which increases the bus load and further reduces the bus utilization, for example, by a further 30%. Therefore, it is imperative to reduce the load on the memory bus.

[0003] An existing method to reduce the load on the memory bus is to render content to the display at a fixed, artificially low rate. However, even if the system is able to provide a faster response, the user may perceive such a system as having poor performance. Therefore, such systems impair the user experience.

[0004] Another existing approach is to provide different rates at which updated data refreshes displayed content from different information sources on the Internet. That is, content that is considered to have a lower priority can be refreshed at a lower rate, while content that is considered to have a higher priority can be refreshed at a higher rate. However, optimizing the bandwidth required to obtain information through the Internet does not necessarily optimize the bandwidth requirements on the memory bus. In particular, after obtaining information from the Internet, the processing of such prior art systems may not optimize the utilization of system resources, such as memory and CPU load, and therefore may not optimize the load on the memory bus.

[0005] Therefore, there is a need for new or improved systems and methods that overcome or at least ameliorate one or more of the above-mentioned disadvantages. Summary of the invention

[0006] Therefore, one object is to provide a multimedia system and related methods to solve the above problems. In particular, one object is to provide a multimedia system and related methods that optimize performance and reduce latency regardless of the number of user applications running on the system. Another object is to provide a multimedia system and related methods that can flexibly manage changing bandwidth requirements on a memory bus.

[0007] To achieve these and other purposes, in a first aspect, a multimedia system in a vehicle is provided, the system being capable of running more than one user application simultaneously, the system comprising: at least one processor; and a computer-readable storage medium comprising a non-transitory computer-readable storage medium storing computer-readable instructions, the system being configured to execute the computer-readable instructions through the processor, wherein the computer-readable instructions comprise a detection module and a display controller, the detection module being configured to detect a condition of the system, and the display controller being configured to determine which user application is a foreground application being displayed on the display and present the display content, wherein if the detection module detects that the condition exceeds a threshold, the display controller is configured to present each frame of the display content at a decay rate, wherein if the detection module detects that the condition is within the threshold, the display controller is configured to present each frame of the display content without any decay.

[0008] The disclosed system utilizes conditions suitable for determining whether the frame rate should be attenuated. For example, a suitable condition may be a condition that reflects the processing load of the system. In some embodiments, the detection module may be configured to detect the load of a bus of a computer-readable storage medium, referred to herein as a bus load or a load of a memory bus. In such embodiments, if the detection module detects that the memory bus load is busy, for example, when the bus load is above about 90%, the display controller may then be configured to present each frame of the display content at an attenuated rate. Conversely, if the detection module detects that the memory bus is relatively flexible in responding to future processing requests, for example, if the bus load is below about 60% or 70%, the display controller may be configured to present each frame of the display content without any attenuation.

[0009] In other embodiments, the detection module can be configured to detect the number of user applications running simultaneously on the system. In such embodiments, if the detection module detects that more than one user application is running on the system simultaneously, and the more than one user application thus causes the processing load on the system to increase, the display controller can then be configured to present each frame of the display content at a decaying rate. Conversely, if the detection module detects that there are no user applications running simultaneously, that is, the foreground application is the only user application running on the system, the display controller can be configured to present each frame of the display content without any decay. Advantageously, the disclosed system can respond flexibly according to the processing load of the system. The disclosed system can dynamically balance the load from more than one user application running simultaneously on the memory bus. This may be particularly advantageous when more than one user application running simultaneously each requires the high bandwidth of the memory bus and / or consumes the high memory bus bandwidth together.

[0010] Thus, the disclosed system can be designed to optimally manage its available resources. It has been found that the rendering of display content uses a large amount of bandwidth compared to other operations such as writing data from the Internet to cache, database processing, and general CPU calculations. Therefore, regardless of whether the system has a large amount of memory or high processing speed at its disposal, the present disclosure optimizes the rate of rendering operations, thereby reducing and / or optimizing the overall bandwidth through the memory bus.

[0011] The decay rate can be determined based on the bus load connected to the computer-readable storage medium, that is, the memory bus load. The load on the memory bus may be caused by the operation of one or more user applications. The memory bus load can be the total load caused by all user applications. In a direct use case where the overhead is negligible, the decay rate can be determined based on the memory bus bandwidth of the or each user application, for example, based on the amount of data required by the user application or each of more than one simultaneously running user applications. The displayed content and therefore the most obvious to the user can be controlled to be displayed at a suitable frame rate according to the use case, such as a frame refresh rate proportional to the amount of new content that needs to be displayed. For example, according to the design or requirements of the multimedia system, a user application that has been displayed with constantly changing content can decay at a lower rate than another user application with static display content. Another design concept may be that, because the content displayed by the user application changes faster than the user's reaction speed, such a user application may decay at a higher rate to reduce the bus load. The decay rate can be selected to minimize the discontinuous motion presented on the continuous frame, so that the user experience has a minimal lag. For example, the user application that usually runs on the vehicle multimedia system is a navigation application, which displays content that changes as the vehicle moves. Therefore, when the foreground application is a navigation application, it is advantageous that the decay rate be selected to minimize discontinuous frame changes that are noticeable to the user's eye. On the other hand, user applications that are not displayed or visible to the user, such as background user applications, may be decayed at a higher rate or refreshed at a lower rate, thereby reducing their contribution to the bandwidth through the memory bus.

[0012] The decay rate can be determined dynamically based on the memory bus load. The memory bus load can be measured periodically so that the decay rate can be dynamically updated or eliminated. The bus load measurement can be fed back to the display controller or user application to implement frame rate decay or eliminate frame rate decay. When multiple user applications are running simultaneously, each user application may be assigned a unique refresh rate decay. Whenever a user application is stopped or closed or a new user application is started, the memory bus load can be detected and / or measured and / or the decay rate determined. Advantageously, the decay rate can be customized for each scene. A fixed percentage reduction in frame rate can be implemented for the or each simultaneous user application. Alternatively, a percentage reduction in frame rate can be implemented for certain types of user applications, and another percentage reduction in frame rate can be implemented for other types of user applications.

[0013] As described above, the detection module can detect the number of user applications to determine whether refresh rate attenuation is required. The detection module can detect all or at least some of the user applications running on the system. The detected user applications can be those that consume high memory bus bandwidth or resource-intensive applications, such as navigation, video playback, voice recognition, and connection with mobile devices. In one example, the detection module can be configured to detect the number of user applications requesting audio output. Audio from different sources or overlapping audio outputs may be an indication of multiple user applications running simultaneously in a multimedia audio system. The detection module can manage user applications that output sound, for example, based on a priority table, so that audio can be appropriately output, for example, through necessary audio channels. Since audio output can be an indication of a resource-intensive user application in a multimedia system, such a detection module may be suitable for detecting the number of resource-intensive user applications.

[0014] The detection module can be a part or addition of the user application, such as a plug-in of the foreground application. Therefore, in one embodiment, the detection module can be an audio plug-in or an audio manager.

[0015] Other ways of determining whether refresh rate reduction is needed are also within the scope of the present disclosure. For example, processing load or memory load can be used to determine whether to present the displayed frame at a reduced rate.

[0016] On the other hand, a method for optimizing the performance of a multimedia system in a vehicle capable of running more than one user application simultaneously is provided, the method comprising: detecting a condition of the system through a detection module of the system; when the condition exceeds a threshold, presenting the content of each frame of the foreground application displayed on the display at a decaying rate through a display controller of the system; and when the condition is within the threshold, presenting each frame of the displayed content without decay through the display controller.

[0017] The method may further include: receiving, by the display controller, a decay rate based on a load of a bus connected to the computer-readable storage medium. The decay rate may be dynamically adjusted based on the load imposed on the memory bus by all operations and user applications running on the system.

[0018] In some embodiments, the method may include detecting, by a detection module, a load on a bus of a computer-readable storage medium of the system or a memory bus load. When it is detected that the bus of the computer-readable storage medium is busy, for example, when the bus load is higher than about 90%, the method may include presenting each frame at a decaying rate. Conversely, when it is detected that the bus to the computer-readable storage medium is able to respond to future processing requests, for example, if the bus load is lower than about 60% or 70%, the method may include presenting each frame without any decay.

[0019] In some embodiments, the method may include detecting, by a detection module, the number of user applications running simultaneously on the system. In order to determine whether refresh rate attenuation is required, the detection step may detect the number of user applications running on the system, for example, by detecting the number of user applications requesting audio output, since audio from different sources or superimposed audio output may be an indication of multiple user applications running simultaneously in a multimedia system. When it is detected that more than one user application is running simultaneously on the system, it may be considered that the memory bus load is busy, so the method may include presenting each frame at an attenuated rate. Conversely, when the foreground application is the only user application running on the system, the method may include presenting each frame without any attenuation. The detection module may be a plug-in to the system. The detection module may be part of or an addition to a user application, such as a plug-in to the foreground application that performs refresh rate attenuation.

[0020] The disclosed method may help optimize performance when a vehicle multimedia system is used to run a navigation application.

[0021] Advantageously, the disclosed method helps balance the load on the memory bus by reducing the refresh rate when multiple user applications are running simultaneously or when certain conditions such as memory bus load exceed a threshold, but presents content at the original frame rate when there is no shortage of computing resources, such as when the foreground application is the only user application running on the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A hardware block diagram of an exemplary multimedia system 100 is shown.

[0023] In the drawings, the same reference numerals denote the same components. DETAILED DESCRIPTION

[0024] Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. The detailed description of the present invention will be provided for the purpose of explaining the principles of the present invention and its practical application, so that those skilled in the art can understand the present invention with respect to various exemplary embodiments and various modifications suitable for the intended specific use. The detailed description is not intended to be exhaustive or to limit the present invention to the specific embodiments disclosed. Modifications and equivalents will be apparent to those skilled in the art and are included within the spirit and scope of the appended claims.

[0025] A multimedia system in a vehicle is provided. The system is capable of running more than one user application at the same time. The system includes: at least one processor; and a computer-readable storage medium, which includes a non-transitory computer-readable storage medium having computer-readable instructions stored on the storage medium, and the system is configured to be executed by the processor. The computer-readable instructions include a detection module and a display controller. The detection module is configured to detect a condition of the system, and the display controller is configured to determine which user application is the foreground application displayed on the display and present the display content. If the detection module detects that the condition exceeds a threshold, the display controller is configured to present each frame of the display content at a decay rate. If the detection module detects that the condition is within a threshold, the display controller is configured to present each frame of the display content without any decay.

[0026] A method for optimizing the performance of a multimedia system in a vehicle capable of running more than one user application simultaneously is also provided. The method includes detecting a condition of the system by a detection module of the system. When the condition exceeds a threshold, the method includes presenting, by a display controller of the system, the content of each frame of the foreground application displayed on the display at a decaying rate. When the condition is within the threshold, the method includes presenting, by the display controller, each frame of the displayed content without any decay.

[0027] Multimedia systems are systems used to play different types of digital media such as text, audio, video, computer graphics, photos and animations. In the context of vehicle multimedia systems, the most common applications include but are not limited to navigation, audio, radio, auxiliary media playback, video playback, voice recognition, such as through Apple CarPlay TM , Baidu CarLife, Android Auto, etc., and connections with mobile devices. Other applications that the in-vehicle multimedia system may have include hands-free voice calls and reading messages and emails aloud. The vehicle multimedia system may also be referred to as or may include a head unit or an infotainment unit. The user of the multimedia system may be the driver or owner or a passenger or occupant of the vehicle.

[0028] To play different types of media, the multimedia system may be, for example, an embedded system or a general-purpose computing device. Figure 1 1 shows a hardware block diagram of an exemplary multimedia system 100. The multimedia system 100 may include one or more processing devices 10, such as a system on a chip or a microcontroller, to implement the functions of the multimedia system. The multimedia system 100 generally includes a plurality of processors and other components connected to a computer-readable storage medium or memory module (generally 118) via a communication bus (generally 30). Although the memory module 118 is generally Figure 110, but the memory module may alternatively or additionally be included in the processing device. The processor may include one or more central processing units, such as ARM cores, and other processors located on the processing device, such as controllers, 2D and / or 3D graphics processors, video processors, image processors, etc. Figure 1 In the embodiment, the processing device 10 includes a central processing unit 120, a video processor 108, an image processor 110, and a graphics processor 112, all of which communicate with a memory module 118. Figure 1 One central processing unit 120 is shown, but the multimedia system may include more than one central processing unit. For example, a multimedia system including two central processing units is generally considered to be a dual-core system, while a multimedia system including four central processing units is generally considered to be a quad-core system. Typically, the central processing unit 120 executes instructions to initialize and run the video processor 108, the image processor 110, and / or the graphics processor 112. Once the central processing unit 120 instructs the processors 108, 110, and / or 112, the processors 108, 110, and / or 112 execute the instructions independently. That is, the processors 120, 108, 110, and 112 can individually write data to and read data from memory modules such as cache memory, RAM (e.g., DDR RAM), and flash memory (e.g., NAND flash memory and NOR flash memory). The computer-readable storage medium or memory module 118 may include temporary and non-temporary computer-readable storage media. For example, the central processing unit 120 can read computer-readable instructions from a non-temporary memory module and write data to a temporary memory module in response to the instructions. In response to the instructions, the central processing unit 120 may write instructions for the video processor 108, the image processor 110, and / or the graphics processor 112 to read and execute. One, some, or all of the processors may be configured to or participate in presenting the content of each user application or running user applications simultaneously, and the memory module connected to the processor may be configured to store the presented content. The memory module may also be used by other peripheral devices such as USB devices, personal mobile devices, DVD or Blu-ray TMCD drive access, although the drivers for peripheral devices usually write to the memory module through direct memory access to improve efficiency. Direct memory access can be a potential memory bus master, which will cause the memory bus load to increase. The multimedia system usually also includes a device for receiving user input, such as a user interface or human-machine interface, a knob, an actuator on a steering wheel, a mouse, a joystick, a touch screen display, a microphone, a tactile actuator and / or other input mechanisms such as voice recognition, gesture recognition, etc. The multimedia system usually also includes a display screen or display for outputting multimedia or any information. The input device can communicate with a display controller, which receives user input from the input device and directs the execution of the user input to, for example, a processor or a processor group, which processes the input to provide output according to the request. The peripheral device, the device for receiving user input, the device for outputting multimedia or information (such as a display) and other peripheral devices of the multimedia system are generally 116.

[0029] In an embodiment, the vehicle multimedia system disclosed herein may be an embedded system. Embedded systems are typically designed to perform one or several dedicated functions, such as the functions disclosed herein. The embedded system may not be configured to include functions that are unrelated to or separate from the dedicated functions. An embedded system may read software or computer-readable instructions from a built-in chip, rather than from a disk as in general-purpose computer systems such as desktop PCs. Therefore, the system resources of an embedded system for running multiple user applications simultaneously are limited. An embedded system may have real-time computing constraints. Although the system may be designed to run more than one user application simultaneously, the system resources of the embedded system may not be sufficient to support the running user applications and / or any new actions. Therefore, in some embodiments, the present disclosure advantageously provides a solution for optimizing the performance of an embedded system.

[0030] The multimedia system may include a software architecture. The software architecture may generally include computer-readable instructions stored on a non-transitory computer-readable storage medium to enable the multimedia system to perform its functions. As known to those skilled in the art, software or code or instructions may be implemented in various layers of a computing system. The lowest layer of code (physical or hardware layer) transmits and receives raw bits from the hardware to serve the layers above it. The highest layer of code interacts directly with the end user. Computer-readable instructions may include code across several layers. The software architecture may include, but is not limited to, an operating system for the multimedia system, application software for each user application, and other software required to interface between the operating system and the application software or to perform other functions required by the multimedia system.

[0031] The user application of the multimedia system is a software program or a set of instructions that works with the hardware components of the system to allow the user to use and experience multimedia. The operations involved in running the multimedia application may include operations such as audio codec, video codec, content presentation, volume change, color space conversion, overlay, etc. In a specific example, the operations involved in running the navigation application may include navigation map presentation, generating audio output for turn-by-turn instructions, audio volume change, overlay, etc. The user application can be manipulated by user actions or user inputs, for example, the user can start the application, switch multimedia sources, such as switching to a video source or an audio source, switch the application interface, or manipulate the application interface, such as scrolling to display another map portion or zooming a map portion. These operations and actions contribute to the bandwidth requirements of the memory bus.

[0032] While the present invention is applicable to any application running simultaneously on a multimedia system, the disclosed invention advantageously assists in configuring a multimedia system to simultaneously run a navigation application and at least one other user application, particularly at least one other resource-intensive user application, such as high-definition video playback or voice recognition.

[0033] Operations and actions that contribute to bandwidth requirements specific to navigation applications on a memory bus are described. A navigation engine typically includes a stored map database and route calculation and / or route optimization capabilities. For the portion of the map that is displayed, the navigation engine typically needs to render roads, buildings, and other details on demand. When the navigation engine is incorporated into a multimedia system, the rendering of the map utilizes the system's graphics processing unit. The rendering of objects such as roads and buildings can be in 2D or 3D form. Thus, a 2D graphics processing unit can use the memory bus to render 2D graphics, while a 3D graphics processing unit can use the memory bus to render 3D graphics. There will therefore be two bus masters, requiring the bus to arbitrate and resulting in an increase in bus load. The current position of the vehicle, such as obtained from the vehicle's GPS system or inertial sensors, can be superimposed on the rendered surface (typically utilizing the system's image processing unit, thereby adding another bus master) to construct a frame for the final display. Turn-by-turn instructions can also be rendered or superimposed on the frame. Alternatively, route guidance can be output in an audio manner, where the audio frame needs to be synchronized with the navigation frame. The rate at which the navigation engine presents each navigation frame is designed to be high enough, such as 15fps or higher, so that frame transitions are smooth when the user scrolls to display another map portion or zooms a map portion (typically utilizing the system's CPU or ARM core).

[0034] When the processor is instructed to perform any action, the processor can control the bus to the memory module, thereby becoming a memory bus master. For example, presenting and displaying the navigation map of the above-mentioned navigation application involves a graphics processing unit, an ARM core, and an image processing unit as a memory bus master. In another example, playing high-definition video in a media file in a USB peripheral device involves a video processing unit, a graphics processing unit, and an image processing unit, so these processors become memory bus masters. Therefore, any user action, such as selecting video playback, requesting navigation route guidance, changing audio source, changing volume, and switching screens, requires one or more bus masters.

[0035] The bus or buses to memory, i.e., the memory bus, should have sufficient bandwidth to transfer data to and from memory, even data for high bandwidth use cases or user applications or resource intensive operations (e.g., user actions). However, when incorporated into a multimedia system, which may not be powerful enough to perform the designed functions of the navigation engine simultaneously with other increasingly complex user applications or operations, the bandwidth of the memory bus may be overwhelmed. Due to the higher priority of graphics and video processing, the system may not respond to user actions or other non-graphics, non-video related actions. The disclosed invention is intended to alleviate such situations where the memory bus does not have sufficient bandwidth to perform all actions and any new actions simultaneously, especially when the system is multitasking and running multiple user applications simultaneously. In general, the capacity of the bus determines how quickly the components of the computer system can communicate with each other. Therefore, the speed of the connection to the memory (e.g., RAM) directly controls how fast the computer system can access instructions and data.

[0036] It has been found that the rendering of content for display on a full display screen utilizes a large amount of bandwidth compared to other operations. This is because each pixel of the display requires a certain amount of data, and since a typical display has thousands of pixels, the amount of data required to render content for a full display screen becomes very large. In one example, to display high-definition video, a common high-definition video display size uses 720 lines of 1280 pixels per line (1280×720). Each color pixel typically uses 32 bits / pixel, or 4 bytes, and a typical high-definition display has a refresh rate of 30 frames per second. Therefore, with one read cycle and one write cycle, the bandwidth required to render 30 video frames to such a high-definition video display in one second is:

[0037] Bandwidth = 1280 pixels × 720 pixels × 4 bytes (true color pixel size) × 30fps × 2 (read and write cycles) = 220MB / second

[0038] The above example shows that user applications with a large number of frames per second, such as 30fps video playback, may be resource-intensive, especially when more than one such resource-intensive application is running at the same time. As used herein, the term "refresh rate" or "frame rate" refers to the number of frames presented for display per second or the rate at which content is presented for display, which is different from and opposite to the refresh rate of the display screen, thereby displaying the content that has been presented or the frames that have been constructed at this refresh rate of the display screen. Therefore, attenuating the rate at which each frame is presented means reducing the number of frames presented per second. When such resource-intensive applications are in the foreground in a multitasking scenario, refresh rate attenuation can be selected so that the user experience is not greatly impaired. For example, for user applications where the displayed content does not change so quickly, the frame rate can be designed or set to a lower rate, such as 10fps, or the frame can be refreshed only when the displayed content changes. When such applications are in the foreground in a multitasking scenario, especially when resource-intensive background applications are involved, the refresh rate attenuation of the foreground application can be reduced as much as possible without affecting the user experience, or, for example, the frame can be refreshed at a specific rate only when the displayed content changes.

[0039] Background applications can also be modified to reduce their bandwidth contribution. In some common examples, background applications may be stopped / paused or have their frame rate decayed or the window that defines it reduced. As long as the user experience is not impaired, the foreground application can be modified. The rate at which each frame of the displayed content is presented is reduced to a reasonable frame rate so that the user experience is not impaired, but other applications or operations such as user actions can act responsively. The decay refresh rate is appropriately determined to advantageously compensate for the increase in memory bus load in multitasking scenarios. For example, when the navigation application is still a foreground application, such as when at least a portion of the displayed navigation application is displayed or when the navigation application is the only displayed user application, the displayed navigation map frame is presented at a reasonably decayed navigation map refresh rate to save certain bandwidth without lag when user actions need to be processed responsively. For foreground user applications whose display content does not change much, the frame rate can be reduced from its original frame rate, such as 10fps, to a lower frame rate, or the frame rate can be kept at its original low frame rate, but the background application is modified to save bandwidth. The frame rate can have a predetermined lower limit, such as 10fps, 8fps or 5fps. Thus, depending on the characteristics of the user application, such as changing display content or inherently low bandwidth, the foreground and / or background applications may be modified accordingly.

[0040] The detection module can detect conditions such as multi-tasking scenarios where bus loading may compromise system performance. For example, the detection can be based on the number of user applications running on the system.

[0041] The detection module can be part of the user application or it can be additional code, such as a plug-in to the foreground application. Thus, in the case where the foreground application is a navigation application, the detection module can be a plug-in to the navigation engine. If the navigation engine is provided by a third party or cannot be modified in other aspects, providing the detection module as a plug-in may be an effective way to add the functionality of identifying whether any other application is activated in addition to the navigation application.

[0042] In the case where the detection module detects the number of user applications running on the system, the detection module can be configured to detect the number of user applications requesting audio output. Therefore, the detection step can include detecting the number of user applications requesting audio output running on the system. In one example, the detection module is a plug-in of the system, in particular an audio plug-in of a navigation engine, to manage any user application requesting audio output so that appropriate channels can be allocated to output audio. For an in-vehicle multimedia system, resource-intensive or memory-intensive user applications typically involve audio output. Therefore, detecting user applications requesting audio output may be an effective way to determine the number of user applications running on the system. For example, when video playback and voice recognition are running simultaneously, the detection module can detect the audio output from the video playback and voice recognition applications, and then the implementation of refresh rate attenuation can be triggered. The detection module is able to identify whether a user application that may be running on the multimedia system is enabled or disabled, started or stopped.

[0043] Alternatively, other ways of detecting multitasking scenarios and deciding whether refresh rate attenuation is needed are also within the scope of the present disclosure. For example, the display controller can identify the number of faces built for each running user application, or the system can identify the processing load or memory load used by each running user application. In an example where the load on the memory bus is detected to be higher than 70%, 75%, 80%, 85%, or 90%, the display controller can be configured to present each frame of the display content at a decaying rate. The detection of the memory bus load can be the total load of all memory buses to the associated memory or storage medium (e.g., DDR memory), such as internal buses (connections within a chip or component or unit) and / or external buses (connections between chips, components, or units). In another example, the amount of processing or processing load required to present the content of the detected user application can be used to determine whether a refresh rate attenuation is needed. For example, the amount of processing required to present a navigation map display based on the density of points of interest that need to be displayed can be calculated to determine whether a refresh rate attenuation is needed.

[0044] The display controller can be part of the user application software, typically as part of one of the top two layers of code or across the top two layers of code. The display controller can be a protocol that specifies the communication between a display server that controls the display of rendered content and its client (e.g., a user application). Alternatively, the display controller can include a rendering module to control the presentation of data and the display of rendered data. The display controller can include the functionality of a display server and / or the functionality of combining faces from each user application to create a graphical user interface that is output to a display. The display controller can control the display of content, transmit communications between a processor and an input device, and / or output images to a display. By virtue of its role in rendering the display of content, the display controller is able to determine which user application is the foreground application being displayed on the display.

[0045] Upon detecting multiple concurrent user applications or other detection methods, the detection module may notify the user application or any part of the software architecture that controls or implements the refresh rate. The computer-readable instructions or logic that trigger the enabling or disabling of the refresh rate attenuation may be part of the user application, typically as part of one of the top layers of code or across the top layers of code. The logic to implement the refresh rate attenuation may be implemented in one of the middle layers of the foreground application. The logic may calculate or determine a decay rate suitable for the multimedia system. The logic may include a determination unit to determine a suitable decay rate. Alternatively, a suitable predetermined decay rate may be obtained from a non-transitory computer-readable storage medium. The logic may then instruct the display controller to present each frame of the display content at a decayed rate.

[0046] As described above, the detection module can notify the logic that implements the refresh rate attenuation of the foreground application or display content and calculate, obtain or otherwise determine the appropriate attenuation rate. The display controller can then be notified of the determined attenuation rate and the display content can be presented at the attenuation rate. The display controller can be notified through information exchanged between the user application and the display server including the determined refresh rate attenuation value.

[0047] With the appropriate decay rates obtained from memory, the system can be designed to have a predetermined decay refresh rate for each simultaneous use case. For example, a decay refresh rate value is selected that achieves a target reduction in bus load or a target bus load. This is possible because the number of user applications is fixed in a typical vehicle multimedia system. Therefore, the matrix for each simultaneous scenario can be input into a non-transitory computer readable storage medium.

[0048] The refresh rate can be attenuated according to the number of user applications running simultaneously. In one example, the refresh rate can be attenuated in proportion to the number of user applications running simultaneously. For example, if two applications are running simultaneously, the design refresh rate of each application can be halved; if three applications are running simultaneously, the refresh rate of each application can be attenuated to one-third of its design refresh rate; in another example, the refresh rate can be attenuated based on the total load on the memory bus. If the total load on the memory bus exceeds a threshold, the display controller can receive a decay rate so that all user applications presenting content are presented at the same rate. Alternatively, when the critical bus load is exceeded, the decay rate may vary from user application to user application and from use case to use case. Taking the foreground navigation application as an example, the frame rate of the map of the straight road can be attenuated more aggressively if necessary compared to the frame rate of the map of the detour. When the detection module detects that the threshold is crossed, the decay rate can be adjusted accordingly. Therefore, the reduction in the refresh rate can be advantageously implemented, for example, according to the number of user applications competing for the memory bus bandwidth when needed. The display controller can receive a dynamically adjustable decay rate based on the memory bus load. Therefore, the reduction in refresh rate can be determined dynamically after each change in conditions, such as a change in the number of user applications.

[0049] When the condition is within the threshold, such as when the foreground application is detected as the only user application running on the system, or when the detection module detects that no other user applications are enabled, the detection module notifies the logic that implements the refresh rate attenuation to remove any attenuation rate or refresh rate cap. The display controller is then notified of the attenuation rate (or lack of attenuation rate) and renders the display content according to the provided refresh rate. Therefore, when the foreground application is running alone, the originally designed refresh rate is advantageously adopted.

Claims

1. An in-vehicle multimedia system, the system being capable of running multiple user applications simultaneously, the system comprising: at least one processor; and a computer-readable storage medium, the computer-readable storage medium comprising a non-transitory computer-readable storage medium storing computer-readable instructions, The system is configured to execute the computer readable instructions via the at least one processor, The system further comprises a detection module and a display controller, wherein the detection module is configured to detect a status of the system, and the display controller is configured to determine a foreground application being displayed on the display and present display content. It is characterized in that In response to the detection module detecting that more than one user application is running on the system simultaneously, the display controller is configured to present each frame of the display content at a decaying rate, and, In response to the detection module detecting that only one user application is running on the system, the display controller is configured to present each frame of the display content without any attenuation.

2. The system according to claim 1, wherein: The detection module is a plug-in of the foreground application.

3. The system according to claim 1, wherein: The foreground application is a navigation application.

4. A method for optimizing the performance of a multimedia system in a vehicle capable of running more than one user application simultaneously, the method comprising: executing, by at least one processor of the multimedia system, computer-readable instructions stored in a computer-readable storage medium, the computer-readable storage medium including a non-transitory computer-readable storage medium, The method further comprises: detecting a condition of the system through a detection module of the multimedia system, determining a foreground application being displayed on a display and presenting display content through a display controller of the multimedia system, each of the at least one processor having a computer-readable storage medium and computer-readable instructions stored thereon; It is characterized in that In response to the detection module detecting that more than one user application is running on the system simultaneously, presenting each frame of display content at a decaying rate by the display controller, and, In response to the detection module detecting that only one user application is running on the system, each frame of the display content is presented by the display controller without any degradation.

5. The method according to claim 4, wherein: The detection module is a plug-in to the system.

6. The method according to claim 4, wherein: The foreground application is a navigation application.

Images