Display method and electronic device

Abstract

The embodiment of the present application discloses a display method and an electronic device, which relates to the field of electronic devices, and can prevent the electronic device from displaying a white image for a long time when the speed of loading multimedia resources in the electronic device is slow, thereby improving the user experience. The specific scheme is: when the electronic device runs a first application, the first image data of the first application is obtained; based on the first image data, the loading state of the first image data is determined; when the loading state of the first image data is a non-slow loading state, the first image data is loaded and displayed based on the first frequency; when the loading state of the first image data is a slow loading state, the frequency of the CPU is adjusted to a second frequency; the second frequency is greater than the first frequency; the second image data of the first application is obtained; the second image data is the image data of the next frame of the first image data; based on the second frequency, the second image data is loaded and displayed.

Description

Technical Field

[0001] The present application relates to the field of electronic devices, and in particular to a display method and electronic devices. Background Art

[0002] With the continuous development of technology of electronic devices such as mobile phones, more and more electronic devices are able to load and display multimedia resources corresponding to different applications, such as images and texts corresponding to the application. When an electronic device displays a multimedia resource corresponding to an application, the electronic device can first load the multimedia resource. When the multimedia resource is loaded successfully, the electronic device can display the multimedia resource. When the multimedia resource is not loaded successfully, the electronic device usually displays a white image instead of the multimedia resource.

[0003] However, due to the influence of network conditions, the amount of multimedia resources and the performance of electronic devices, when electronic devices load multimedia resources, the loading speed is usually slow. This causes the electronic device to display a white image for a long time, and the user cannot know the multimedia resources corresponding to the application during this long time, which affects the user's experience. Summary of the invention

[0004] The embodiments of the present application provide a display method and an electronic device, which can prevent the electronic device from displaying a white image for a long time when the electronic device loads multimedia resources slowly, thereby improving the user experience.

[0005] In order to achieve the above purpose, the embodiment of the present application adopts the following technical solution:

[0006] In a first aspect, an embodiment of the present application provides a display method, which is applied to an electronic device. The display method may include: when the electronic device runs a first application, obtaining first image data of the first application; based on the first image data, determining a loading status of the first image data; the loading status includes a slow loading status and a non-slow loading status; when the loading status of the first image data is a non-slow loading status, loading and displaying the first image data based on a first frequency; the first frequency is the initial frequency of a central processing unit CPU of the electronic device; when the loading status of the first image data is a slow loading status, adjusting the frequency of the CPU to a second frequency; the second frequency is greater than the first frequency; obtaining second image data of the first application; the second image data is an image data of a frame after the first image data; and loading and displaying the second image data based on the second frequency.

[0007] Based on the method described in the first aspect, when the electronic device loads the first image data corresponding to the application, the electronic device can determine the loading state of the first image data based on the first image data. When the loading state of the first image data is a slow loading state, the electronic device can increase the frequency of the CPU, so that the electronic device can process subsequent image data based on the increased frequency of the CPU. Since the electronic device can increase the frequency of the CPU when the loading state of the first image data is a slow loading state, after the frequency of the CPU is increased, the speed at which the electronic device loads image data can be increased, thereby avoiding the electronic device displaying a white image for a long time when the speed at which the electronic device loads multimedia resources is slow, thereby improving the user experience.

[0008] In combination with the first aspect, in another possible implementation, the above-mentioned determining the loading status of the first image data based on the first image data may include: obtaining histogram data of the third image data; the third image data is the previous frame image data of the first image data; obtaining the histogram data of the first image data, and determining the loading status of the first image data based on the histogram data of the first image data and the histogram data of the third image data.

[0009] Based on this possible implementation, since the electronic device can accurately determine the loading status of the first image data based on the histogram data of the first image data and the histogram data of the third image data. And since the histogram detection uses the pixel value distribution information of the image to determine the loading status of the image, the histogram detection can determine whether the electronic device is loaded with a white image, thereby avoiding the electronic device from displaying a white image for a long time. In addition, by determining the loading status of the image data through the histogram data, the overhead of setting up a listener for each image is avoided, which can save the resources of the electronic device and improve the overall efficiency of the electronic device system.

[0010] In combination with the first aspect, in another possible implementation method, the above-mentioned determining the loading status of the first image data based on the histogram data of the first image data and the histogram data of the third image data may include: determining the image data change value and the duration of the image data change according to the histogram data of the first image data and the histogram data of the third image data; when the image data change value is less than the change threshold and the image data change duration is greater than the duration threshold, determining that the loading status of the first image data is a slow loading state.

[0011] Based on this possible implementation method, since the electronic device can determine the image data change value and the duration of the image data change based on the histogram data of the first image data and the histogram data of the third image data, it can accurately determine that the loading state of the first image data is a slow loading state based on the image data change value and the duration of the image data change.

[0012] In combination with the first aspect, in another possible implementation method, the above-mentioned non-slow loading state includes a normal loading state and a loading waiting state. The above-mentioned determination of the loading state of the first image data based on the histogram data of the first image data and the histogram data of the third image data also includes: when the image data change value is greater than the change threshold and the image data change duration is greater than the duration threshold, determining that the loading state of the first image data is a normal loading state; when the image data change value is less than the change threshold and the image data change duration is less than the duration threshold, determining that the loading state of the first image data is a loading waiting state.

[0013] Based on this possible implementation method, since the electronic device can determine the image data change value and the duration of the image data change based on the histogram data of the first image data and the histogram data of the third image data, it can accurately determine whether the loading state of the first image data is a normal loading state or a loading waiting state based on the image data change value and the duration of the image data change.

[0014] In combination with the first aspect, in another possible implementation, the display method may further include: when the loading state of the first image data is a normal loading state or a loading waiting state, loading and displaying the second image data based on the first frequency.

[0015] Based on this possible implementation, when the loading state of the first image data is a normal loading state or a loading waiting state, the electronic device does not need to increase the frequency, thereby avoiding high power consumption of the electronic device.

[0016] In combination with the first aspect, in another possible implementation, the above-mentioned acquisition of the histogram data of the first image data may include: determining a loading path for the first image data based on the first image data; the loading path includes a graphics processor GPU of the electronic device and a hardware synthesizer HWC of the electronic device; when the loading path is GPU, performing a histogram detection on the first image data through the GPU to obtain the histogram data of the first image data; when the loading path is HWC, obtaining the histogram data of the first image data from a histogram database of the electronic device through the HWC.

[0017] Based on this possible implementation, the electronic device can obtain the histogram data of the first image data through the GPU of the electronic device or the HWC of the electronic device, so as to determine the loading status of the first image data according to the histogram data of the first image data.

[0018] In combination with the first aspect, in another possible implementation method, the above-mentioned determining the loading path of the first image data based on the first image data may include: determining the loading path of the first image data based on at least one of the processing requirements of the first application for the first image data, attribute information of the first image data, and hardware capabilities of the electronic device.

[0019] Based on this possible implementation method, the electronic device can accurately determine the loading path of the first image data according to the processing requirements of the first application for the first image data, the attribute information of the first image data and the hardware capabilities of the electronic device, and thus can accurately determine the histogram data of the first image data according to the loading path of the first image data.

[0020] In combination with the first aspect, in another possible implementation, the above-mentioned display method may also include: obtaining fourth image data of the first application; the fourth image data is a frame of image data after the second image data; based on the fourth image data, determining the loading status of the fourth image data; when the loading status of the fourth image data is a non-slow loading state, adjusting the frequency of the CPU from the second frequency to the first frequency; obtaining fifth image data of the first application; the fifth image data is a frame of image data after the fourth image data; based on the first frequency, loading and displaying the fifth image data.

[0021] Based on this possible implementation, when the loading state of the image data is not in a slow loading state, the electronic device can adjust the frequency of the CPU back to the initial frequency, thereby avoiding high power consumption of the electronic device.

[0022] In combination with the first aspect, in another possible implementation method, the above-mentioned determining the loading status of the first image data based on the first image data may include: determining the scene of the first application based on the first image data; the scene of the first application includes a scene that requires histogram detection and a scene that does not require histogram detection; when the scene of the first application is a scene that requires histogram detection, determining the loading status of the first image data based on the first image data.

[0023] Based on this possible implementation method, the electronic device may need to determine the loading status of the first image data when the application scenario is a scenario that requires histogram detection, and does not need to determine the loading status of the first image data when the application scenario is a scenario that does not require histogram detection, thereby avoiding waste of resources of the electronic device.

[0024] In a second aspect, an embodiment of the present application provides a display device, which can be applied to an electronic device to implement the method in the first aspect. The function of the display device can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, for example, an acquisition module, a determination module, a loading module, a display module, and an adjustment module.

[0025] The acquisition module may be used to acquire first image data of the first application when the electronic device runs the first application.

[0026] The determination module can be used to determine the loading state of the first image data based on the first image data; the loading state includes a slow loading state and a non-slow loading state.

[0027] The loading module can be used to load the first image data based on the first frequency when the loading state of the first image data is not in the slow loading state; the first frequency is the initial frequency of the central processing unit CPU of the electronic device.

[0028] The display module can be used to display the first image data loaded by the loading module.

[0029] The adjustment module can be used to adjust the CPU frequency to a second frequency when the loading state of the first image data is a slow loading state; the second frequency is greater than the first frequency.

[0030] The acquisition module can also be used to acquire second image data of the first application; the second image data is a frame of image data subsequent to the first image data.

[0031] The loading module can also be used to load the second image data based on the second frequency.

[0032] The display module can also be used to display the second image data loaded by the loading module.

[0033] In a third aspect, a display device is provided, which has the function of implementing the method described in the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0034] In a fourth aspect, a display device is provided, comprising: a processor and a memory; the memory is used to store computer execution instructions, and when the display device is running, the processor executes the computer execution instructions stored in the memory to enable the display device to perform a display method as described in any one of the above-mentioned first aspects.

[0035] In a fifth aspect, a display device is provided, comprising: a processor; the processor is used to couple with a memory, and after reading instructions in the memory, execute the display method as described in any one of the above-mentioned first aspects according to the instructions.

[0036] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by an electronic device, the electronic device implements a display method as described in any one of the first aspect or possible implementations of the first aspect.

[0037] In a seventh aspect, an embodiment of the present application provides a computer program product, comprising a computer-readable code, which, when executed in an electronic device, enables the electronic device to implement a display method as described in any one of the first aspect or possible implementations of the first aspect.

[0038] In an eighth aspect, a device (for example, the device may be a chip system) is provided, the device including a processor for supporting an electronic device to implement the functions involved in the first aspect above. In one possible design, the device also includes a memory for storing program instructions and data necessary for the electronic device. When the device is a chip system, it may be composed of a chip, or may include a chip and other discrete devices.

[0039] It should be understood that the beneficial effects of the second to eighth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of a call scenario provided in an embodiment of the present application;

[0041] Figure 2 A schematic diagram of the principle of the display method provided in the embodiment of the present application;

[0042] Figure 3 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0043] Figure 4 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application;

[0044] Figure 5 A schematic diagram of a display method provided in an embodiment of the present application Figure 1 ;

[0045] Figure 6 A schematic diagram of a display interface of an electronic device provided in an embodiment of the present application;

[0046] Figure 7A schematic diagram of a display method provided in an embodiment of the present application Figure 2 ;

[0047] Figure 8 A schematic diagram of the structure of a display device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0049] The terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, unless otherwise specified, "plurality" means two or more.

[0050] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to be limiting of the present application. As used in the specification and appended claims of the present application, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to also include plural expressions, unless the context clearly indicates otherwise. It should also be understood that " / " means or, for example, A / B can mean A or B; "and / or" in the text is merely a description of the association relationship of associated characters, indicating that three relationships can exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0051] With the continuous development of technology of electronic devices such as mobile phones, more and more electronic devices are able to load and display multimedia resources corresponding to different applications, such as images, texts and other contents corresponding to different applications such as news applications and shopping applications.

[0052] When an electronic device displays a multimedia resource corresponding to an application, the electronic device may first load the multimedia resource. When the multimedia resource is loaded successfully, the electronic device may display the multimedia resource. When the multimedia resource is not loaded successfully, the electronic device usually displays a white image (or an image with incomplete content) instead of the multimedia resource.

[0053] For example, taking the electronic device as a mobile phone, the mobile phone can display multimedia resources corresponding to a news application (such as images and texts corresponding to the news application) as an example, the process of displaying multimedia resources corresponding to the news application on the mobile phone is schematically described. When the user opens the homepage of the news application on the mobile phone, the mobile phone can load the multimedia resources corresponding to the homepage of the news application, that is, the images and texts corresponding to the homepage of the news application. When the images and texts corresponding to the homepage of the news application are not loaded, such as Figure 1 As shown in (a) in FIG. 1 , the mobile phone may display a white image 01 at the position corresponding to the image and text to replace the image and text.

[0054] When the image and text are loaded, Figure 1 As shown in (b) in FIG. 1 , the mobile phone can display the image and text at the positions corresponding to the image and text, that is, Figure 1 02 in (b).

[0055] However, due to the network conditions (such as slow network speed), the data volume of multimedia resources (such as large image data volume) and the performance of electronic devices (such as low frequency of the central processing unit (CPU) of electronic devices), when electronic devices load multimedia resources, the loading speed will usually be slow. This causes the electronic device to display a white image (or an image with incomplete content) for a long time, and the user cannot know the multimedia resources corresponding to the application during this long time, resulting in a low user experience. It should be noted that the image data volume refers to the amount or size of the image data to be loaded and displayed by the electronic device, that is, the total amount of image data or the size of the resources occupied in the electronic device.

[0056] That is to say, electronic devices face a series of important challenges in the process of transmitting and loading streaming media resources (ie, multimedia resources), and these challenges have a significant impact on user experience that cannot be ignored.

[0057] For example, when a user browses a news application through an electronic device, the electronic device can load and display the multimedia resources corresponding to the news application. However, due to the slow loading speed of the multimedia resources, the multimedia resources corresponding to the news application, such as news images, may take a long time to load. In this case, the relevant area corresponding to the news image may be displayed as blank (such as a white image) or an image with incomplete content. This display condition may last for several seconds, greatly damaging the user's experience quality.

[0058] Since the electronic device system itself does not have the ability to perceive such problems, that is, the electronic device system itself cannot detect the loading status of the multimedia resources corresponding to the news application, such as whether the multimedia resources are loading normally or loading slowly. At the same time, due to security factors, it is not feasible to arbitrarily intercept the multimedia resources (such as images) in the user's application. This further leads to the inability to accurately identify the loading status of the multimedia resources corresponding to the application.

[0059] Due to the lack of determination of the loading status of multimedia resources corresponding to the application, it is impossible to obtain effective information from performance data tracking and the loading time of multimedia resources corresponding to the application, and thus it is impossible to effectively optimize the loading performance of multimedia resources.

[0060] Therefore, if the loading status of the multimedia resources corresponding to the application cannot be determined, the following problems will occur: 1. The user experience is reduced. When the multimedia resources corresponding to the application are loaded slowly, the display interface corresponding to the application may be missing or display errors, thereby affecting the user's overall experience of the application. This causes users to feel confused and disappointed, and even affects the user's trust and loyalty to the application. 2. The functions of the application are limited. If the multimedia resources corresponding to the application are loaded slowly, the functions related to the multimedia resources corresponding to the application may not be used normally, thereby affecting the functional integrity of the application. This will seriously affect the user's convenience and the practicality of the application, thereby weakening the attractiveness of the application. 3. User loss. Due to the impaired user experience, the user may lose interest in the application due to the slow loading of the multimedia resources corresponding to the application, and choose to leave the application, resulting in user loss. This poses a potential threat to the user retention rate and user growth of the application. 4. The evaluation of the application is reduced. The user's evaluation of the application in the application store or social media may deteriorate due to image loading problems, thereby affecting the word-of-mouth and reputation of the application.

[0061] Therefore, how to determine the loading status of multimedia resources corresponding to an application is crucial to improving the user experience and functional stability of the application.

[0062] In the related art, in order to optimize the loading performance of multimedia resources, the loading status of the multimedia resources corresponding to the application (such as the loading status of the images included in the multimedia resources) can be determined in the following three ways: the first way is to use a listener; the second way is to use an image loading library; the third way is to use WebView (visual component) to load images.

[0063] For the first method, using a listener, that is, when using an ImageView (image view component) to display images, you can set a listener (i.e. OnLoadCompleteListener) to listen for the image loading completion event. The listener will trigger a callback after the image is loaded, so that the developer can know the image loading status at the appropriate time.

[0064] For the second method, using an image loading library, if the application uses an image loading library (such as Glide or Picasso), these libraries usually provide a monitoring mechanism similar to the first method. Developers can use the callback function provided by the image loading library to detect whether the image loading is complete and make corresponding processing based on the loading status.

[0065] For the third method, use WebView to load images. When loading images in WebView, you can monitor the loading status of the images through the web view client (i.e. WebViewClient). WebViewClient provides corresponding callback functions so that developers can know the progress and results of image loading.

[0066] The above three related technologies all determine the loading status of the multimedia resources corresponding to the application (such as the loading status of the images included in the multimedia resources) by monitoring the loading status of the image and obtaining the callback of loading completion. In other words, the three related technologies all use the listener callback method, and need to register a listener for each image separately. Registering a listener for each image separately may increase memory overhead and processing burden.

[0067] However, when the multimedia resource is not loaded successfully, the electronic device usually displays a white image (or an image with incomplete content) instead of the multimedia resource, and the above three related technologies cannot detect whether the electronic device is loading a white image. Therefore, the above three related technologies cannot accurately detect the loading status of the multimedia resource corresponding to the application.

[0068] In addition, after the above three related technologies determine the loading status of the multimedia resources corresponding to the application (such as the loading status of the images included in the multimedia resources), the loading status results are returned to the application, and the application performs corresponding processing (such as the application adjusts the images to be loaded), thereby optimizing the loading performance of the multimedia resources. However, when the network condition is poor (such as the network speed is slow) or the performance of the electronic device (such as the processor frequency of the electronic device is low), the application performs corresponding processing (such as the application adjusts the images to be loaded), which cannot well optimize the loading performance of the multimedia resources, and thus cannot improve the user experience and functional stability of the application.

[0069] That is to say, although the above three related technologies can determine the loading status of multimedia resources corresponding to the application (such as the loading status of images included in the multimedia resources), they cannot accurately detect the loading status of multimedia resources corresponding to the application (such as the electronic device is loading a white image). Moreover, the above three related technologies cannot optimize the loading performance of multimedia resources when the network condition is poor (such as slow network speed) or the performance of the electronic device (such as low frequency of the processor of the electronic device). Therefore, the above three related technologies do not solve the above problem well, that is, how to determine the loading status of multimedia resources corresponding to the application, so as to improve the user experience and functional stability of the application.

[0070] In response to the above problems, an embodiment of the present application provides a display method, which is applied to an electronic device. Taking the example that the multimedia resource corresponding to the electronic device loading the application is an image, when the electronic device loads the first image corresponding to the application, the electronic device can first determine the scene of the application, and the CPU of the electronic device executes the loading and display of the first image based on the first frequency. When the electronic device determines that the scene of the application is a scene that requires histogram detection, the electronic device can perform a histogram detection on the first image corresponding to the application, thereby obtaining the histogram data corresponding to the first image. Afterwards, the electronic device can determine the loading status of the image corresponding to the application based on the histogram data corresponding to the first image. When the loading status is a slow loading status, the CPU of the electronic device can execute the loading and display of the second image based on a second frequency, the second frequency is greater than the first frequency, and the second image is an image after the first image.

[0071] Histogram detection is a histogram-based image processing technique used to determine whether an image has been loaded. During the image loading process, the pixel values ​​of the image will gradually change from the unloaded state to the loaded state. Histogram detection uses the pixel value distribution information of the image to make judgments.

[0072] A histogram is a statistical graph that represents the number of pixel values ​​in an image. During the image loading process, the pixel values ​​of the unloaded part will be concentrated in a certain range, while the pixel values ​​of the loaded part will be distributed in a wider range. By analyzing the changes in the histogram during the image loading process, it can be determined whether the image has been loaded. In other words, the histogram is a statistical information of the image brightness distribution. By analyzing the histogram, it can be more reliably determined whether the image has been fully loaded. This histogram-based detection method shows higher stability and accuracy under different network conditions and loading environments.

[0073] The commonly used histogram detection method is to determine the threshold value of loading completion by calculating the standard deviation of the histogram. When the image is loaded, the distribution of the histogram is relatively stable and the standard deviation is small; when the image is not loaded, the distribution of the histogram is relatively unstable and the standard deviation is large. According to the set threshold, if the standard deviation of the histogram is less than the threshold, it can be judged that the image is loaded.

[0074] The solution of the present application is to perform a histogram detection on an image, and based on the histogram data, it is possible to determine the loading state of the image, that is, to determine whether the loading state of the image is a slow loading state or a normal loading state. And because the histogram detection uses the pixel value distribution information of the image to judge the loading state of the image, it is possible to determine whether the electronic device is loading a white image through the histogram detection. Therefore, the loading state of the image can be accurately determined through the histogram detection. In addition, in the case where the loading state is a slow loading state, the solution of the present application is to increase the frequency of the CPU of the electronic device, and execute the loading of subsequent images based on the increased frequency, which can speed up the loading speed of subsequent images, thereby better optimizing the loading performance of multimedia resources.

[0075] In addition, compared with the method of determining the loading status of multimedia resources corresponding to the application by monitoring the loading status of the image and obtaining the callback of loading completion in the related art, it is necessary to register a listener for each image separately, which will increase memory overhead and processing burden. The solution of the present application performs histogram detection on the image and determines the loading status of the image based on the histogram data, thus avoiding the overhead caused by setting a listener for each image separately. It effectively saves the resources of the electronic device and improves the overall efficiency of the electronic device system.

[0076] Below, combined with the attached Figure 2 , taking the multimedia resource corresponding to the application loaded by the electronic device as an image as an example, the principle of the display method provided in the embodiment of the present application is schematically explained.

[0077] like Figure 2 As shown, when the electronic device loads the first image corresponding to the application, the electronic device may first determine the scene of the application, and the CPU of the electronic device executes the loading of the first image and displays the first image based on the first frequency.

[0078] The electronic device can determine the scene of the application according to the extended attribute of the layer corresponding to the first image, that is, determine whether the image corresponding to the application needs histogram detection.

[0079] When an electronic device determines that the scene of an application is a scene that requires histogram detection, the electronic device can first determine whether to perform a histogram detection on the first image corresponding to the application through a hardware composer (HWC) of the electronic device to obtain histogram data, or to perform a histogram detection on the first image corresponding to the application through a graphics processing unit (GPU) of the electronic device to obtain histogram data.

[0080] It should be noted that the electronic device can dynamically adjust whether the rendering and synthesis of the first image is performed by the GPU or the HWC according to the view of the first image (i.e., View), the hardware conditions of the electronic device, the requirements of the application, and other factors to ensure the best performance. When the rendering and synthesis of the first image is performed by the GPU, the electronic device performs a histogram detection on the first image through the GPU. When the rendering and synthesis of the first image is performed by the HWC, the electronic device performs a histogram detection on the first image through the HWC.

[0081] After obtaining the histogram data, the HWC of the electronic device or the GPU of the electronic device will report the histogram data to the display synthesis system (ie, surfaceflinger) of the electronic device. The display synthesis system of the electronic device determines the image loading state according to the histogram data.

[0082] When the image loading state is a slow loading state, the electronic device can determine a frequency increase strategy, that is, the electronic device can increase the frequency of the CPU, so that the CPU of the electronic device can execute the loading of the image after the first image (that is, the second image) based on the increased frequency (that is, the second frequency) and display the second image.

[0083] That is to say, the solution of this application adopts a method of histogram detection in the rendering synthesis stage. By statistically analyzing the output data of the GPU and the hardware synthesizer HWC, the change in the application histogram data is detected, so as to accurately identify the current application image loading status, and adjust the CPU scheduling strategy in real time to achieve fast completion of image loading.

[0084] The display method provided in the embodiment of the present application is described below.

[0085] The display method provided in the embodiment of the present application can be applied to the above-mentioned electronic device. In some embodiments, the above-mentioned electronic device can be a mobile phone, a tablet computer, a handheld computer, a personal computer (PC), a cellular phone, a personal digital assistant (PDA), etc., which can load and display multimedia resources corresponding to the application. The embodiment of the present application does not limit the specific form of the electronic device.

[0086] For example, taking the electronic device as a mobile phone, Figure 3 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown.

[0087] like Figure 3 As shown, the electronic device may include a processor 310, an external memory interface 320, an internal memory 321, a universal serial bus (USB) interface 330, a charging management module 340, a power management module 341, a battery 342, an antenna 1, an antenna 2, a mobile communication module 350, a wireless communication module 360, an audio module 370, a speaker 370A, a receiver 370B, a microphone 370C, an earphone interface 370D, a sensor module 380, a button 390, a motor 391, an indicator 392, a camera 393, a display screen 394, and a subscriber identification module (SIM) card interface 395, etc. Among them, the sensor module 380 may include a pressure sensor 380A, a gyroscope sensor 380B, an air pressure sensor 380C, a magnetic sensor 380D, an acceleration sensor 380E, a distance sensor 380F, a proximity light sensor 380G, a fingerprint sensor 380H, a temperature sensor 380J, a touch sensor 380K, an ambient light sensor 380L, a bone conduction sensor 380M, etc.

[0088] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0089] The processor 310 may include one or more processing units, for example, the processor 310 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0090] The controller can be the nerve center and command center of the electronic device. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0091] The processor 310 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 310 is a cache memory. The memory may store instructions or data that the processor 310 has just used or cyclically used. If the processor 310 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 310, and thus improves the efficiency of the system.

[0092] In some embodiments, the processor 310 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0093] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, mobile communication module 350, wireless communication module 360, modem processor and baseband processor.

[0094] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0095] The mobile communication module 350 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied in electronic devices. The mobile communication module 350 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 250 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 350 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 250 can be set in the processor 310. In some embodiments, at least some of the functional modules of the mobile communication module 350 can be set in the same device as at least some of the modules of the processor 310.

[0096] The wireless communication module 360 ​​can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to electronic devices. The wireless communication module 360 ​​can be one or more devices integrating at least one communication processing module. The wireless communication module 360 ​​receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 310. The wireless communication module 360 ​​can also receive the signal to be sent from the processor 310, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0097] In some embodiments, the antenna 1 of the electronic device is coupled to the mobile communication module 250, and the antenna 2 is coupled to the wireless communication module 360, so that the electronic device can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology.

[0098] The electronic device implements the display function through a GPU, a display screen 394, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 394 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 310 may include one or more GPUs that execute program instructions to generate or change display information.

[0099] The display screen 394 is used to display images, videos, etc. The display screen 394 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc.

[0100] In some embodiments, the electronic device may include 1 or N display screens 394 , where N is a positive integer greater than 1.

[0101] The electronic device can realize the shooting function through the ISP, the camera 393, the video codec, the GPU, the display screen 394 and the application processor, etc. In some embodiments, the electronic device can include 1 or N cameras 393, where N is a positive integer greater than 1.

[0102] The internal memory 321 can be used to store computer executable program codes, which include instructions. The processor 310 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 321. The internal memory 321 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data established during the use of the electronic device (such as audio data, a phone book, etc.), etc. In addition, the internal memory 321 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0103] The acceleration sensor 380E can periodically collect acceleration data of the electronic device at a certain frequency. For example, the acceleration magnitude of the electronic device in various directions (generally XYZ three-axis directions) can be collected.

[0104] Of course, it is understandable that the above Figure 3 The figure is only an exemplary description when the electronic device is in the form of a mobile phone. If the electronic device is in the form of a tablet computer, a handheld computer, a PC, a PDA, a wearable device (such as a smart watch, a smart bracelet), etc., the structure of the electronic device may include Figure 3 The structure shown in the figure is less than Figure 3 More structures are shown in the figure, which are not limited here.

[0105] It is understandable that, generally speaking, the realization of electronic device functions requires not only hardware support but also software cooperation.

[0106] The software system of the electronic device can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. Taking the system as an example, the software structure of the electronic device is illustrated.

[0107] Figure 4 It is a software structure block diagram of the electronic device provided in the embodiment of the present application.

[0108] like Figure 4 As shown, the system used by electronic equipment is In the embodiment of the present application, the hierarchical architecture divides the software of the electronic device into multiple layers, which are the application layer (i.e., APP layer), the application framework layer (i.e., Framework Java layer), the system library and Runtime( runtime) (i.e. FrameworkNative layer), HAL layer (hardware abstraction layer) and kernel layer (also called driver layer, i.e. Kernel layer).

[0109] The application layer may include a series of applications. The application may be a third-party application or an application that comes with the electronic device. The application may be an application that can display multimedia resources, such as a news application, a shopping application, etc. The specific type of application for the electronic device is not limited in the implementation of this application. It should be noted that the application layer may also include applications such as camera, gallery, calendar, map, WLAN, Bluetooth, music, video, short message, call, etc.

[0110] The application framework layer can provide an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions. For example, the application framework layer can include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc.

[0111] like Figure 4 As shown, in order to implement the display method provided in the embodiment of the present application, the application framework layer can receive the image (also referred to as image data) sent by the application, and mark the image corresponding to the image sent by the application, that is, the application framework layer can specially mark the relevant layers of the image sent by the application and send it down.

[0112] It should be noted that the application framework layer marks the different layers of the image and passes them to the underlying rendering engine for processing. In application development, images are often composed of multiple layers. For example, an application interface containing text, pictures, and buttons, each part may be a separate layer. These layers are marked at the application framework layer, and can be marked in different ways, such as marking a unique identifier or adding special properties to the layer object.

[0113] By marking different layers, applications can process them individually. For example, apply different animation effects, transparency or transformation effects on the layers. These tags will be passed to the underlying rendering engine to correctly render the application's layer structure.

[0114] The purpose of sending the relevant layers of the application image with special marks is to achieve more complex and flexible interface effects. By marking different layers as special types or attaching specific attributes, the application can process them differently as needed, such as animation, transition, effects, etc. This helps to improve the user experience and interactivity of the application.

[0115] Runtime The runtime includes the core library and the virtual machine. Runtime Runtime is responsible for Scheduling and management of the system.

[0116] The core library consists of two parts: one is the function that the Java language needs to call, and the other is The core library.

[0117] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0118] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0119] like Figure 4 As shown, in order to implement the display method provided in the embodiment of the present application, the system library and The runtime may also include a display composition system (i.e., surfaceflinger).

[0120] The display composition system can be used to receive images sent by the application framework layer (that is, images marked with different layers). The display composition system can also determine the rendering composition path of the image based on the received image.

[0121] In the case where the rendering synthesis path of the image is GPU, the display synthesis system sends the image to the GPU of the electronic device. The GPU can be used to render and synthesize the image sent by the display synthesis system, and send the rendered and synthesized image to the display of the electronic device through the hardware abstraction layer and the kernel layer for display. In the case where the rendering synthesis path of the image is HWC, the display synthesis system sends the image to the HWC of the hardware abstraction layer. The HWC can be used to render and synthesize the image sent by the display synthesis system, and send the rendered and synthesized image to the display of the electronic device through the kernel layer for display.

[0122] The display composition system can also be used to determine the scene of the application, that is, to determine whether the image sent by the application needs to be histogram detected. When the scene of the application indicates that histogram detection is required, the display composition system can determine the histogram detection path of the image.

[0123] In the case where the histogram detection path of the image is GPU, the display synthesis system sends a histogram detection instruction to the GPU. The GPU can determine the histogram data of the image according to the histogram detection instruction, and send the histogram data to the display synthesis system (i.e., histogram data reporting). In the case where the histogram detection path of the image is HWC, the display synthesis system sends a histogram detection instruction to the HWC. The HWC can determine the histogram data of the image according to the histogram detection instruction, and send the histogram data to the display synthesis system (i.e., histogram data reporting).

[0124] The display synthesis system can also be used to receive histogram data and determine the image loading state based on the histogram data. When the image loading state is a slow loading state, the display synthesis system can determine the frequency increase strategy of the CPU of the electronic device. The display synthesis system can also be used to send the frequency after the frequency increase to the CPU of the electronic device.

[0125] The HAL layer is an interface layer between the operating system kernel and the hardware circuit, and its purpose is to abstract the hardware. It hides the hardware interface details of a specific platform and provides a virtual hardware platform for the operating system, making it hardware-independent and portable on multiple platforms. The HAL layer provides a standard interface to display device hardware functions to the higher-level Java API framework (i.e., the framework layer). The HAL layer contains multiple library modules, each of which implements an interface for a specific type of hardware component, such as: audio HAL audio module, bluetooth HAL Bluetooth module, camera HAL camera module, sensors HAL sensor module (or Isensor service, sensor service).

[0126] like Figure 4As shown, in order to implement the display method provided in the embodiment of the present application, the HAL layer may also include a HWC (ie, a hardware synthesizer).

[0127] The HWC may be used to receive an image sent by a display composition system, perform rendering and composition on the object, and send the rendered and composed image to a display of an electronic device through a kernel layer for display.

[0128] The HWC may also be configured to receive a histogram detection instruction sent by a display composition system, determine histogram data of an image based on the histogram detection instruction, and send the histogram data to the display composition system.

[0129] It should be noted that HWC can determine the histogram data of the image through the data processing unit (DPU) in the kernel layer. That is, the DPU can perform histogram detection on the image to determine the histogram data of the image. After the DPU determines the histogram data, the DPU can report the histogram data to the HWC through the clustered language technology center manager (CLSTC Manager). As a hardware abstraction layer component, CLSTC Manager provides an interface and mechanism for communicating with hardware components, which can help the DPU transfer data to the HWC or other target devices.

[0130] After HWC obtains the histogram data sent by DPU, HWC can send the histogram data to the display composition system through the display engine hardware abstraction layer (DE HAL). DE HAL is a software layer that provides an abstract interface between the operating system and the display engine hardware. Through DE HAL, the operating system can communicate with a variety of different display engine hardware without having to worry about hardware details and specific implementations. This can reduce the complexity of operating system and application development and improve portability and compatibility.

[0131] The kernel layer is a layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, sensor driver, etc., which are not limited in the embodiments of the present application. Among them, the sensor driver can specifically include the driver of each sensor included in the electronic device, such as a nine-axis sensor driver, which can include a three-axis gyroscope sensor driver, a three-axis magnetic sensor driver, and a three-axis acceleration sensor driver.

[0132] The following takes the multimedia resource corresponding to the application loaded by the electronic device as an example as an image. Figure 5 , the display method provided in the embodiment of the present application is described in detail. Figure 5 As shown, the display method may include the following S501-S532.

[0133] S501: An application of an electronic device receives a display operation from a user.

[0134] The application of the electronic device (which may also be referred to as the first application in the embodiment of the present application) may be an application that can display multimedia resources, such as a news application, a shopping application, etc. The specific type of the application is not limited in the embodiment of the present application. Multimedia resources may include images, texts, etc. The specific type of multimedia resources is not limited in the embodiment of the present application. The multimedia resources are images as an example for illustration in the embodiment of the present application.

[0135] When the user needs to display the content of the application through the electronic device, the application of the electronic device can receive the display operation of the user, so that the electronic device can display the content of the application.

[0136] In some examples, the user's display operation may be that the electronic device receives the user's operation to start an application, so that the electronic device displays the main interface corresponding to the application, and the main interface corresponding to the application may include multimedia resources corresponding to the application (such as images, etc.). The user's display operation may also be that the electronic device receives the user's operation to switch a page of the application, so that the electronic device displays the interface corresponding to the page, and the interface corresponding to the page may include multimedia resources corresponding to the page (such as images, etc.).

[0137] For example, when the application is a news application, the user's display operation may be that the electronic device receives a start operation of the news application from the user, so that the electronic device displays the main interface corresponding to the news application, and the main interface corresponding to the news application may include an image corresponding to the news application. The user's display operation may also be that the electronic device receives a switch operation of the homepage or video of the news application from the user, so that the electronic device displays the interface corresponding to the homepage or video of the news application, and the interface corresponding to the homepage or video may include an image corresponding to the homepage or video.

[0138] S502: An application of the electronic device sends first image data to a display synthesis system of the electronic device.

[0139] After the application of the electronic device receives the display operation of the user, the application of the electronic device can send the first image data to the display synthesis system of the electronic device, so that the display synthesis system of the electronic device can determine the rendering synthesis path of the first image data. The first image data is the first frame of image data to be synthesized.

[0140] In some examples, an application of an electronic device sends first image data to a display synthesis system of the electronic device, which may include the application of the electronic device sending the first image data to an application framework layer of the electronic device. Afterwards, the application framework layer of the electronic device creates multiple layers corresponding to the first image data. Afterwards, the application framework layer of the electronic device sends one or more layers corresponding to the first image data to the display synthesis system of the electronic device. It should be noted that after the application framework layer of the electronic device receives the first image data, the application framework layer may specially mark one or more layers corresponding to the first image data, thereby achieving a more complex and flexible interface effect. By marking different layers as special types or attaching specific attributes, the application can process them differently as needed, such as animation, transition, effects, etc. This helps to improve the user experience and interactivity of the application.

[0141] That is, the application of the electronic device sends the first image data to the display composition system of the electronic device, which may include one or more layers corresponding to the first image data.

[0142] S503: The display synthesis system of the electronic device receives the first image data and determines a rendering synthesis path of the first image data.

[0143] After the display synthesis system of the electronic device receives the first image data sent by the application, the display synthesis system of the electronic device can determine the rendering synthesis path corresponding to the first image data based on the first image data. The rendering synthesis path may include rendering synthesis by the HWC of the electronic device and rendering synthesis by the GPU of the electronic device.

[0144] In some examples, the first image data may include one or more layers corresponding to the first image, and an identifier corresponding to each layer. Layers are different elements that make up an image, and each layer may contain independent content, effects, and properties. Layers can be superimposed on each other to create a final composite effect. Rendering synthesis refers to the process of synthesizing different images (or layers) or elements into a final image or video. In other words, the first image data may include a first frame of layers to be synthesized. It should be noted that the first image data may also include an identifier of an application.

[0145] In some examples, the display synthesis system of the electronic device determines, based on the first image data, the rendering synthesis path corresponding to the first image data, which may include: the display synthesis system of the electronic device determines, based on the size of the layer corresponding to the first image data or the number of layers corresponding to the first image data, the rendering synthesis path corresponding to the first image data. For example, when the display synthesis system of the electronic device determines that the size of the layer corresponding to the first image data is large or the number of layers corresponding to the first image data is large, the electronic device may determine that the rendering synthesis path corresponding to the first image data is to be rendered and synthesized by the GPU of the electronic device. When the display synthesis system of the electronic device determines that the size of the layer corresponding to the first image data is small or the number of layers corresponding to the first image data is small, the electronic device may determine that the rendering synthesis path corresponding to the first image data is to be rendered and synthesized by the HWC of the electronic device.

[0146] In other examples, the display synthesis system of the electronic device may also determine the rendering synthesis path corresponding to the first image data according to the content of the layer corresponding to the first image data, the hardware condition of the electronic device, the requirements of the application program, and other factors. The specific manner in which the display synthesis system of the electronic device determines the rendering synthesis path corresponding to the first image data in the embodiments of the present application is not limited.

[0147] It should be noted that, for each frame of image data (ie, each frame of a layer to be synthesized) sent by an application of an electronic device, the display synthesis system of the electronic device needs to determine a rendering synthesis path for the image data.

[0148] S504: When the rendering synthesis path of the first image data is HWC, the display synthesis system of the electronic device sends the first image data to the HWC of the electronic device.

[0149] When the display synthesis system of the electronic device determines that the rendering synthesis path corresponding to the first image data is rendered and synthesized by the HWC of the electronic device (that is, the rendering synthesis path of the first image data is the HWC), the display synthesis system of the electronic device can send the first image data to the HWC of the electronic device. Thus, the HWC of the electronic device can render and synthesize the first image data (also referred to as loading).

[0150] S505: The HWC of the electronic device receives the first image data, and renders and synthesizes the first image data based on the first frequency of the CPU.

[0151] After the HWC of the electronic device receives the first image data, the HWC of the electronic device can render and synthesize the first image data based on the first frequency of the CPU of the electronic device (also referred to as loading). It should be noted that the first frequency of the CPU of the electronic device can be the default frequency of the CPU of the electronic device.

[0152] S506. The HWC of the electronic device sends the rendered and synthesized first image data to the display screen of the electronic device.

[0153] After the HWC of the electronic device renders and synthesizes the first image data based on the first frequency of the CPU, the HWC of the electronic device sends the rendered and synthesized first image data to the display screen of the electronic device, so that the display screen can display the rendered and synthesized first image data.

[0154] S507: The display screen of the electronic device receives and displays the rendered and synthesized first image data.

[0155] S508: When the rendering synthesis path of the first image data is a GPU, the display synthesis system of the electronic device sends the first image data to the GPU of the electronic device.

[0156] When the display synthesis system of the electronic device determines that the rendering synthesis path corresponding to the first image data is rendered and synthesized by the GPU of the electronic device (that is, the rendering synthesis path of the first image data is the GPU), the display synthesis system of the electronic device can send the first image data to the GPU of the electronic device. Thus, the GPU of the electronic device can render and synthesize the first image data.

[0157] S509: The GPU of the electronic device receives the first image data, and renders and synthesizes the first image data based on the first frequency of the CPU.

[0158] After the GPU of the electronic device receives the first image data, the GPU of the electronic device may render and synthesize the first image data based on the first frequency of the CPU of the electronic device. It should be noted that the first frequency of the CPU of the electronic device may be the default frequency of the CPU of the electronic device.

[0159] S510: The GPU of the electronic device sends the rendered and synthesized first image data to the display screen of the electronic device.

[0160] After the GPU of the electronic device renders and synthesizes the first image data based on the first frequency of the CPU, the HWC of the electronic device sends the rendered and synthesized first image data to the display screen of the electronic device, so that the display screen can display the rendered and synthesized first image data.

[0161] S511. The display screen of the electronic device receives and displays the rendered and synthesized first image data.

[0162] S512: The display synthesis system of the electronic device determines a scene of the application, and when the scene of the application indicates that histogram detection is required, determines a histogram detection path for the first image data.

[0163] After the display composition system of the electronic device receives the first image data sent by the application, the display composition system of the electronic device can determine the scene of the application. The scene of the application can be used to indicate whether it is necessary to perform histogram detection on the first image data. That is, the scene of the application can include a scene that requires histogram detection and a scene that does not require histogram detection.

[0164] In some examples, when the first image data includes an identifier of an application, the display synthesis system of the electronic device determines the scene of the application, which may include the display synthesis system of the electronic device determining the scene of the application based on the first image data and a preset application whitelist.

[0165] The preset application whitelist may include identifiers of multiple applications that require histogram detection. When the identifier of the application included in the first image data is an identifier of an application in the preset application whitelist, the display synthesis system of the electronic device may determine that the scene of the application is a scene that requires histogram detection. When the identifier of the application included in the first image data is not an identifier of an application in the preset application whitelist, the display synthesis system of the electronic device may determine that the scene of the application is a scene that does not require histogram detection.

[0166] In other examples, when the first image data includes one or more layers corresponding to the first image and an identifier corresponding to each layer, the display synthesis system of the electronic device determines the scene of the application, which may include the display synthesis system of the electronic device determining the scene of the application based on the identifiers corresponding to the layers included in the first image data and a preset layer whitelist.

[0167] The identifier corresponding to the layer can be a name corresponding to the layer, etc., which is not limited in the embodiments of the present application. The preset layer whitelist may include identifiers of multiple layers that require histogram detection. When the identifier of the layer included in the first image data is an identifier of a layer in the preset layer whitelist, the display synthesis system of the electronic device can determine that the scene of the application is a scene that requires histogram detection. When the identifier of the layer included in the first image data is not an identifier of a layer in the preset layer whitelist, the display synthesis system of the electronic device can determine that the scene of the application is a scene that does not require histogram detection.

[0168] In other examples, when the first image data includes one or more layers corresponding to the first image, the display synthesis system of the electronic device determines the scene of the application, which may include the display synthesis system of the electronic device determining the scene of the application according to the layer attribute information (such as layer extension attributes) corresponding to the layer included in the first image data. That is, when a specific layer contains image decoding resources or has features related to the loading of the first image data (streaming media resources), the layer will be identified and requires subsequent histogram detection, that is, determining that the scene of the application requires histogram detection.

[0169] The extended properties of a layer can include blending modes, opacity and transparency masks, layer styles, adjustment layers, layer masks, layer groups, 3D layer effects, etc.

[0170] Blending modes, that is, by changing the way a layer blends with the layers below, different effects can be produced, such as overlay, color deepening, lightening, etc. Opacity and transparency masks, that is, you can adjust the overall opacity of a layer, or apply a transparency mask to a layer to achieve a partial transparency effect. Layer styles, that is, by adding style effects such as shadows, glows, strokes, gradients, etc., you can make a layer look more three-dimensional and vivid. Adjustment layers, that is, you can make non-destructive adjustments to the color, brightness, contrast, etc. in a layer without affecting the original image. Layer masks, that is, by applying a layer mask, you can control the visibility of a layer and only display the area specified in the mask. Layer groups, that is, you can combine multiple layers together to form a separate group and apply unified properties or effects to the group. 3D layer effects, that is, support the creation and editing of 3D layers, and you can apply effects such as rotation, tilt, and scaling to them to make them three-dimensional.

[0171] Exemplarily, when the extended attributes of the layer include a blending mode, the display synthesis system of the electronic device determines the scene of the application based on the extended attributes of the layer corresponding to the layer included in the first image data, and can determine whether the layer applies a blending mode for the display synthesis system of the electronic device. If a blending mode is applied to the layer, such as hue / saturation, color balance, curve adjustment, etc., then a histogram detection may be required to ensure that the color adjustment does not lose image details or introduce color deviation. That is, when the extended attributes of the layer indicate that a blending mode is applied to the layer, the display synthesis system of the electronic device can determine that the scene of the application is a scene that requires histogram detection. When the extended attributes of the layer indicate that no blending mode is applied to the layer, the display synthesis system of the electronic device can determine that the scene of the application is a scene that does not require histogram detection.

[0172] For another example, when the extended attributes of the layer include a layer mask, the display composition system of the electronic device determines the scene of the application according to the extended attributes of the layer corresponding to the layer included in the first image data, and can determine whether the layer applies a layer mask for the display composition system of the electronic device. When the extended attributes of the layer indicate that a layer mask is applied to the layer, the display composition system of the electronic device can determine that the scene of the application is a scene that requires histogram detection. When the extended attributes of the layer indicate that a layer mask is not applied to the layer, the display composition system of the electronic device can determine that the scene of the application is a scene that does not require histogram detection.

[0173] When the display composition system of the electronic device determines that the scene indication of the application requires histogram detection, the display composition system of the electronic device can determine a histogram detection path for the first image data. The histogram detection path for the first image data can include histogram detection by the GPU of the electronic device or histogram detection by the HWC of the electronic device.

[0174] In some examples, the display synthesis system of the electronic device determines the histogram detection path of the first image data, and the electronic device may determine the histogram detection path of the first image data based on the rendering synthesis path of the first image data. For example, in a case where the rendering synthesis path of the first image data is rendered and synthesized by the GPU of the electronic device, the display synthesis system of the electronic device may determine that the histogram detection path of the first image data is performed by the GPU of the electronic device for histogram detection. In a case where the rendering synthesis path of the first image data is rendered and synthesized by the HWC of the electronic device, the display synthesis system of the electronic device may determine that the histogram detection path of the first image data is performed by the HWC of the electronic device for histogram detection.

[0175] It should be noted that after the display synthesis system of the electronic device receives the first image data sent by the application, the electronic device may first determine the rendering synthesis path of the first image data, and then determine the scene of the application. The electronic device may also first determine the scene of the application, and then determine the rendering synthesis path of the first image data. The electronic device may also simultaneously determine the scene of the application and the rendering synthesis path of the first image data. In the embodiment of the present application, the order of determining the rendering synthesis path of the first image data and determining the scene of the application is not limited. In the embodiment of the present application, the rendering synthesis path of the first image data is first determined, and then the scene of the application is determined as an example for schematic explanation.

[0176] S513: When the histogram detection path of the first image data is HWC, the display synthesis system of the electronic device sends a histogram detection instruction to the HWC of the electronic device.

[0177] When the display composition system of the electronic device determines that the histogram detection path of the first image data is the HWC, the display composition system of the electronic device may send a histogram detection instruction to the HWC of the electronic device. The histogram detection instruction is used to instruct the HWC of the electronic device to obtain the histogram data corresponding to the first image data.

[0178] S514: The HWC of the electronic device receives a histogram detection instruction, and acquires histogram data of the first image data based on the histogram detection instruction.

[0179] After the HWC of the electronic device receives the histogram detection instruction, the HWC of the electronic device can obtain the histogram data corresponding to the first image data based on the histogram detection instruction.

[0180] In some examples, the HWC of the electronic device obtains the histogram data of the first image data based on the histogram detection instruction, and the HWC of the electronic device can obtain the histogram data corresponding to the first image data from the histogram database (ie, libhistogram) of the electronic device.

[0181] For example, the DPU of the electronic device may perform histogram detection on the first image data to obtain histogram data corresponding to the first image data. Afterwards, the DPU of the electronic device may store the histogram data corresponding to the first image data in a histogram database. Afterwards, the HWC may determine the histogram data corresponding to the first image data from the histogram database.

[0182] It should be noted that when the rendering and synthesis path of the first image data is HWC, when HWC renders and synthesizes the first image data, the DPU will perform histogram detection on the first image data and store the histogram data corresponding to the first image data in the histogram database. In other words, the HWC of the electronic device obtains the histogram data of the first image data from the known histogram database without the need for additional histogram detection.

[0183] The histogram data may include bin values ​​in a histogram corresponding to the first image data, and the number of pixels corresponding to each bin value (also referred to as frequency).

[0184] The bin value in the histogram refers to the distribution range of colors in the color space. When calculating the color histogram, the color space is divided into several small color intervals, namely the bins of the histogram. For example, a histogram has 10 bins, which means that the color space is divided into 10 equally divided intervals.

[0185] The color histogram is obtained by calculating the pixels in each color interval. The more bin values ​​there are, the stronger the resolution of the histogram for color. The histogram is a statistical analysis of data, which organizes the statistical values ​​into a series of pre-defined bin values. The value in the bin value is the statistical value of the pixel of the color represented by the bin value calculated from the image data.

[0186] For example, when the bin value in the histogram is 255, it indicates the corresponding white pixel in the first image data, that is, when the bin value in the histogram is 255, the number of corresponding pixels is the number of white pixels in the first image data. Figure 6 As shown in (a), when the bin value in the histogram is 255, the corresponding number of pixels is the number of white pixels in the first image data is 2382636, that is, the number of white pixels in the first image data is 2382636, which means that the electronic device displays a white image at this time.

[0187] like Figure 6 As shown in (b), when the bin value in the histogram is 255, the corresponding number of pixels is 1096076, that is, the number of white pixels in the first image data is 1096076, which means that the electronic device displays a partial white image at this time. The electronic device can display part of the content in the first image data, such as the text part in the first image data.

[0188] like Figure 6 As shown in (c), when the bin value in the histogram is 255, the corresponding number of pixels is the number of white pixels in the first image data is 1075145, that is, the number of white pixels in the first image data is 1075145, which means that the electronic device does not display a white image at this time, but can display all the contents in the first image data, such as the text part and the image part in the first image data.

[0189] S515. The HWC of the electronic device sends the histogram data to the display synthesis system of the electronic device.

[0190] S516: When the histogram detection path of the first image data is the GPU, the display synthesis system of the electronic device sends a histogram detection instruction to the GPU of the electronic device.

[0191] When the display composition system of the electronic device determines that the histogram detection path of the first image data is the GPU, the display composition system of the electronic device can send a histogram detection instruction to the GPU of the electronic device. The histogram detection instruction is used to instruct the GPU of the electronic device to obtain the histogram data corresponding to the first image data.

[0192] S517: The GPU of the electronic device receives the histogram detection instruction, and determines the histogram data of the first image data based on the histogram detection instruction.

[0193] After the GPU of the electronic device receives the histogram detection instruction, the GPU of the electronic device may perform histogram detection on the first image data based on the histogram detection instruction, thereby acquiring histogram data corresponding to the first image data.

[0194] S518: The GPU of the electronic device sends the histogram data to the display synthesis system of the electronic device.

[0195] S519: The display synthesis system of the electronic device receives the histogram data, and determines the image loading state based on the histogram data.

[0196] After the display synthesis system of the electronic device receives the histogram data, the electronic device may determine the image loading state (which may also be referred to as the loading state in the embodiment of the present application) based on the histogram data.

[0197] In some examples, the image loading status may include a normal image loading status (which may also be referred to as a normal loading status in the embodiment of the present application), a slow image loading status (which may also be referred to as a slow loading status in the embodiment of the present application), and an image loading waiting process status (which may also be referred to as a loading waiting status in the embodiment of the present application).

[0198] The normal image loading state means that the image loading speed is normal and the electronic device does not need to intervene. The slow image loading state means that the image loading speed is slow and the electronic device displays a white image for a long time, so the electronic device can increase the CPU frequency of the electronic device to speed up the image loading. The image loading waiting process state means that the image loading speed is normal and the electronic device does not need to intervene.

[0199] In some examples, when the histogram data includes a bin value in a histogram, the display synthesis system of the electronic device determines the image loading state based on the histogram data, which may include: the display synthesis system of the electronic device determines the image change difference (which may also be referred to as the image data change value in the embodiment of the present application) and determines the accumulated time of the image change (which may also be referred to as the duration of the image data change in the embodiment of the present application) according to the number of pixels corresponding to the preset bin value in the histogram corresponding to the first image data and the number of pixels corresponding to the preset bin value in the histogram corresponding to the third image data. Afterwards, the display synthesis system of the electronic device may determine the image loading state according to the image change difference and the accumulated time of the image change.

[0200] The image change difference is the absolute value of the difference between the number of pixels corresponding to the preset bin value of the first image data and the number of pixels corresponding to the preset bin value of the third image data.

[0201] The third image data is the image data before the first image data. That is, when the first image data is the image data corresponding to the current frame, the third image data is the image data corresponding to the previous frame of the current frame.

[0202] The preset bin value can be a bin value in the histogram data, or multiple bin values. The specific number of preset bin values ​​is not limited in the embodiment of the present application. In the embodiment of the present application, the preset bin value can be a bin value in the histogram data, and the preset bin value is 255 as an example for schematic illustration.

[0203] The accumulated time of image change (may also be referred to as duration threshold in the embodiment of the present application), that is, the time from the time when the current image change difference is greater than the image change threshold (may also be referred to as change threshold in the embodiment of the present application) to the time when the last image change difference was greater than the image change threshold.

[0204] In some examples, the display synthesis system of the electronic device determines the image loading state based on the image change difference and the accumulated time of the image change, which may include: when the image change difference is greater than the image change threshold and the accumulated time of the image change is less than the accumulated time threshold, the display synthesis system of the electronic device determines that the image loading state is a normal image loading state; when the image change difference is less than the image change threshold and the accumulated time of the image change is greater than the accumulated time threshold, the display synthesis system of the electronic device determines that the image loading state is a slow image loading state; when the image change difference is less than the image change threshold and the accumulated time of the image change is less than the accumulated time threshold, the display synthesis system of the electronic device determines that the image loading state is an image loading waiting state.

[0205] It should be noted that the specific value of the image change threshold can be set according to actual conditions, and this is not limited in the embodiments of the present application. The specific value of the cumulative time threshold can be set according to actual conditions, and this is not limited in the embodiments of the present application.

[0206] S520: When the image loading state is slow loading, the display synthesis system of the electronic device determines that the frequency of the CPU of the electronic device is a second frequency.

[0207] When the display synthesis system of the electronic device determines that the image loading state is slow loading, the display synthesis system of the electronic device can determine that the frequency of the CPU of the electronic device is a second frequency. The second frequency can be greater than the first frequency. That is, when it is determined that the image loading state is slow loading, the CPU of the electronic device can increase the frequency, thereby increasing the speed of image loading.

[0208] It should be noted that the specific value of the second frequency can be set according to actual conditions. The specific value of the second frequency is not limited in the embodiment of the present application, and the second frequency can be greater than the first frequency.

[0209] S521: The display synthesis system of the electronic device sends a second frequency to the CPU of the electronic device.

[0210] After the display composition system of the electronic device determines that the frequency of the CPU of the electronic device is the second frequency, the display composition system of the electronic device may send the second frequency to the CPU of the electronic device.

[0211] S522: The CPU of the electronic device receives the second frequency, and adjusts the frequency from the first frequency to the second frequency.

[0212] S523: The application of the electronic device sends the second image data to the display synthesis system of the electronic device.

[0213] After the CPU of the electronic device adjusts the frequency from the first frequency to the second frequency, the application of the electronic device can send the second image data to the display synthesis continuation system of the electronic device, so that the display synthesis system of the electronic device can continue to determine the rendering synthesis path of the second image data.

[0214] The second image data is image data subsequent to the first image data. For example, when the first image data is image data of the first frame to be synthesized, the second image data may be image data of the second frame to be synthesized.

[0215] In some examples, an application of an electronic device sends the second image data to a display synthesis system of the electronic device, which may include the application of the electronic device sending the second image data to an application framework layer of the electronic device. Afterwards, the application framework layer of the electronic device creates multiple layers corresponding to the second image data. Afterwards, the application framework layer of the electronic device sends one or more layers corresponding to the second image data to the display synthesis system of the electronic device. It should be noted that after the application framework layer of the electronic device receives the second image data, the application framework layer may specially mark one or more layers corresponding to the second image data, thereby achieving a more complex and flexible interface effect. By marking different layers as special types or attaching specific attributes, the application can process them differently as needed, such as animation, transition, effects, etc. This helps to improve the user experience and interactivity of the application.

[0216] That is, the application of the electronic device sends the second image data to the display composition system of the electronic device, which may include one or more layers corresponding to the second image data.

[0217] S524: The display synthesis system of the electronic device receives the second image data and determines a rendering synthesis path for the second image data.

[0218] After the display composition system of the electronic device receives the second image data sent by the application, the display composition system of the electronic device may determine a rendering composition path corresponding to the second image data according to the second image data.

[0219] In some examples, the second image data may include one or more layers corresponding to the first image and an identifier corresponding to each layer. The second image data may also include an identifier of an application.

[0220] In some examples, the display synthesis system of the electronic device determines the rendering synthesis path corresponding to the second image data based on the second image data, which may include: the display synthesis system of the electronic device determines the rendering synthesis path corresponding to the second image data based on the size of the layer corresponding to the second image data or the number of layers corresponding to the second image data.

[0221] In other examples, the display synthesis system of the electronic device may also determine the rendering synthesis path corresponding to the second image data according to the content of the layer corresponding to the second image data, the hardware condition of the electronic device, the requirements of the application program, and other factors. The specific manner in which the display synthesis system of the electronic device determines the rendering synthesis path corresponding to the second image data in the embodiments of the present application is not limited.

[0222] S525: When the rendering synthesis path of the second image data is HWC, the display synthesis system of the electronic device sends the second image data to the HWC of the first electronic device.

[0223] When the display synthesis system of the electronic device determines that the rendering synthesis path corresponding to the second image data is rendered and synthesized by the HWC of the electronic device (that is, the rendering synthesis path of the second image data is the HWC), the display synthesis system of the electronic device can send the second image data to the HWC of the electronic device. Thus, the HWC of the electronic device can render and synthesize the second image data.

[0224] S526: The HWC of the electronic device receives the second image data, and renders and synthesizes the second image data based on the second frequency of the CPU.

[0225] After the HWC of the electronic device receives the second image data, the HWC of the electronic device may render and synthesize the second image data based on the second frequency of the CPU of the electronic device.

[0226] The HWC of the electronic device has a close relationship with the CPU of the electronic device when rendering and synthesizing the second image data. During the rendering and synthesis process of the image data, the CPU and HWC of the electronic device need to work together. The CPU of the electronic device is responsible for calculating and scheduling rendering tasks, distributing rendering tasks to the HWC of the electronic device for processing, and also processing some rendering tasks that cannot be processed by the HWC of the electronic device. The HWC of the electronic device is responsible for using its powerful computing power to quickly render images, and closely cooperates with the CPU of the electronic device to complete the rendering and synthesis tasks. Therefore, after the frequency of the CPU is increased from the first frequency to the second frequency, the CPU of the electronic device can execute instructions faster, thereby accelerating the loading, switching and various computing operations of the application. At the same time, the rendering and synthesis speed of the second image data by the HWC of the electronic device is also correspondingly improved.

[0227] S527. The HWC of the electronic device sends the rendered and synthesized second image data to the display screen of the electronic device.

[0228] After the HWC of the electronic device renders and synthesizes the second image data to obtain the rendered and synthesized second image data, the HWC of the electronic device can send the rendered and synthesized second image data to the display screen of the electronic device, so that the electronic device can display the rendered and synthesized second image data.

[0229] S528. The display screen of the electronic device receives and displays the rendered and synthesized second image data.

[0230] After the display screen of the electronic device receives and displays the rendered and synthesized second image data, the application of the electronic device may send fourth image data to the display synthesis system of the electronic device, and the fourth image data may be image data after the second image data. Afterwards, the display synthesis system of the electronic device may determine the scene of the application, and when the scene of the application indicates that histogram detection is required, determine the histogram detection path of the fourth image data. Afterwards, the display synthesis system of the electronic device may obtain the histogram data corresponding to the fourth image data based on the histogram detection path of the fourth image data. Afterwards, the display synthesis system of the electronic device may determine the image loading state based on the histogram data corresponding to the fourth image data.

[0231] When the image loading state is the slow image loading state, the display composition system of the electronic device may determine that the frequency of the CPU of the electronic device continues to be the second frequency.

[0232] When the image loading state is the normal state of image loading or the waiting process of image loading, the display synthesis system of the electronic device can determine that the frequency of the CPU of the electronic device is the first frequency. The display synthesis system of the electronic device can send the first frequency to the CPU of the electronic device. Thus, the CPU of the electronic device can adjust the frequency from the second frequency to the first frequency. Afterwards, the electronic device can obtain the fifth image data of the first application. The fifth image data is the image data of the next frame of the fourth image data. Afterwards, the electronic device can load and display the fifth image data based on the first frequency.

[0233] S529: When the rendering synthesis path of the second image data is the GPU, the display synthesis system of the electronic device sends the second image data to the GPU of the electronic device.

[0234] When the display synthesis system of the electronic device determines that the rendering synthesis path corresponding to the second image data is to be rendered and synthesized by the GPU of the electronic device (i.e., the rendering synthesis path of the second image data is the GPU), the display synthesis system of the electronic device may send the second image data to the GPU of the electronic device. Thus, the GPU of the electronic device may perform rendering synthesis on the second image data.

[0235] S530: The GPU of the electronic device receives the second image data, and renders and synthesizes the second image data based on the second frequency of the CPU.

[0236] After the GPU of the electronic device receives the second image data, the GPU of the electronic device may render and synthesize the second image data based on the second frequency of the CPU of the electronic device.

[0237] In image rendering and synthesis, the CPU is usually responsible for managing tasks such as task scheduling and data transmission, while the GPU is responsible for actual image calculation and rendering. Therefore, the performance and frequency of the CPU will also affect the overall image rendering performance. A CPU with a higher frequency can process and transmit data faster, thereby working more effectively with the GPU. Therefore, after the frequency of the CPU is increased from the first frequency to the second frequency, the CPU of the electronic device can execute instructions faster, thereby accelerating the loading, switching and various computing operations of the application. At the same time, the rendering and synthesis speed of the second image data by the GPU of the electronic device is also correspondingly improved.

[0238] S531. The GPU of the electronic device sends the rendered and synthesized second image data to the display screen of the electronic device.

[0239] After the HWC of the electronic device renders and synthesizes the second image data to obtain the rendered and synthesized second image data, the HWC of the electronic device can send the rendered and synthesized second image data to the display screen of the electronic device, so that the electronic device can display the rendered and synthesized second image data.

[0240] S532: The display screen of the electronic device receives and displays the rendered and synthesized second image data.

[0241] After the display screen of the electronic device receives and displays the rendered and synthesized second image data, the application of the electronic device may send fourth image data to the display synthesis system of the electronic device, and the fourth image data may be image data after the second image data. Afterwards, the display synthesis system of the electronic device may determine the scene of the application, and when the scene of the application indicates that histogram detection is required, determine the histogram detection path of the fourth image data. Afterwards, the display synthesis system of the electronic device may obtain the histogram data corresponding to the fourth image data based on the histogram detection path of the fourth image data. Afterwards, the display synthesis system of the electronic device may determine the image loading state based on the histogram data corresponding to the fourth image data.

[0242] When the image loading state is the slow image loading state, the display composition system of the electronic device may determine that the frequency of the CPU of the electronic device continues to be the second frequency.

[0243] When the image loading state is the normal state of image loading or the waiting process of image loading, the display synthesis system of the electronic device can determine that the frequency of the CPU of the electronic device is the first frequency. The display synthesis system of the electronic device can send the first frequency to the CPU of the electronic device. Thus, the CPU of the electronic device can adjust the frequency from the second frequency to the first frequency. Afterwards, the electronic device can obtain the fifth image data of the first application. The fifth image data is the image data of the next frame of the fourth image data. Afterwards, the electronic device can load and display the fifth image data based on the first frequency.

[0244] In the solution of the present application, when an electronic device loads the first image data corresponding to an application, the electronic device may first determine the scene of the application, and the CPU of the electronic device executes loading and displaying of the first image based on a first frequency. When the electronic device determines that the scene of the application is a scene that requires histogram detection, the electronic device may perform histogram detection on the first image corresponding to the application, thereby obtaining histogram data corresponding to the first image. Afterwards, the electronic device may determine the loading status of the image corresponding to the application based on the histogram data corresponding to the first image. When the loading status is a slow loading status, the CPU of the electronic device may execute loading and display of the second image based on a second frequency, the second frequency being greater than the first frequency, and the second image is an image after the first image.

[0245] The solution of the present application is to perform a histogram detection on an image, and based on the histogram data, it is possible to determine the loading state of the image, that is, to determine whether the loading state of the image is a slow loading state or a normal loading state. And because the histogram detection uses the pixel value distribution information of the image to judge the loading state of the image, it is possible to determine whether the electronic device is loading a white image through the histogram detection. Therefore, the loading state of the image can be accurately determined through the histogram detection. In addition, in the case where the loading state is a slow loading state, the solution of the present application is to increase the frequency of the CPU of the electronic device, and execute the loading of subsequent images based on the increased frequency, which can speed up the loading speed of subsequent images, thereby better optimizing the loading performance of multimedia resources.

[0246] The solution of the present application detects the histogram of the image and determines the loading status of the image based on the histogram data, thereby avoiding the overhead caused by setting a listener for each image separately, effectively saving the resources of the electronic device and improving the overall efficiency of the electronic device system.

[0247] For ease of understanding, the following Figure 7 The display method provided in the embodiment of the present application is described. Figure 7 As shown, the display method may include the following S701-S706.

[0248] S701. When a first application is running on an electronic device, first image data of the first application is obtained.

[0249] The first application is an application in the electronic device that can display multimedia resources, such as a news application, a shopping application, etc. The specific type of the first application is not limited in the embodiment of the present application.

[0250] Multimedia resources may include images. Multimedia resources may also include text and other content. The specific types of multimedia resources are not limited in the embodiments of the present application.

[0251] The electronic device running the first application includes the user opening the first application through the electronic device, or the user switching the display interface of the first application through the electronic device, etc. The specific manner in which the electronic device runs the first application is not limited in the embodiments of the present application.

[0252] When the electronic device runs the first application, the electronic device can obtain the first image data of the first application, so that the electronic device can load and display the first image data.

[0253] Specifically, in the embodiment of the present application, when the electronic device runs the first application, the specific implementation method of obtaining the first image data of the first application can refer to the specific implementation methods in the above S501-S503, which will not be repeated in the embodiment of the present application.

[0254] S702: Determine a loading state of the first image data based on the first image data; the loading state includes a slow loading state and a non-slow loading state.

[0255] After the electronic device acquires the first image data, the electronic device can determine the loading status of the first image data based on the first image data, so that the electronic device can determine whether the CPU of the electronic device needs to be frequency-increased according to the loading status of the first image data.

[0256] The loading state may include a slow loading state and a non-slow loading state. The slow loading state means that the electronic device loads the first image data slowly, and may also be referred to as the electronic device rendering and synthesizing the first image data slowly. When the electronic device loads the first image data, the electronic device may display a preset image such as a white image, and the electronic device loads the first image data slowly, that is, the electronic device displays preset images such as a white image for a long time. Therefore, when the loading state of the first image data is a slow loading state, the electronic device may increase the frequency of the CPU, thereby increasing the loading speed of the electronic device for the image data, that is, increasing the rendering and synthesis speed of the electronic device for the image data.

[0257] The non-slow loading state may include a normal loading state and a loading waiting state. The normal loading state means that the electronic device loads the first image data at a normal speed, which may also be referred to as the normal speed of the electronic device rendering and synthesizing the first image data. The electronic device loads the first image data at a normal speed, which means that the time for the electronic device to display preset images such as white images is normal. Therefore, when the loading state of the first image data is the normal loading state, the electronic device does not need to increase the frequency of the CPU.

[0258] The loading waiting state means that the electronic device loads the first image data at a normal speed and the first image data is being loaded. It can also be said that the electronic device renders and synthesizes the first image data at a normal speed and the electronic device is rendering and synthesizing the first image data. The electronic device loads the first image data at a normal speed, that is, the time for the electronic device to display preset images such as white images is normal. Therefore, when the loading state of the first image data is the loading waiting state, the electronic device does not need to increase the frequency of the CPU.

[0259] In some examples, determining the loading status of the first image data based on the first image data may include: the electronic device acquires histogram data of the third image data. The third image data is image data of a previous frame of the first image data. The electronic device may acquire the histogram data of the first image data, and determine the loading status of the first image data based on the histogram data of the first image data and the histogram data of the third image data.

[0260] Histogram data may include bin values ​​in a histogram corresponding to the image data. The bin values ​​in a histogram refer to the distribution range of colors in the color space corresponding to the image data. For example, a histogram has 10 bins, which means that the color space is divided into 10 equally divided intervals. The number of pixels corresponding to each bin value may represent the number of pixels in the image data within the range corresponding to the bin value.

[0261] Since the electronic device can accurately determine the loading status of the first image data based on the histogram data of the first image data and the histogram data of the third image data. And since the histogram detection uses the pixel value distribution information of the image to determine the loading status of the image, the histogram detection can determine whether the electronic device is loaded with a white image, thereby avoiding the electronic device from displaying a white image for a long time. In addition, by determining the loading status of the image data through the histogram data, the overhead of setting up a listener for each image is avoided, which can save the resources of the electronic device and improve the overall efficiency of the electronic device system.

[0262] In some examples, determining the loading state of the first image data based on the histogram data of the first image data and the histogram data of the third image data may include: the electronic device determines the image data change value and the duration of the image data change based on the histogram data of the first image data and the histogram data of the third image data. When the image data change value is less than the change threshold and the image data change duration is greater than the duration threshold, the electronic device determines that the loading state of the first image data is a slow loading state.

[0263] The specific value of the change threshold can be set according to actual conditions, and is not set in the embodiment of the present application. The specific value of the duration threshold can be set according to actual conditions, and is not set in the embodiment of the present application.

[0264] The image data change value may be a difference between the number of pixels corresponding to a preset bin value of the histogram data of the first image data and the number of pixels corresponding to the preset bin value of the histogram data of the third image data.

[0265] The duration of image data change, that is, the duration from the time when the current image change value is greater than the change threshold to the time when the last image change value was greater than the change threshold.

[0266] It should be noted that the preset bin value can be one bin value or multiple different bin values. For example, when it is necessary to determine whether the loaded image is a white image, the preset bin value can be 255. When it is necessary to determine whether the color distribution of the loaded layer is consistent with expectations, the preset bin value can be multiple different bin values. When it is necessary to determine whether the content of the loaded layer is consistent with expectations, the preset bin value can be multiple different bin values.

[0267] Since the electronic device can determine the image data change value and the duration of the image data change based on the histogram data of the first image data and the histogram data of the third image data, it can accurately determine that the loading state of the first image data is a slow loading state based on the image data change value and the duration of the image data change.

[0268] In some examples, determining the loading status of the first image data based on the histogram data of the first image data and the histogram data of the third image data may also include: when the image data change value is greater than the change threshold, and the image data change duration is greater than the duration threshold, the electronic device determines that the loading status of the first image data is a normal loading status; when the image data change value is less than the change threshold, and the image data change duration is less than the duration threshold, the electronic device determines that the loading status of the first image data is a loading waiting status.

[0269] Since the electronic device can determine the image data change value and the duration of the image data change based on the histogram data of the first image data and the histogram data of the third image data, it can accurately determine whether the loading state of the first image data is a normal loading state or a loading waiting state based on the image data change value and the duration of the image data change.

[0270] In some examples, obtaining the histogram data of the first image data may include: the electronic device determines the loading path of the first image data based on the first image data. The loading path may include a graphics processor GPU of the electronic device and a hardware synthesizer HWC of the electronic device. Afterwards, when the loading path is the GPU, the electronic device performs a histogram detection on the first image data through the GPU to obtain the histogram data of the first image data. When the loading path is the HWC, the electronic device obtains the histogram data of the first image data from the histogram database of the electronic device through the HWC.

[0271] The electronic device can obtain the histogram data of the first image data through the GPU of the electronic device or the HWC of the electronic device, so as to determine the loading status of the first image data according to the histogram data of the first image data.

[0272] In some examples, determining a loading path for the first image data based on the first image data may include: the electronic device determining the loading path for the first image data based on at least one of a first application's processing requirements for the first image data, attribute information of the first image data, and hardware capabilities of the electronic device.

[0273] The electronic device can accurately determine the loading path of the first image data based on the processing requirements of the first application for the first image data, the attribute information of the first image data and the hardware capabilities of the electronic device, and thus can accurately determine the histogram data of the first image data based on the loading path of the first image data.

[0274] In other examples, determining the loading status of the first image data based on the first image data may also include: the electronic device determining the scene of the first application based on the first image data. The scene of the first application includes a scene that requires histogram detection and a scene that does not require histogram detection. When the scene of the first application is a scene that requires histogram detection, the loading status of the first image data is determined based on the first image data. When the scene of the first application is a scene that does not require histogram detection, the electronic device may load and display the first image data based on the first frequency.

[0275] Since the electronic device needs to determine the loading status of the first image data when the application scenario requires histogram detection, and does not need to determine the loading status of the first image data when the application scenario does not require histogram detection, waste of resources of the electronic device can be avoided.

[0276] Specifically, the specific implementation method of determining the loading status of the first image data based on the first image data in the embodiment of the present application can refer to the specific implementation methods in S512-S519 above, which will not be repeated in the embodiment of the present application.

[0277] S703: When the loading state of the first image data is not in a slow loading state, the first image data is loaded and displayed based on a first frequency; the first frequency is an initial frequency of a central processing unit (CPU) of the electronic device.

[0278] When the loading state of the first image data is not a slow loading state, that is, when the speed at which the electronic device loads the first image data is normal, the electronic device does not need to increase the frequency of the CPU. The electronic device can load and display the first image data based on the first frequency. The first frequency can be the initial frequency of the central processing unit CPU of the electronic device.

[0279] Specifically, the specific implementation method of determining the loading status of the first image data based on the first image data in the embodiment of the present application can refer to the specific implementation methods in S504-S511 above, which will not be repeated in the embodiment of the present application.

[0280] S704: When the loading state of the first image data is a slow loading state, adjust the CPU frequency to a second frequency; the second frequency is greater than the first frequency.

[0281] When the loading state of the first image data is a slow loading state, that is, when the electronic device loads the first image data slowly, the electronic device needs to increase the frequency of the CPU. The electronic device can adjust the frequency of the CPU from the first frequency to the second frequency, so that the electronic device can load the image data after the first image data based on the second frequency.

[0282] The specific value of the second frequency can be set according to actual conditions. In the embodiment of the present application, the specific value of the second frequency is not limited, and the second frequency can be greater than the first frequency. Since the second frequency is greater than the first frequency, the speed at which the electronic device loads the image data after the first image data is correspondingly increased.

[0283] In some examples, the non-slow loading state includes a normal loading state and a waiting loading state. When the loading state of the first image data is the normal loading state or the waiting loading state, the electronic device can load and display the second image data based on the first frequency. When the loading state of the first image data is the normal loading state or the waiting loading state, the electronic device does not need to increase the frequency, thereby avoiding high power consumption of the electronic device.

[0284] It should be noted that, when the loading state of the first image data is a slow loading state, that is, when the electronic device loads the first image data slowly, the electronic device also loads and displays the first image data based on the first frequency.

[0285] Specifically, in the embodiment of the present application, when the loading state of the first image data is a slow loading state, the specific implementation method of adjusting the CPU frequency to the second frequency can refer to the specific implementation methods in the above S520-S522, which will not be repeated in the embodiment of the present application.

[0286] S705 , obtaining second image data of the first application; the second image data is a frame of image data subsequent to the first image data.

[0287] After the electronic device adjusts the frequency of the CPU to the second frequency, the electronic device can obtain the second image data of the first application. The second image data can be a frame of image data subsequent to the first image data.

[0288] S706: Load and display the second image data based on the second frequency.

[0289] After the electronic device acquires the second image data of the first application, it can load and display the second image data based on the second frequency.

[0290] Specifically, the specific implementation method of loading and displaying the second image data based on the second frequency in the embodiment of the present application can refer to the specific implementation method in the above S524-S532, which will not be repeated in the embodiment of the present application.

[0291] In some examples, after loading and displaying the second image data based on the second frequency, the electronic device may also obtain fourth image data of the first application. The fourth image data is a frame of image data after the second image data. Afterwards, the electronic device may determine the loading state of the fourth image data based on the fourth image data. When the loading state of the fourth image data is not a slow loading state, the electronic device may adjust the frequency of the CPU from the second frequency to the first frequency.

[0292] Afterwards, the electronic device may obtain fifth image data of the first application. The fifth image data is a frame of image data subsequent to the fourth image data. The electronic device may load and display the fifth image data based on the first frequency.

[0293] When the loading state of the image data is not in the slow loading state, the electronic device may adjust the frequency of the CPU back to the initial frequency, thereby avoiding high power consumption of the electronic device.

[0294] In the solution of the present application, when the electronic device loads the first image data corresponding to the application, the electronic device can determine the loading status of the first image data based on the first image data. When the loading status of the first image data is a slow loading status, the electronic device can increase the frequency of the CPU, so that the electronic device can process subsequent image data based on the increased frequency of the CPU. Since the electronic device can increase the frequency of the CPU when the loading status of the first image data is a slow loading status, after the frequency of the CPU is increased, the speed of loading the image data by the electronic device can be increased, so that when the speed of loading multimedia resources by the electronic device is slow, the electronic device can avoid displaying a white image for a long time, thereby improving the user experience.

[0295] Corresponding to the method in the above-mentioned embodiment, the embodiment of the present application further provides a display device. The display device can be applied to an electronic device to implement the method in the above-mentioned embodiment. The function of the display device can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-mentioned functions.

[0296] For example, Figure 8 A schematic diagram of the structure of a display device 800 is shown. Figure 8 As shown, the display device 800 may include: an acquisition module 801, a determination module 802, a loading module 803, a display module 804, and an adjustment module 805, etc.

[0297] The acquisition module 801 may be used to acquire first image data of a first application when the electronic device runs the first application.

[0298] The determination module 802 may be configured to determine a loading state of the first image data based on the first image data; the loading state includes a slow loading state and a non-slow loading state.

[0299] The loading module 803 may be used to load the first image data based on a first frequency when the loading state of the first image data is not a slow loading state; the first frequency is an initial frequency of a central processing unit (CPU) of the electronic device.

[0300] The display module 804 may be used to display the first image data loaded by the loading module.

[0301] The adjustment module 805 may be used to adjust the CPU frequency to a second frequency when the loading state of the first image data is a slow loading state; the second frequency is greater than the first frequency.

[0302] The acquisition module 801 can also be used to acquire second image data of the first application; the second image data is a frame of image data subsequent to the first image data.

[0303] The loading module 803 may also be used to load the second image data based on the second frequency.

[0304] The display module 804 may also be used to display the second image data loaded by the loading module.

[0305] In another possible implementation, the acquisition module 801 may also be used to acquire histogram data of third image data; the third image data is a frame of image data previous to the first image data.

[0306] The acquisition module 801 may also be used to acquire histogram data of the first image data.

[0307] The determination module 802 may also be configured to determine a loading status of the first image data based on the histogram data of the first image data and the histogram data of the third image data.

[0308] In another possible implementation, the determination module 802 may also be configured to determine the image data change value and the duration of the image data change according to the histogram data of the first image data and the histogram data of the third image data.

[0309] The determination module 802 may also be used to determine that the loading state of the first image data is a slow loading state when the image data change value is less than a change threshold and the image data change duration is greater than a duration threshold.

[0310] In another possible implementation, the determination module 802 may also be used to determine that the loading state of the first image data is a normal loading state when the image data change value is greater than a change threshold and the image data change duration is greater than a duration threshold.

[0311] The determination module 802 may also be configured to determine that the loading state of the first image data is a loading waiting state when the image data change value is less than a change threshold and the image data change duration is less than a duration threshold.

[0312] In another possible implementation, the loading module 803 may also be configured to load the second image data based on the first frequency when the loading state of the first image data is a normal loading state or a loading waiting state.

[0313] The display module 804 may also be used to display the loaded second image data.

[0314] In another possible implementation, the determination module 802 may also be used to determine a loading path for the first image data based on the first image data; the loading path includes a graphics processor GPU of the electronic device and a hardware synthesizer HWC of the electronic device.

[0315] The acquisition module 801 may also be used to perform histogram detection on the first image data through the GPU to obtain the histogram data of the first image data when the loading path is the GPU.

[0316] The acquisition module 801 may also be used to acquire the histogram data of the first image data from the histogram database of the electronic device through the HWC when the loading path is the HWC.

[0317] In another possible implementation, the determination module 802 may also be used to determine a loading path for the first image data based on at least one of the processing requirements of the first application for the first image data, attribute information of the first image data, and hardware capabilities of the electronic device.

[0318] In another possible implementation, the acquisition module 801 may also be used to acquire fourth image data of the first application; the fourth image data is a subsequent frame of image data of the second image data.

[0319] The determination module 802 may also be configured to determine a loading status of the fourth image data based on the fourth image data.

[0320] The adjustment module 805 may also be configured to adjust the CPU frequency from the second frequency to the first frequency when the loading state of the fourth image data is not in the slow loading state.

[0321] The acquisition module 801 can also be used to acquire fifth image data of the first application; the fifth image data is a frame of image data subsequent to the fourth image data.

[0322] The loading module 803 may also be configured to load the fifth image data based on the first frequency.

[0323] The display module 804 may also be used to display the loaded fifth image data.

[0324] In another possible implementation, the determination module 802 may also be used to determine a scene of the first application based on the first image data; the scene of the first application includes a scene requiring histogram detection and a scene not requiring histogram detection.

[0325] The determination module 802 may also be configured to determine the loading status of the first image data based on the first image data when the scenario of the first application program is a scenario requiring histogram detection.

[0326] It should be understood that the division of units or modules (hereinafter referred to as units) in the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or physically separated. And the units in the device can all be implemented in the form of software calling through processing elements; or all in the form of hardware; or some units can be implemented in the form of software calling through processing elements, and some units can be implemented in the form of hardware.

[0327] For example, each unit can be a separately established processing element, or it can be integrated in a certain chip of the device. In addition, it can also be stored in a memory in the form of a program, and called and executed by a certain processing element of the device. In addition, all or part of these units can be integrated together, or they can be implemented independently. The processing element described here can also be called a processor, which can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each unit above can be implemented by an integrated logic circuit of hardware in the processor element or in the form of software called by a processing element.

[0328] In one example, the unit in the above apparatus may be one or more integrated circuits configured to implement the above method, such as one or more ASICs, or one or more DSPs, or one or more FPGAs, or a combination of at least two of these integrated circuit forms.

[0329] For another example, when the units in the device can be implemented in the form of a processing element scheduling program, the processing element can be a general-purpose processor, such as a CPU or other processor that can call a program. For another example, these units can be integrated together and implemented in the form of a system on a chip SOC.

[0330] In one implementation, the units of the above apparatus implementing the corresponding steps in the above method can be implemented in the form of a processing element scheduling program. For example, the apparatus may include a processing element and a storage element, and the processing element calls the program stored in the storage element to execute the method described in the above method embodiment. The storage element may be a storage element on the same chip as the processing element, that is, an on-chip storage element.

[0331] In another implementation, the program for executing the above method may be in a storage element on a different chip from the processing element, i.e., an off-chip storage element. In this case, the processing element calls or displays the program on the on-chip storage element from the off-chip storage element to call and execute the method described in the above method embodiment.

[0332] For example, an embodiment of the present application may also provide a device, such as an electronic device, which may include a processor and a memory for storing instructions executable by the processor. When the processor is configured to execute the above instructions, the electronic device implements the display method described in the above embodiment. The memory may be located inside the electronic device or outside the electronic device. And the processor includes one or more.

[0333] In another implementation, the unit of the device implementing each step in the above method may be configured as one or more processing elements, which may be arranged on the corresponding electronic device, and the processing element here may be an integrated circuit, for example: one or more ASICs, or one or more DSPs, or one or more FPGAs, or a combination of these integrated circuits. These integrated circuits may be integrated together to form a chip.

[0334] For example, the present application also provides a chip, which can be applied to the above electronic device. The chip includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected through a line; the processor receives and executes computer instructions from the memory of the electronic device through the interface circuit to implement the method described in the above method embodiment.

[0335] An embodiment of the present application also provides a computer program product, including computer instructions executed by the above-mentioned electronic device.

[0336] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0337] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0338] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0339] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0340] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product, such as a program. The software product is stored in a program product, such as a computer-readable storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0341] For example, the embodiment of the present application may also provide a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by an electronic device, the electronic device implements the display method described in the above method embodiment.

[0342] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A display method, characterized in that: Applied to electronic equipment, the method comprises: When the electronic device runs a first application, acquiring first image data of the first application; Based on the first image data, determining the loading state of the first image data; the loading state includes a slow loading state and a non-slow loading state; the determining the loading state of the first image data based on the first image data includes: obtaining the histogram data of the third image data; the third image data is the image data of the previous frame of the first image data; obtaining the histogram data of the first image data, and determining the loading state of the first image data based on the histogram data of the first image data and the histogram data of the third image data; the histogram data includes: the bin value in the histogram corresponding to the image data to which the histogram data belongs, and the number of pixels corresponding to each bin value; the histogram corresponding to the image data to which the histogram data belongs is used to characterize the number of each pixel value in the image data to which the histogram data belongs; the bin value in the histogram is the distribution range of the color in the color space corresponding to the image data to which the histogram data belongs; When the loading state of the first image data is the non-slow loading state, the first image data is loaded and displayed based on a first frequency; the first frequency is an initial frequency of a central processing unit (CPU) of the electronic device; When the loading state of the first image data is the slow loading state, adjusting the frequency of the CPU to a second frequency; the second frequency is greater than the first frequency; Acquire second image data of the first application; the second image data is a frame of image data subsequent to the first image data; Based on the second frequency, the second image data is loaded and displayed.

2. The method according to claim 1, characterized in that: The determining the loading status of the first image data based on the histogram data of the first image data and the histogram data of the third image data includes: Determine an image data change value and a duration of image data change according to the histogram data of the first image data and the histogram data of the third image data; When the image data change value is smaller than the change threshold and the image data change duration is greater than the duration threshold, it is determined that the loading state of the first image data is the slow loading state.

3. The method according to claim 2, characterized in that The non-slow loading state includes a normal loading state and a loading waiting state, and the determining the loading state of the first image data based on the histogram data of the first image data and the histogram data of the third image data further includes: When the image data change value is greater than the change threshold, and the image data change duration is greater than the duration threshold, determining that the loading state of the first image data is the normal loading state; When the image data change value is smaller than the change threshold, and the image data change duration is smaller than the duration threshold, it is determined that the loading state of the first image data is the loading waiting state.

4. The method according to claim 3, characterized in that The method further comprises: When the loading state of the first image data is the loading normal state or the loading waiting state, the second image data is loaded and displayed based on the first frequency.

5. The method according to any one of claims 1 to 4, characterized in that The acquiring the histogram data of the first image data comprises: Determine a loading path of the first image data according to the first image data; the loading path includes a graphics processor GPU of the electronic device and a hardware synthesizer HWC of the electronic device; In a case where the loading path is the GPU, performing a histogram detection on the first image data by the GPU to obtain histogram data of the first image data; In a case where the loading path is the HWC, the histogram data of the first image data is acquired from a histogram database of the electronic device through the HWC.

6. The method according to claim 5, characterized in that The step of determining a loading path of the first image data according to the first image data includes: A loading path for the first image data is determined according to at least one of a processing requirement of the first application for the first image data, attribute information of the first image data, and a hardware capability of the electronic device.

7. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Acquire fourth image data of the first application; the fourth image data is a frame of image data subsequent to the second image data; Based on the fourth image data, determining a loading state of the fourth image data; When the loading state of the fourth image data is the non-slow loading state, adjusting the frequency of the CPU from the second frequency to the first frequency; Acquire fifth image data of the first application; the fifth image data is a frame of image data subsequent to the fourth image data; Based on the first frequency, the fifth image data is loaded and displayed.

8. The method according to any one of claims 1 to 4, characterized in that The determining, based on the first image data, a loading status of the first image data includes: Based on the first image data, determining a scene of the first application; the scene of the first application includes a scene requiring histogram detection and a scene not requiring histogram detection; In a case where the scenario of the first application is a scenario requiring histogram detection, a loading state of the first image data is determined based on the first image data.

9. An electronic device, characterized in that: The electronic device comprises a processor and a memory for storing instructions executable by the processor; when the processor is configured to execute the instructions, the electronic device implements the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having computer program instructions stored thereon; characterized in that: When the computer program instructions are executed by an electronic device, the electronic device implements the method according to any one of claims 1 to 8.

Images