Control display method and device, storage medium and chip

Abstract

The invention relates to a control display method and device, a storage medium and a chip, and relates to the technical field of image processing.The method comprises the steps that in response to a received display instruction of a control, a synthesis thread is newly built, and the synthesis thread is used for obtaining target image data corresponding to the control; and rendering and displaying the control according to the target image data and the data corresponding to the control. The synthesis thread is newly established, so that computing resources of other threads do not need to be occupied, the target image data can be acquired more quickly, required data can be provided for control rendering more quickly, and the speed and efficiency of acquiring the control are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a control display method, device, storage medium and chip. Background Art

[0002] As electronic devices display more and more content, in order to improve the user experience, richer display effects can be given to the displayed content. For example, the background of the control can be blurred. However, in order to achieve such richer display effects, electronic devices need to spend more processing time, and the relevant technology cannot take into account both rich display effects and high processing efficiency. Summary of the invention

[0003] In order to overcome the problems existing in the related art, the present disclosure provides a control display method, device, storage medium and chip.

[0004] According to a first aspect of an embodiment of the present disclosure, a control display method is provided, including:

[0005] In response to receiving a display instruction of the control, creating a new synthesis thread, wherein the synthesis thread is used to obtain target image data corresponding to the control;

[0006] The control is rendered and displayed according to the target image data and the data corresponding to the control.

[0007] Optionally, in response to receiving a display instruction of the control, creating a new synthesis thread includes:

[0008] In response to receiving a display instruction of the control, sending an image data request to a display composition system;

[0009] According to a response message of the display synthesis system to the image data request, target image data corresponding to the control is obtained, wherein the response message includes the target image data synthesized by a newly created synthesis thread of the display synthesis system according to image data corresponding to multiple levels below the level of the control.

[0010] Optionally, rendering and displaying the control according to the target image data and the data corresponding to the control includes:

[0011] Processing the target image data to generate a container image;

[0012] According to the rendering mode of the control, the data corresponding to the control is processed to obtain a corresponding rendering image;

[0013] The container image and the rendering image are rendered according to the color mixing mode to display the control.

[0014] Optionally, the processing of data corresponding to the control according to the rendering mode of the control to obtain a corresponding rendered image includes:

[0015] When the rendering mode of the control is an off-screen rendering mode, marking the control;

[0016] Inserting a marked control in a non-off-screen rendering process, and drawing the rendered image according to data corresponding to the control;

[0017] The rendering of the container image and the rendering image according to the color mixing mode to display the control includes:

[0018] In the color mixing mode, the marked control is rendered and displayed according to the container image and the rendering image.

[0019] Optionally, the processing the target image data to generate a container image includes:

[0020] In the case that the control is a non-irregular-shaped control, the container image is generated according to the target image data and the position information of the non-irregular-shaped control, and the container image is the smallest rectangular image that can accommodate the non-irregular-shaped control.

[0021] Optionally, the processing the target image data to generate a container image includes:

[0022] In the case where the control is a special-shaped control, a background image is obtained according to the target image data and the position information of the control, wherein the background image is an image capable of accommodating the special-shaped control;

[0023] According to the data corresponding to the special-shaped control, a container image having the same size as the special-shaped control is determined from the background image.

[0024] Optionally, the color mixing method includes at least one of the following:

[0025] Overlaying the container image with an image in the data corresponding to the control;

[0026] The image in the data corresponding to the control is replaced with the container image.

[0027] Optionally, the processing the target image data to generate a container image includes:

[0028] The container image is generated by performing blurring and / or color mixing processing according to the target image data.

[0029] Optionally, acquiring target image data corresponding to the control according to a response of the display composition system to the image data request includes:

[0030] Setting the priority of the receiving thread corresponding to the control to a preset receiving priority;

[0031] Based on the preset receiving priority, the target image data corresponding to the control synthesized by the newly created synthesis thread of the display synthesis system is received.

[0032] Optionally, rendering and displaying the control according to the target image data and the data corresponding to the control includes:

[0033] Setting the rendering thread priority corresponding to the control to a preset rendering priority;

[0034] Based on the preset rendering priority, the control is displayed after rendering according to the target image data and the data corresponding to the control.

[0035] Optionally, in response to receiving a display instruction of the control, sending an image data request to a display composition system includes:

[0036] In response to a display instruction of the control, determining whether the control needs to be displayed with a preset display effect;

[0037] When it is determined that the control needs to be displayed with a preset display effect, sending an image data request to the display synthesis system;

[0038] The method further comprises:

[0039] When it is determined that the control does not need to be displayed with the preset display effect, rendering and displaying the control according to the data corresponding to the control;

[0040] A stop notification is sent to the display composition system, so that the display composition system stops sending the target image data to the corresponding control.

[0041] According to a second aspect of an embodiment of the present disclosure, there is provided a control display device, comprising:

[0042] An image data processing module is configured to create a new synthesis thread in response to receiving a display instruction of the control, wherein the synthesis thread is used to obtain target image data corresponding to the control;

[0043] The control processing module is configured to render and display the control according to the target image data and the data corresponding to the control.

[0044] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0045] processor;

[0046] a memory for storing processor-executable instructions;

[0047] The processor is configured to execute the steps of any one of the methods described in the first aspect.

[0048] According to a fourth aspect of an embodiment of the present disclosure, there is provided a computer-readable storage medium on which computer program instructions are stored, and when the program instructions are executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0049] According to a fifth aspect of an embodiment of the present disclosure, a chip is provided, comprising a processor and an interface; the processor is used to read instructions to execute any one of the methods described in the first aspect.

[0050] The technical solution shown in the embodiments of the present disclosure may have the following beneficial effects:

[0051] In the embodiment of the present disclosure, after receiving the display instruction of the control, in response to the instruction, the target image data corresponding to the control is obtained by creating a new synthesis thread. The control can be rendered according to the target image data and the data corresponding to the control. Since the synthesis thread is newly created, it does not need to occupy the computing resources of other threads, and the target image data can be obtained faster, providing the required data for the control rendering faster, thereby improving the speed and efficiency of obtaining the control.

[0052] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0054] Figure 1 The figure is a flowchart of a control display method according to an exemplary embodiment.

[0055] Figure 2 The diagram is a schematic diagram showing a control displayed on an electronic device according to an exemplary embodiment.

[0056] Figure 3 The figure is a flowchart showing steps of creating a new synthesis thread according to an exemplary embodiment.

[0057] Figure 4 The present invention is a flowchart showing steps of rendering a display control according to an exemplary embodiment.

[0058] Figure 5The figure is a flowchart showing steps of obtaining a rendered image according to an exemplary embodiment.

[0059] Figure 6 The present invention is a flowchart of steps for generating a container image according to an exemplary embodiment.

[0060] Figure 6a The diagram is a schematic diagram showing a control displayed on an electronic device according to an exemplary embodiment.

[0061] Figure 7 The present invention is a flowchart showing the steps of sending target image data to a control according to an exemplary embodiment.

[0062] Figure 8 The present invention is a flowchart showing steps of rendering a display control according to an exemplary embodiment.

[0063] Fig. 9 The present invention is a flowchart showing the steps of sending an image data request to a display composition system according to an exemplary embodiment.

[0064] Fig.10 The figure is a block diagram of a control display device according to an exemplary embodiment.

[0065] Fig.11 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0066] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0067] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0068] Electronic devices can display various controls on the display screen to display various information to users, or can receive user input operations through controls to trigger corresponding functions. In the related art, controls can only be displayed in a fixed pre-set display mode during the display process, and the user viewing experience is weak. The inventors found that in some scenarios, if multiple controls need to be displayed at the same time, and multiple controls need to be displayed as their respective corresponding effects, the electronic device may not be able to provide the resources required for display of multiple controls at the same time due to the large number of display controls, resulting in display freezes and reduced user viewing experience.

[0069] In view of this, an embodiment of the present disclosure provides a control display method. Figure 1 FIG. 1 is a flowchart of a control display method according to an exemplary embodiment. Figure 1 As shown, the control display method includes the following steps S10 to S20.

[0070] In step S10, in response to receiving a display instruction of the control, a new synthesis thread is created, and the synthesis thread is used to obtain target image data corresponding to the control.

[0071] The display instructions for the controls can be generated by the electronic device itself, or generated by the electronic device after receiving the user's input operation. For example, when certain services in the electronic device receive new data, they can prompt the user in the form of display controls on the display screen. Before these controls that need to be displayed are displayed, display instructions for these controls can be generated. Alternatively, there are certain controls in the electronic device that will only be generated after receiving the user's specified operation. Such controls can generate display instructions for these controls after the user performs the specified operation. The embodiments of the present disclosure do not limit the method of displaying controls and the form of generating display instructions.

[0072] After generating a display instruction for a control, in response to the display instruction, the electronic device may create a new synthesis thread, and the synthesis thread may obtain target image data corresponding to the control. The number of controls to be displayed may be one or more, and multiple controls may be displayed on the display screen simultaneously or sequentially. In the case where multiple threads need to be displayed, the synthesis thread may obtain target image data corresponding to each control according to the display instruction of each control.

[0073] The target image data can be the data that provides the background image for the control during the rendering process. The control can obtain the background image for the final display from the corresponding target image data. The target image data can be obtained by synthesizing the image data corresponding to multiple levels below the level where the control is located. The image data corresponding to the multiple levels below the level where the control is located is different. At the same time, the target image data is different when the control is displayed at different locations.

[0074] For easier understanding, see Figure 2 , Figure 2 is a schematic diagram showing a control displayed on an electronic device according to an exemplary embodiment. Figure 2 In the example, the electronic device displays multiple controls in the currently displayed window, including control 100A, control 100B, and control 100C. Control 100A, control 100B, and control 100C can be displayed on the display screen simultaneously or sequentially. Before a control is displayed, a display instruction for the control is generated, and target image data corresponding to each control can be obtained according to the display instruction.

[0075] exist Figure 2 In the example, the wallpaper of the current window is a portrait, and the background of the control 100A includes a blurred image of the upper half of the portrait's head. Then the target image data corresponding to the control 100A can include an image of the upper half of the portrait's head. After synthesizing the image data of multiple levels below the location of the control 100A in the wallpaper, the target image data corresponding to the control 100A is obtained.

[0076] Similarly, Figure 2 The background of the control 100B in the figure includes the lower half of the head of the portrait. After synthesizing the image data of multiple levels below the position of the control 100B, the target image data corresponding to the control 100B can be obtained. It can be understood that the corresponding target image data can also be obtained for the control 100C.

[0077] In a possible implementation, when acquiring the target image data, the synthesis thread may perform synthesis in parallel based on the image data of multiple levels under each control. The embodiment of the present disclosure does not limit the order of acquiring the target image data.

[0078] In step S20, the control is rendered and displayed according to the target image data and the data corresponding to the control.

[0079] After obtaining the target image data corresponding to the control, the control can be rendered and displayed. The data corresponding to the control may include the text, image and other data that the control needs to display. Each control can obtain a background image that matches the control according to the target image data. The data corresponding to the control is combined with the background image to obtain a complete control.

[0080] For example Figure 2 As shown, after the control 100A that has not been rendered and displayed obtains the target image data including the upper half of the portrait's head, it is combined with other text and image data corresponding to the control 100A to render the complete control 100A, which can be displayed on the display screen. The multiple lines of text data in the control 100A are the data corresponding to the control 100A. The target image data and the data corresponding to the control can be rendered together to obtain the control, and the rendered control is displayed on the display screen. In this way, the complete control is displayed on the display screen of the electronic device. Based on a similar process, other controls can be rendered, and multiple different controls can be displayed on the display screen at the same time.

[0081] Thus, in the embodiment of the present disclosure, after receiving the display instruction of the control, in response to the instruction, the target image data corresponding to the control is obtained by creating a new synthesis thread. The control can be rendered according to the target image data and the data corresponding to the control. Since the synthesis thread is newly created, it does not need to occupy the computing resources of other threads, and the target image data can be obtained faster, providing the required data for the rendering of the control faster, thereby improving the speed and efficiency of obtaining the control.

[0082] At the same time, the rendering process of the control is not only based on the data corresponding to the control, but also based on the image data of multiple levels below the level where the control is located, so that the control has a display effect based on the images of multiple levels below the level where it is located. The rendering and display process of the control takes into account the rendering speed and efficiency while also having rich display effects.

[0083] Figure 3 FIG. 1 is a flowchart of steps for rendering a display control according to an exemplary embodiment. Figure 3 As shown, in a possible implementation, in response to receiving a display instruction of a control, creating a new synthesis thread may include steps S101 and 102.

[0084] In step S101 , in response to receiving a display instruction of a control, an image data request is sent to a display composition system.

[0085] In the embodiment of the present disclosure, when it is necessary to obtain the target image data corresponding to the control, after receiving the display instruction of the control, in response to the instruction, an image data request can be sent to the display synthesis system. In a possible implementation, the display synthesis system can be a display synthesis process (Surface flinger) process in the electronic device or other processes that can obtain image data. Controls in different locations require different target image data, and different image data requests can be sent to the display synthesis system.

[0086] In step S102, target image data corresponding to the control is acquired according to a response message of the display composition system to the image data request.

[0087] The response message may include the target image data synthesized by the newly created synthesis thread of the display synthesis system according to the image data corresponding to the multiple levels below the level of the control. The display synthesis system responds to the image data request, that is, through the synthesis thread, synthesizes the image data corresponding to the multiple levels below the level of the control, and the target image data can be obtained after synthesis.

[0088] In the disclosed embodiment, the display synthesis system can not only improve the overall speed of the control display by creating a new synthesis thread, but also can obtain multiple target image data corresponding to multiple controls in parallel during the process of synthesizing the target image data, and send the obtained multiple target image data to the controls in parallel, thereby further improving the overall speed of the control display.

[0089] Figure 4 FIG. 1 is a flowchart of steps for rendering a display control according to an exemplary embodiment. Figure 4 As shown, in a possible implementation, the step of rendering and displaying the control according to the target image data and the data corresponding to the control may include the following steps S201 to S203.

[0090] In step S201 , the target image data is processed to generate a container image.

[0091] After acquiring the target image data, the target image data may be processed to obtain the container image. For example, the target image data may be blurred based on a blur algorithm, or the target image data may be mixed. Of course, the target image data may be blurred first, and then the blurred target image data may be mixed. The disclosed embodiment does not limit the processing method of the target image data, which may be determined according to actual needs.

[0092] When blurring the target image data, the blurring algorithm can also be determined according to actual needs. For example, for electronic devices with strong hardware capabilities, the Gaussian Blur algorithm can be used to blur the target image data, thereby obtaining a relatively good blurring effect. For electronic devices with weak hardware capabilities, the Kawase Blur algorithm can be used to blur the target image data, thereby improving the speed and efficiency of blurring.

[0093] In addition, the method of blurring the target image data may also include blur algorithms such as Box Blur, Dual Blur, Bokeh Blur, TiltShift Blur, Iris Blur, Grainy Blur, Radial Blur, and Directional Blur. The embodiments of the present disclosure do not limit the algorithm selected for blurring.

[0094] By processing the target image data, a container image can be obtained, and the container image can be used as the material and basis for drawing controls, and further different controls can be obtained.

[0095] In step S202, data corresponding to the control is processed according to the rendering mode of the control to obtain a corresponding rendering image.

[0096] After processing the container image, different controls can be rendered based on different container images and combined with the data corresponding to the controls. In the process of rendering the controls, different controls have different rendering methods and color mixing methods, and their rendering processes are also different. In the embodiments of the present disclosure, even if the final presentation content of the controls is different, in order to ensure that all controls or part of the controls that are finally displayed have the same or similar display effects, different controls can be rendered based on their respective corresponding container images. For example Figure 2 As shown, the text contents finally displayed by the three controls shown in the figure, control 100A, control 100B and control 100C are completely different, but the backgrounds of the three controls have similar display effects. The backgrounds of the three controls are all images after blurring the wallpaper image. It can be understood that the control finally displayed is related to the position of the control, and the backgrounds of controls at different positions are different.

[0097] In the embodiment of the present disclosure, the background finally displayed by the control is obtained based on the container image. The container image can be regarded as an abstract concept in the embodiment of the present disclosure. The container image is not directly displayed on the display screen, but is used as a material in the process of drawing and rendering the control, and as a background image in the process of rendering the control.

[0098] In a possible implementation, the rendering mode of the control may include an off-screen rendering mode and a non-off-screen rendering mode. Different rendering modes have different rendering processes. According to the rendering mode of the control, based on the data corresponding to the control, a rendering image corresponding to the control can be obtained. The rendering image can be rendered together with the container image to obtain a complete control.

[0099] In step S203, the container image and the rendering image are rendered according to the color mixing method to display the control.

[0100] After rendering to obtain the rendered image, it is then re-rendered with the container image based on the color mixing method of the control to obtain a complete control. In a possible implementation, the color mixing method of the control may include superimposing the container image with the rendered image, that is, superimposing the container image with the data corresponding to the control, or the color mixing method of the control may include replacing the rendered image with the container image, that is, replacing the data corresponding to the control with the container image. The rendering method and color mixing method of the control can be predetermined according to the specific function of the control. After receiving the corresponding target image data, in the process of drawing and rendering the control, the control is drawn and rendered accordingly according to the rendering method and color mixing method of the control, so as to obtain the final displayable control.

[0101] It should be noted that different color mixing methods can produce controls with different display effects. For the color mixing method of superimposing the container image with the data corresponding to the control, please refer to Figure 2 , Figure 2 The text of the control 100A can be a rendering result obtained according to the data corresponding to the control 100A, that is, a rendered image. The background of the control 100A is the container image. The container image is superimposed with the data corresponding to the control, and the effect of displaying the text on the basis of a blurred or mixed background is obtained. The container image of the control is an image that is synthesized from the image data of multiple levels below the level of the control 100A and then blurred or mixed. In the display result on the display screen, the container image of the control 100A only blurs the upper half of the head of the portrait. However, during the control display process, the container image corresponding to the control 100A may include not only part of the portrait in the wallpaper, but also one or more layers of image data under the wallpaper. However, in the display result on the display screen, the effect viewed by the user is part of the portrait in the wallpaper.

[0102] The content that the control needs to display is not displayed on the display screen before it is rendered together with the container image. However, in a possible implementation, the content that the control needs to display can be preliminarily determined first, and then the data corresponding to the preliminarily determined control can be adjusted according to the color mixing method. For example, it can be determined first that the image of the background of the control is a blue image. This parameter is preliminarily determined as the data corresponding to the control, and the color mixing method of the control is preliminarily determined to replace the image in the data corresponding to the control with the container image. Then, during the drawing and rendering process of the control, the blue image of the control background can be replaced with the container image according to the color mixing method of the control.

[0103] In this way, a container image can be obtained by processing the target image data. As mentioned above, the container image can be an image that can contain a control and serve as the background in the control rendering process. The control can process the container image according to its corresponding rendering method and color mixing method to obtain the final control displayed on the display screen. By setting the color mixing method of the control, different controls can have similar display effects, so that electronic devices can display controls with richer effects and more uniform display effects, improve the user's perception, and enhance the user's experience.

[0104] Figure 5 FIG. 1 is a flowchart of steps for obtaining a rendered image according to an exemplary embodiment. Figure 5 As shown, in a possible implementation, the step of processing the data corresponding to the control according to the rendering mode of the control to obtain the corresponding rendered image may include the following steps S221 and S222.

[0105] In step S221, when the rendering mode of the control is off-screen rendering mode, the control is marked.

[0106] As mentioned above, the rendering mode of the control may include an off-screen rendering mode and a non-off-screen rendering mode. In the related art, if the control is rendered in a normal off-screen rendering mode, due to the limitation of the rendering mechanism, the off-screen rendering cannot process the display content located below the control, and also cannot process the background image below the off-screen rendered control, so that the background of the control cannot be obtained as a blurred or mixed background image.

[0107] Therefore, in a possible implementation of the present disclosure, the off-screen rendering process is modified. Before rendering the control, the rendering mode of the control is determined first, and after determining that the rendering mode of the control is the off-screen rendering mode, the control is marked first, and the specific marking form is not limited.

[0108] In step S222, the marked control is inserted into the non-off-screen rendering process, and a rendering image is drawn according to the data corresponding to the control.

[0109] After a control whose rendering mode is off-screen rendering mode is marked, the original off-screen rendering drawing process can be skipped, that is, the marked control is not rendered off-screen, but the marked control is inserted into the non-off-screen rendering process, and the control is rendered non-off-screen. Due to the setting of the rendering mechanism, during the non-off-screen rendering process, the content below the control can be drawn and rendered, that is, the container image where the control is located can be drawn and rendered, and then based on the control's color mixing method, a control whose background image is the container image can be obtained. It can be understood that if the rendering mode of the control is non-off-screen rendering, then the control can be drawn and rendered according to the established non-off-screen rendering process.

[0110] Correspondingly, in the case where the rendering mode of the control is an off-screen rendering mode, the step of rendering the container image and the rendering image according to the color mixing mode and displaying the control may include step S231.

[0111] In step S231, in a color mixing mode, the marked control is rendered and displayed according to the container image and the rendering image.

[0112] As mentioned above, after the rendered image is obtained according to the data rendering corresponding to the control, the rendered image needs to be rendered together with the container image to obtain a complete control. The color mixing method of the control may include superimposing the container image with the rendered image, or the color mixing method of the control may include replacing the rendered image with the container image. In the case where the rendering method of the control is an off-screen rendering method, the container image and the rendered image corresponding to the marked control can be processed to superimpose the rendered image with the container image or replace the rendered image with the container image to obtain the final control. It can be understood that in the case where the rendering method of the control is non-off-screen rendering, the final control can be obtained directly according to the container image and the rendered image based on the color mixing method of the control.

[0113] In the disclosed embodiment, before drawing and rendering the controls, they can be classified according to their rendering modes. For controls whose rendering mode is off-screen rendering, their drawing and rendering process can be changed. By marking the controls and skipping the established off-screen rendering process, the controls can be rendered non-off-screen, so that the controls that originally needed to be rendered off-screen can be rendered non-off-screen, thereby avoiding the problem of not being able to obtain the container image under the controls in the off-screen rendering process. The controls whose rendering mode is off-screen rendering can also obtain the container image obtained based on the target image data processing, so that the background based on the container image can be obtained, thereby improving the display effect of the controls and enhancing the user's viewing experience.

[0114] In a possible implementation manner, the step of processing the target image data to generate a container image may include step S211.

[0115] In step S211, when the control is a non-irregular-shaped control, a container image is generated according to the target image data and the position information of the non-irregular-shaped control, and the container image is the smallest rectangular image that can accommodate the non-irregular-shaped control.

[0116] If controls are classified based on the rendering method, as mentioned above, controls can be divided into off-screen rendering controls and non-off-screen rendering controls. If controls are classified based on their shapes, controls can be divided into special-shaped controls and non-special-shaped controls. Special-shaped controls are controls that are not regular shapes, such as controls that are non-rectangular or non-rectangular, such as controls in the shape of clouds or five-pointed stars. Non-special-shaped controls can be the shape of a rectangle with chamfered corners.

[0117] For example Figure 2 As shown, Figure 2 The shapes of the controls 100A, 100B, and 100C are all chamfered rectangles. From the perspective of shape, these three controls are all non-special-shaped controls. For such non-special-shaped controls, the container image is the smallest rectangular image that can accommodate the controls. As mentioned above, the container image is the target image data that has been blurred or mixed. Based on such a rectangular image, an image of the same size as the non-special-shaped control can be obtained as the background of the non-special-shaped control. Combined with the color mixing method of the non-special-shaped control, a complete non-special-shaped control can be obtained.

[0118] In another possible implementation, Figure 6 FIG. 1 is a flowchart of steps for generating a container image according to an exemplary embodiment. Figure 6 As shown, the step of processing the target image data to generate a container image may include the following steps S212 and S213.

[0119] In step S212, when the control is a special-shaped control, a background image is obtained according to the target image data and the position information of the control, and the background image is an image that can accommodate the special-shaped control.

[0120] In step S213, according to the data corresponding to the control, a container image having the same size as the special-shaped control is determined from the background image.

[0121] Different from the process of obtaining a container image for a non-special-shaped control, to obtain a container image corresponding to a special-shaped control, you can first obtain a background image that can accommodate the special-shaped control, and then determine the container image corresponding to the special-shaped control from the background image.

[0122] In this embodiment, after obtaining the background image, the image of the control can be superimposed with the background image by superposition to determine the overlapping area between the two. In the superposition process, the overlapping area is maximized as the goal, and the overlapping area in the background image is retained, which can be used as the container image of the special-shaped control. For non-special-shaped controls, the smallest rectangle that can accommodate the control at the position of the control in the background image can be used as the container image.

[0123] Figure 6a FIG. 1 is a schematic diagram showing a special-shaped control displayed on an electronic device according to an exemplary embodiment. Figure 6a As shown, Figure 6a The control 100D shown in the figure is a special-shaped control. The shape of the control 100D is irregular. The control 100D is the time on the display interface. Figure 6a The container image is not shown, but it can be understood that the container image can be an image of a part of the wallpaper under the displayed time. First, the background image that can accommodate the control 100D is determined through the location information of the control 100D and the target image data, and then the image of the background image with the same size as the control 100D is used as the container image. Figure 6a The number "9" in the second row of the image shows part of the wallpaper in a blurred effect, and this blurred effect can be part of the background image of the control 100D. In this way, the background of the special-shaped control is displayed as a blurred or mixed color image under the control, which improves the display effect of the control.

[0124] In this way, a blurred or mixed color container image of the control can be obtained according to the target image data, so that different controls can be displayed with the same or similar display effects, improving the consistency of the display effects of different controls on the display screen and improving the user's viewing experience.

[0125] Figure 7 FIG. 1 is a flow chart showing a method of sending target image data to a control according to an exemplary embodiment. Figure 7As shown, in a possible implementation, the step of acquiring the target image data corresponding to the control according to the response of the display composition system to the image data request may include the following steps S121 and S122.

[0126] In step S121, the priority of the receiving thread corresponding to the control is set to a preset receiving priority.

[0127] In order to improve the speed and efficiency of displaying multiple controls, in the embodiment of the present disclosure, the target image data can be sent to the control in parallel. In addition, when the control receives the target image data, if the priority of the receiving thread is low, the transmission data may be blocked, so that the control cannot receive the target image data in time, which will also reduce the speed of the control drawing and display. Therefore, in the embodiment of the present disclosure, the priority of the receiving thread of the control receiving the target image data can be set to a preset receiving priority, and the preset receiving priority can be set to a higher or highest priority, which can ensure the timeliness of receiving data and prevent the transmission data from being blocked.

[0128] In step S122, based on the preset receiving priority, the receiving display composition system sends the target image data corresponding to the widget in parallel in response to the multiple image data requests.

[0129] The receiving threads of multiple controls receiving target image data can be set to a preset receiving priority, and the preset receiving priority can be set to a higher priority, so that the target image data can be received quickly. The control can only start drawing and rendering after receiving the target image data. By setting the receiving data to be maintained at a higher priority, the speed of obtaining the control is accelerated, the efficiency of the control display on the display is improved, and the occurrence of freezes and the like is avoided.

[0130] Figure 8 FIG. 1 is a flowchart of a rendering display control according to an exemplary embodiment. Figure 8 As shown, similarly, in a possible implementation manner, rendering and displaying the control according to the target image data and the data corresponding to the control may also include the following steps S204 and S205.

[0131] In step S204, the priority of the rendering thread corresponding to the control is set to a preset rendering priority.

[0132] Not only can the receiving thread of the control receiving the target image data be kept at a higher priority, but also during the control rendering process, the rendering thread of the control can be increased to a preset higher preset rendering priority, thereby increasing the priority of the rendering control and speeding up the rendering speed.

[0133] In step S205, based on the preset rendering priority, the control is rendered and displayed according to the target image data and the image data corresponding to the control.

[0134] Based on the higher rendering priority, the rendering resources can be prevented from being preempted, and the control can be rendered faster. In addition, during the rendering process, the faster processing core in the GPU (Graphic Processing Unit) can be used for rendering to ensure the rendering speed of the control.

[0135] In this way, during the control display process, by keeping the priority of the receiving thread high and the priority of the rendering thread high, the control can be drawn and rendered faster, speeding up the display of the control on the display, avoiding display freezes caused by slow drawing and rendering speed, and improving the user's viewing and usage experience.

[0136] Fig. 9 FIG. 1 is a flowchart showing steps of sending an image data request to a display composition system according to an exemplary embodiment. Fig. 9 As shown, in a possible implementation, in response to receiving a display instruction of a control, the step of sending an image data request to a display composition system may include the following steps S111 to S114.

[0137] In step S111 , in response to receiving a display instruction of a control, it is determined whether the control needs to be displayed with a preset display effect.

[0138] After receiving a display instruction for a control, in response to the instruction, the purpose of first determining whether the control is displayed with a preset display effect is to determine whether it is necessary to obtain the target image data corresponding to the control. In the disclosed embodiment, the control needs to be displayed with a preset display effect, which means that the control needs to use the target image data or processed target image data as the display background. Alternatively, it can be understood that when the control needs to be displayed with a preset display effect, it needs to be rendered based on one of the above-mentioned color mixing methods and then displayed. Whether the control needs to be displayed with a preset display effect does not pre-set the background of the control, but only sets whether the control needs target image data.

[0139] The control needs to be displayed with the preset display effect, indicating that the target image data is required for rendering the control. The control does not need to be displayed with the preset display effect, indicating that the target image data is not required for rendering the control. When it is determined that the control needs to be displayed with the above-mentioned color mixing effect, it indicates that the image data of multiple levels below the control location needs to be processed, that is, the above-mentioned target image data is required for rendering the control.

[0140] In step S112, when it is determined that the control needs to be displayed with a preset display effect, an image data request is sent to the display synthesis system.

[0141] When it is determined that the rendering control needs to be displayed in the above-mentioned color mixing manner, an image data request can be sent to the display composition system to notify the display composition system to transmit the target image data to the control.

[0142] When the control does not need to be displayed with a preset display effect, step S113 and step S114 are executed.

[0143] In step S113, when it is determined that the control does not need to be displayed with a preset display effect, the control is rendered and displayed according to the data corresponding to the control.

[0144] In step S114, a stop notification is sent to the display composition system so that the display composition system stops sending the target image data to the corresponding control.

[0145] When the control does not need to be displayed with a preset display effect, or the control does not need to be rendered based on one of the above color mixing methods before displaying, it means that the control only needs to be rendered according to its own corresponding data, and does not need the target image data or the processed target image data as the background. The control can be directly drawn according to the data corresponding to the control or the control can be directly invisible. At this time, it is not necessary to send an image data request to the display synthesis system, thereby stopping the transmission of the target image data.

[0146] In this way, the target image data is acquired only when the control needs to be displayed with the preset display effect. When the control does not need to be displayed with the preset display effect, the target image data acquisition is stopped, so that the action of acquiring the target image data is controllable, providing a controllable mechanism for the fast drawing and rendering of the control.

[0147] Fig.10 is a block diagram of a control display device according to an exemplary embodiment. Fig.10 The control display device 1100 includes an image data processing module 1101 and a control processing module 1102, wherein:

[0148] The image data processing module 1101 is configured to create a new synthesis thread in response to receiving a display instruction of a control, wherein the synthesis thread is used to obtain target image data corresponding to the control;

[0149] The control processing module 1102 is configured to render and display the control according to the target image data and the data corresponding to the control.

[0150] Optionally, the image data processing module 1101 includes:

[0151] a request submodule, configured to send an image data request to a display composition system in response to receiving a display instruction of the control;

[0152] The image acquisition submodule is configured to acquire target image data corresponding to the control according to the response of the display synthesis system to the image data request, wherein the response is that a newly created synthesis thread of the display synthesis system synthesizes the target image data according to the image data corresponding to multiple levels below the level of the control.

[0153] Optionally, the control processing module 1102 includes:

[0154] a container image generation submodule, configured to process the target image data to generate a container image;

[0155] A rendering submodule is configured to process the data corresponding to the control according to the rendering mode of the control to obtain a corresponding rendered image;

[0156] The color mixing submodule is configured to render the container image and the rendering image according to the color mixing mode to display the control.

[0157] Optionally, the rendering submodule includes:

[0158] a marking unit, configured to mark the control when the rendering mode of the control is an off-screen rendering mode;

[0159] A non-off-screen rendering unit, configured to insert a marked control in the non-off-screen rendering process, and draw the rendered image according to data corresponding to the control;

[0160] The color mixing module comprises:

[0161] The color mixing unit is configured to render and display the marked control according to the container image and the rendering image in the color mixing mode.

[0162] Optionally, the container image generation submodule includes:

[0163] The non-irregular control processing unit is configured to generate the container image according to the target image data and the position information of the non-irregular control when the control is a non-irregular control. The container image is the smallest rectangular image that can accommodate the non-irregular control.

[0164] Optionally, the container image generation submodule includes:

[0165] The special-shaped control processing unit is configured to obtain a background image according to the target image data and the position information of the control when the control is a special-shaped control, wherein the background image is an image capable of accommodating the special-shaped control;

[0166] According to the data corresponding to the special-shaped control, a container image having the same size as the special-shaped control is determined from the background image.

[0167] Optionally, the color mixing module performs color mixing in at least one of the following ways:

[0168] Overlaying the container image with an image in the data corresponding to the control;

[0169] The image in the data corresponding to the control is replaced with the container image.

[0170] Optionally, the container image generation module is specifically configured to:

[0171] The container image is generated by performing blurring and / or color mixing processing according to the target image data.

[0172] Optionally, the image data processing module is specifically configured as follows:

[0173] Setting the priority of the receiving thread corresponding to the control to a preset receiving priority;

[0174] Based on the preset receiving priority, the target image data corresponding to the control synthesized by the newly created synthesis thread of the display synthesis system is received.

[0175] Optionally, the control processing module 1102 is specifically configured to:

[0176] Setting the rendering thread priority corresponding to the control to a preset rendering priority;

[0177] Based on the preset rendering priority, the control is displayed after rendering according to the target image data and the data corresponding to the control.

[0178] A determination submodule, configured to determine whether the control needs to be displayed with a preset display effect in response to a display instruction of the control;

[0179] Optionally, the image data processing module 1101 includes:

[0180] A first request processing submodule is configured to send an image data request to the display synthesis system when it is determined that the control needs to be displayed with a preset display effect;

[0181] The control processing module 1102 further includes:

[0182] A second request processing submodule is configured to render and display the control according to data corresponding to the control when it is determined that the control does not need to be displayed with the preset display effect;

[0183] The notification submodule is configured to send a stop notification to the display composition system so that the display composition system stops sending the target image data to the corresponding control.

[0184] Regarding the control display device 1100 in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0185] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, which implement the steps of the control display method provided by the present disclosure when the program instructions are executed by a processor.

[0186] Fig.11 8 is a block diagram of an electronic device for widget display according to an exemplary embodiment. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0187] Reference Fig.11 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output interface 812 , a sensor component 814 , and a communication component 816 .

[0188] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned control display method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0189] The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0190] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0191] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0192] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0193] The input / output interface 812 provides an interface between the processing component 802 and the peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0194] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the electronic device 800, the relative positioning of the components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0195] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0196] In an exemplary embodiment, the electronic device 800 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned control display method.

[0197] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by a processor 820 of an electronic device 800 to complete the control display method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0198] In addition to being an independent electronic device, the above-mentioned device can also be a part of an independent electronic device. For example, in one embodiment, the device can be an integrated circuit (IC) or a chip, wherein the integrated circuit can be an IC or a collection of multiple ICs; the chip can include but is not limited to the following types: GPU (Graphics Processing Unit), CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), SOC (System on Chip, SoC, system on chip or system-level chip), etc. The above-mentioned integrated circuit or chip can be used to execute executable instructions (or codes) to implement the above-mentioned control display method. The executable instructions can be stored in the integrated circuit or chip, or can be obtained from other devices or equipment, for example, the integrated circuit or chip includes a processor, a memory, and an interface for communicating with other devices. The executable instruction can be stored in the memory, and when the executable instruction is executed by the processor, the above-mentioned control display method is implemented; alternatively, the integrated circuit or chip can receive the executable instruction through the interface and transmit it to the processor for execution, so as to implement the above-mentioned control display method.

[0199] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program that can be executed by a programmable device. The computer program has a code portion for executing the above-mentioned widget display method applied to a widget when executed by the programmable device.

[0200] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0201] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A control display method, characterized in that: include: In response to receiving a display instruction of the control, creating a new synthesis thread, wherein the synthesis thread is used to obtain target image data corresponding to the control; The control is rendered and displayed according to the target image data and the data corresponding to the control.

2. The control display method according to claim 1, characterized in that: In response to receiving a display instruction of the control, creating a new synthesis thread includes: In response to receiving a display instruction of the control, sending an image data request to a display composition system; According to a response message of the display synthesis system to the image data request, target image data corresponding to the control is obtained, wherein the response message includes the target image data synthesized by a newly created synthesis thread of the display synthesis system according to image data corresponding to multiple levels below the level of the control.

3. The control display method according to claim 1, characterized in that: The rendering and displaying of the control according to the target image data and the data corresponding to the control includes: Processing the target image data to generate a container image; According to the rendering mode of the control, the data corresponding to the control is processed to obtain a corresponding rendering image; The container image and the rendering image are rendered according to a color mixing method to display the control.

4. The control display method according to claim 3, characterized in that: The step of processing the data corresponding to the control according to the rendering mode of the control to obtain a corresponding rendered image includes: When the rendering mode of the control is an off-screen rendering mode, marking the control; Inserting a marked control in a non-off-screen rendering process, and drawing the rendered image according to data corresponding to the control; The rendering of the container image and the rendering image according to the color mixing mode to display the control includes: In the color mixing mode, the marked control is rendered and displayed according to the container image and the rendering image.

5. The control display method according to claim 2, characterized in that: The step of processing the target image data to generate a container image includes: In the case that the control is a non-irregular-shaped control, the container image is generated according to the target image data and the position information of the non-irregular-shaped control, and the container image is the smallest rectangular image that can accommodate the non-irregular-shaped control.

6. The control display method according to claim 2, characterized in that: The step of processing the target image data to generate a container image includes: In the case where the control is a special-shaped control, a background image is obtained according to the target image data and the position information of the control, wherein the background image is an image capable of accommodating the special-shaped control; According to the data corresponding to the special-shaped control, a container image having the same size as the special-shaped control is determined from the background image.

7. The control display method according to claim 3, characterized in that: The color mixing method includes at least one of the following: Overlaying the container image with an image in the data corresponding to the control; The image in the data corresponding to the control is replaced with the container image.

8. The control display method according to claim 2, characterized in that: The step of processing the target image data to generate a container image includes: The container image is generated by performing blurring and / or color mixing processing according to the target image data.

9. The control display method according to claim 2, characterized in that: The acquiring target image data corresponding to the control according to the response of the display composition system to the image data request comprises: Setting the priority of the receiving thread corresponding to the control to a preset receiving priority; Based on the preset receiving priority, the target image data corresponding to the control synthesized by the newly created synthesis thread of the display synthesis system is received.

10. The control display method according to any one of claims 1 to 9, characterized in that: The rendering and displaying of the control according to the target image data and the data corresponding to the control includes: Setting the rendering thread priority corresponding to the control to a preset rendering priority; Based on the preset rendering priority, the control is displayed after rendering according to the target image data and the data corresponding to the control.

11. The control display method according to claim 2, characterized in that: The step of sending an image data request to a display composition system in response to receiving a display instruction of the control comprises: In response to a display instruction of the control, determining whether the control needs to be displayed with a preset display effect; When it is determined that the control needs to be displayed with a preset display effect, sending an image data request to the display synthesis system; The method further comprises: When it is determined that the control does not need to be displayed with the preset display effect, rendering and displaying the control according to the data corresponding to the control; A stop notification is sent to the display composition system, so that the display composition system stops sending the target image data to the corresponding control.

12. A control display device, characterized in that: include: An image data processing module is configured to create a new synthesis thread in response to receiving a display instruction of the control, wherein the synthesis thread is used to obtain target image data corresponding to the control; The control processing module is configured to render and display the control according to the target image data and the data corresponding to the control.

13. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the method described in any one of claims 1 to 11 are implemented.

15. A chip, characterized in that: The invention comprises a processor and an interface; the processor is used to read instructions to execute the method according to any one of claims 1 to 11.

Images