A window rendering method and related equipment

Abstract

The embodiment of the present invention discloses a window rendering method and related equipment, comprising: first determining the business function implemented by each of the multiple functional areas contained in the target window and the function type of the business function; then decomposing the target window according to the function type to obtain a first sub-window and a second sub-window; then rendering the first sub-window in an off-screen rendering manner and rendering the second sub-window in an on-screen rendering manner. By adopting the embodiment of the present invention, the rendering performance of the special-shaped window can be improved, thereby improving the smoothness of the window screen display.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a window rendering method and related equipment. Background Art

[0002] In the window display process, commonly used rendering methods include screen rendering and off-screen rendering. Among them, screen rendering refers to the way in which the browser engine renders the Hypertext Markup Language (HTML) / Cascading Style Sheets (CSS) / JavaScript (JS) content and directly outputs it to the screen. Off-screen rendering refers to the way in which the browser engine renders the HTML / CSS / JS content into a bitmap and outputs it to the memory. At present, off-screen rendering is commonly used for the processing of live windows. When off-screen rendering is performed, the application opens up a separate area to draw the bitmap, and the graphics processing unit (GPU) accelerated synthesis is that the central processing unit (CPU) hands the drawing work to the GPU to complete and present it to the screen by the GPU. Therefore, although the browser kernel provides invalid areas and pixel buffers for the application, it does not support accelerated synthesis, resulting in low window rendering performance. Especially when playing videos in the window, the frame rate is low. If no special processing is added, the user will obviously feel the stuttering phenomenon of the high-definition picture. Summary of the invention

[0003] The present invention provides a window rendering method and related equipment, which can improve the rendering performance of special-shaped windows, thereby improving the fluency of window picture display.

[0004] In a first aspect, an embodiment of the present invention provides a window rendering method, comprising:

[0005] Determine the business function implemented by each of the multiple functional areas included in the target window, and the function type of the business function;

[0006] Decomposing the target window according to the function type to obtain a first sub-window and a second sub-window;

[0007] The first sub-window is rendered in an off-screen rendering manner, and the second sub-window is rendered in an on-screen rendering manner.

[0008] Wherein, the first sub-window and the second sub-window include at least one functional area among the plurality of functional areas; the functional type includes a pixel mixing type, and the business function of the pixel mixing type is realized based on the pixel mixing technology;

[0009] Decomposing the target window according to the function type to obtain a first sub-window and a second sub-window comprises:

[0010] If the business functions implemented by the functional area include the pixel mixing type business function, it is determined that the functional area belongs to the first sub-window; otherwise, it is determined that the functional area belongs to the second sub-window.

[0011] After rendering the first sub-window in an off-screen rendering manner and rendering the second sub-window in an on-screen rendering manner, the method further includes:

[0012] The first sub-window and the second sub-window are synchronously processed, and the synchronous processing includes synchronous displacement.

[0013] The step of synchronously processing the first sub-window and the second sub-window comprises:

[0014] Acquire the position information of the first sub-window and the second sub-window at a preset time interval;

[0015] Determine, according to the position information, whether there is another window between the first sub-window and the second sub-window;

[0016] When other windows are spaced between the first sub-window and the second sub-window, the position of the first sub-window and / or the second sub-window is adjusted.

[0017] The step of synchronously processing the first sub-window and the second sub-window comprises:

[0018] controlling the second sub-window to monitor the displacement synchronization message of the first sub-window, and controlling the second sub-window to shift according to the displacement synchronization message; and

[0019] The first sub-window is controlled to monitor the focus message of the second sub-window, and the first sub-window is controlled to move according to the focus message.

[0020] Wherein, the target window corresponds to a main thread and a sub-thread;

[0021] The rendering of the first sub-window in an off-screen rendering manner and the rendering of the second sub-window in an on-screen rendering manner include:

[0022] The first sub-window is rendered in the main thread according to the off-screen rendering method, and the second sub-window is rendered in the sub-thread according to the screen rendering method.

[0023] After rendering the first sub-window in an off-screen rendering manner and rendering the second sub-window in an on-screen rendering manner, the method further includes:

[0024] The first sub-window and the second sub-window are synchronously processed in the sub-thread.

[0025] The first child window is the logical parent window of the second child window; and the second child window is a top-level window.

[0026] In a second aspect, an embodiment of the present invention provides a window rendering device, including:

[0027] An information determination module, used to determine the business function implemented by each of the multiple functional areas included in the target window, and the function type of the business function;

[0028] A window decomposition module, used for decomposing the target window according to the function type to obtain a first sub-window and a second sub-window;

[0029] The window rendering module is used to render the first sub-window in an off-screen rendering manner and to render the second sub-window in an on-screen rendering manner.

[0030] Wherein, the first sub-window and the second sub-window include at least one functional area among the plurality of functional areas; the functional type includes a pixel mixing type, and the business function of the pixel mixing type is realized based on the pixel mixing technology;

[0031] The window decomposition module is also used for:

[0032] If the business functions implemented by the functional area include the pixel mixing type business function, it is determined that the functional area belongs to the first sub-window; otherwise, it is determined that the functional area belongs to the second sub-window.

[0033] The device further includes a synchronization processing module, which is used to:

[0034] The first sub-window and the second sub-window are synchronously processed, and the synchronous processing includes synchronous displacement.

[0035] Wherein, the synchronization processing module is also used for:

[0036] Acquire the position information of the first sub-window and the second sub-window at a preset time interval;

[0037] Determine, according to the position information, whether there is another window between the first sub-window and the second sub-window;

[0038] When other windows are spaced between the first sub-window and the second sub-window, the position of the first sub-window and / or the second sub-window is adjusted.

[0039] Wherein, the synchronization processing module is also used for:

[0040] controlling the second sub-window to monitor the displacement synchronization message of the first sub-window, and controlling the second sub-window to shift according to the displacement synchronization message; and

[0041] The first sub-window is controlled to monitor the focus message of the second sub-window, and the first sub-window is controlled to move according to the focus message.

[0042] Wherein, the target window corresponds to a main thread and a sub-thread;

[0043] The window rendering module is also used for:

[0044] The first sub-window is rendered in the main thread according to the off-screen rendering method, and the second sub-window is rendered in the sub-thread according to the screen rendering method.

[0045] Wherein, the synchronization processing module is also used for:

[0046] The synchronization process is performed on the first sub-window and the second sub-window in the sub-thread.

[0047] The first child window is the logical parent window of the second child window; and the second child window is a top-level window.

[0048] In a third aspect, an embodiment of the present invention provides a window rendering device, comprising: a processor, a memory and a communication bus, wherein the communication bus is used to realize connection and communication between the processor and the memory, and the processor executes a program stored in the memory to implement the steps in a window rendering method provided in the first aspect above.

[0049] In a possible design, the entity identification device provided by the present invention may include a module for executing the behavior corresponding to the above method. The module may be software and / or hardware.

[0050] Another aspect of an embodiment of the present invention provides a computer-readable storage medium, in which a plurality of instructions are stored. The instructions are suitable for being loaded by a processor and executing the methods described in the above aspects.

[0051] Another aspect of the embodiments of the present invention provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the methods described in the above aspects.

[0052] In implementing the embodiment of the present invention, the business function implemented by each of the multiple functional areas contained in the target window and the function type of the business function are first determined; then the target window is decomposed according to the function type to obtain a first sub-window and a second sub-window; then the first sub-window is rendered in an off-screen rendering manner, and the second sub-window is rendered in an on-screen rendering manner. This can not only improve the fluency of the window screen and the performance of window rendering, but also reduce the occupied content and reduce the burden on the CPU. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the background technology, the drawings required for use in the embodiments of the present invention or the background technology will be described below.

[0054] Figure 1 It is a flowchart of a window rendering method provided by an embodiment of the present invention;

[0055] Figure 2 is a schematic diagram of a stacked window stack provided by an embodiment of the present invention;

[0056] Figure 3 is a schematic diagram of a target window provided by an embodiment of the present invention;

[0057] Figure 4 is a flowchart of another window rendering method provided by an embodiment of the present invention;

[0058] Figure 5 It is a schematic diagram of an actual application scenario provided by an embodiment of the present invention

[0059] Figure 6 It is a structural schematic diagram of a window rendering device provided by an embodiment of the present invention;

[0060] Figure 7 It is a structural schematic diagram of a window rendering device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] See also Figure 1 , Figure 1 : is a flowchart of a window rendering method provided by an embodiment of the present invention, the method includes but is not limited to the following steps:

[0063] S101. Determine the business functions implemented by each functional area in the target window and the functional types of the business functions.

[0064] In specific implementation, the target window can be a special-shaped window that can be decomposed into multiple sub-windows, such as a "hui"-shaped window, a "tian"-shaped window, etc. At the same time, the target window is also a layer window, and it can include multiple functional areas, and each functional area is used to implement at least one business function. For example, as Figure 2 shown, the online classroom window includes a live broadcast area for teachers to give lectures online and a chat area for teachers and students to communicate online. Among them, the business functions implemented by the live broadcast area include functions such as the pen function used by teachers during lectures, loading web pages (such as web pages searched by teachers online and shown to students), or other program windows (such as PowerPoint windows for displaying courseware). The business function implemented by the chat area is real-time interaction between multiple users (teachers and students). In the present invention, all business functions implemented by the target window are divided into pixel-mixed business functions and non-pixel-mixed business functions. As the name implies, pixel-mixed business functions are implemented based on pixel mixing technology. For example, the pen function. Among them, common pixel mixing technologies include alpha (α) mixing technology. Alpha mixing technology means covering one window control (frame) on another frame, and the upper frame will not completely block the lower frame, that is, some areas in the upper frame are transparent. Among them, the pixels of the two frames are mixed according to a preset α value. Non-pixel-mixed business functions refer to other business functions that do not need to use pixel mixing technology to be implemented.

[0065] S102. Decompose the target window according to the functional type to obtain a first sub-window and a second sub-window.

[0066] In specific implementation, the first sub-window and the second sub-window respectively include at least one functional area. Among them, for each functional area, if the business functions implemented by the functional area include pixel-mixed business functions, it is determined that the functional area belongs to the first sub-window; otherwise, it is determined that the functional area belongs to the second sub-window. As Figure 2As shown, the online classroom window based on the Windows system includes a live broadcast area and a chat area, wherein the brush function implemented in the live broadcast area is a pixel blending type of business function, so the live broadcast area is divided into the first sub-window. The multi-user real-time interactive function implemented in the chat area does not require the use of pixel blending technology and is a non-pixel blending type of business function, so the chat area is divided into the second sub-window. Finally, the first sub-window composed of the live broadcast area - the live broadcast window, and the second sub-window composed of the chat area - the chat window are obtained. In addition, as Figure 2 As shown, the window display areas of the live broadcast window and the chat window can be hollowed out using the interface provided by the Windows system, and the hollowed-out area currently displays the computer's background desktop.

[0067] It should be noted that starting from Windows 2000, the Windows operating system began to adopt a layered window mechanism. Under this mechanism, multiple windows can be stacked to form a stack of stacked windows. The stack of windows is positioned according to an imaginary axis that is perpendicular to the screen and extends outside the screen. This axis is usually called the Z axis. Each window in the stack of stacked windows is stacked in sequence with the Z axis as the reference axis. Each window has a stacking order (z-order) that indicates the layer of the stack of stacked windows that the window is located in, and can also indicate the order in which the windows are displayed. Figure 3 As shown, the stacked window stack includes 3 windows, among which the top window has a z-order of 3, the second window is stacked below the top window along the Z axis, and its z-order is 2, and the third window is also the bottom window, which is stacked below the second window along the Z axis, and its z-order is 1. In addition, there are subordinate relationships such as parent / child relationship and sibling relationship between windows. Among them, the child window is the next level window derived from the parent window, and the sibling window is two windows derived from the same level directory of the same parent window.

[0068] Based on the above description, the present invention further describes the first child window and the second child window: 1) the second child window is a window without a parent window and is usually placed at the top level as a top-level window; 2) the first child window is the logical parent window of the second child window, and the second child window saves the handle of the first child window.

[0069] S103: Render the first sub-window in an off-screen rendering manner, and render the second sub-window in an on-screen rendering manner.

[0070] In the specific implementation, in order to achieve the fluency of real-time screen display, the window that needs to realize the business function of pixel mixing should be rendered using the off-screen rendering method. The business function of non-pixel mixing only requires the normal display of the picture, and the requirement for the real-time display of the picture is lower than that of the pixel mixing type, so it is only necessary to use the screen rendering method to render the second sub-window. The fundamental reason is that in the screen rendering method, the rendering operation of the GPU is performed in the current screen buffer, and the off-screen rendering method also opens up another buffer outside the current screen buffer to complete the rendering of the window, so when the screen refresh rate is required to be high, the off-screen rendering can better meet the requirements of the picture fluency. However, it can also be seen from this that compared with screen rendering, off-screen rendering occupies more memory and increases the processing burden of the CPU. Therefore, the present invention first decomposes the target window into multiple sub-windows, and then selects a suitable rendering method according to the actual situation of each sub-window, so as to achieve the purpose of reducing the occupied memory and alleviating the CPU burden.

[0071] It should be noted that, in extreme cases, each of the multiple functional areas of the target window does not contain a pixel mixing business function. In this case, the target window can be decomposed into multiple logical sub-windows according to the logical relationship between the various functional areas, and each logical sub-window can be rendered directly using screen rendering.

[0072] In an embodiment of the present invention, the business function implemented by each of the multiple functional areas included in the target window and the function type of the business function are first determined; then the target window is decomposed according to the function type to obtain a first sub-window and a second sub-window; then the first sub-window is rendered in an off-screen rendering manner, and the second sub-window is rendered in an on-screen rendering manner. The memory occupied by window rendering can be reduced, and the burden on the CPU can be reduced, so as to improve the smoothness of window screen display and the performance of window rendering.

[0073] See also Figure 4 , Figure 4 : is a flow chart of another window rendering method provided by an embodiment of the present invention, the method includes but is not limited to the following steps:

[0074] S401, determining the business function implemented by each of the multiple functional areas included in the target window and the function type of the business function. This step is the same as S101 in the previous embodiment, and this step will not be repeated.

[0075] S402, decomposing the target window according to the function type to obtain a first sub-window and a second sub-window. This step is the same as S102 in the previous embodiment, and will not be described in detail.

[0076] S403, rendering the first sub-window in an off-screen rendering manner, and rendering the second sub-window in an on-screen rendering manner. This step is the same as S103 in the previous embodiment, and will not be described in detail.

[0077] S404: Synchronize the first sub-window and the second sub-window.

[0078] In a specific implementation, the first sub-window is the logical parent window of the second sub-window. When either the first sub-window or the second sub-window moves, the other sub-window needs to be synchronously displaced. On the one hand, the second sub-window can monitor the displacement synchronization message of the first sub-window, wherein the displacement synchronization message may refer to, but is not limited to, the displacement information broadcast or sent to the second sub-window after the first sub-window moves to the surroundings. The displacement information may include the direction and amplitude of the displacement, and the second sub-window may perform synchronous displacement according to the displacement synchronization message. The synchronous displacement may also include synchronously hiding the second sub-window / first sub-window when the first sub-window / second sub-window is hidden.

[0079] On the other hand, the first sub-window can be controlled to monitor the focus message of the second sub-window, and the first sub-window can be controlled to shift according to the monitored focus message. Among them, in the Windows system, the focus determines which input box in that window / control / window receives the keyboard input information. When the user uses the keyboard to input using a certain input (such as Sogou input method), a candidate box for the input content will appear accordingly. The candidate box comes from the application corresponding to the input method. Therefore, the window receiving the keyboard input needs to be synchronously shifted accordingly to make the candidate box displayed in a suitable place. Corresponding to the present invention, if the second sub-window undergoes a synchronous shift corresponding to the input content candidate box, the first sub-window also needs to be shifted synchronously with the second sub-window.

[0080] After the target window is decomposed into the first sub-window and the second sub-window, although the first sub-window is the logical parent window of the second sub-window, there is no actual parent-child relationship between the first sub-window and the second sub-window. Therefore, the operating system will not synchronize the two, so the first sub-window / second sub-window moves, and the second sub-window / first sub-window cannot be synchronized with it. The problem often occurs, resulting in two separate sub-windows from the user's perspective. Synchronous displacement processing of the first sub-window and the second sub-window can effectively solve this problem.

[0081] Optionally, the stacking order of the first sub-window and the second sub-window in the stacked window stack where they are located can be synchronously adjusted. Specifically, the position information of the first sub-window and the second sub-window can be obtained at a preset time interval, wherein the preset time interval can be 10 microseconds, 10 milliseconds, and so on. The position information can be, but is not limited to, z-order. For a stacked window stack, the windows therein can be traversed through the GetTopWindow function and the GetNextWindow function to obtain the z-order of the desired window. Then, based on the position information, determine whether there are other windows between the first sub-window and the second sub-window, that is, whether the first sub-window and the second sub-window are adjacent to each other on the Z-axis and whether there are other windows (such as browser windows) sandwiched between them. For example, if Figure 3 As shown, the z-orders of the first sub-window and the second sub-window are 1 and 3 respectively, the second sub-window is the top-level window, and the first sub-window is the third-level (bottom-level) window. At this time, there is another window between the first sub-window and the second sub-window, and the window is sandwiched between the first sub-window and the second sub-window, thereby forming a "sandwich window". At this time, the window sandwiched between the first sub-window and the second sub-window makes it impossible for the first sub-window and the second sub-window to be effectively synthesized into a target window, that is, the target window cannot achieve the same display effect as the conventional rendering method after being rendered according to the method provided by the present invention. Therefore, if there are other windows between the first sub-window and the second sub-window, the positions of the first sub-window and the second sub-window need to be adjusted, wherein the positions of the two windows can be adjusted at the same time, or the position of one of the sub-windows can be kept unchanged while adjusting the position of the other sub-window. For example, when the z-orders of the first sub-window and the second sub-window are 3 and 1 respectively, the z-order of the first sub-window can be adjusted to 2, and it can be adjusted from the third-level window to the second-level window, so that the first sub-window and the second sub-window are close to each other.

[0082] Optionally, the target window corresponds to a main thread and a sub-thread. The operation of rendering the first sub-window in an off-screen rendering manner can be performed in the main thread, and the operation of rendering the second sub-window in an on-screen rendering manner can be performed in the sub-thread. At the same time, the synchronization processing operation of the first sub-window and the second sub-window is also performed in the sub-thread. Considering that if the creation (rendering) of the two sub-windows is placed in the main thread, it is easy to cause thread congestion when handling the synchronization problem of the two sub-windows, the above method of decomposing the tasks into sub-threads is adopted.

[0083] In an embodiment of the present invention, first, determine the service function implemented by each functional area in the target window and the function type of the service function; then decompose the target window according to the function type to obtain a first sub-window and a second sub-window; then render the first sub-window in an off-screen rendering manner and render the second sub-window in a screen rendering manner; finally, synchronize the first sub-window and the second sub-window. Taking Figure 2 the online classroom window shown as an example, before decomposition, the online classroom window includes a live broadcast area and a chat area. Then, since the live broadcast area includes a pixel hybrid service function - the brush function, it is thus divided into the first sub-window to obtain the live broadcast window, and the chat area does not include any pixel hybrid service function, so it is divided into the second sub-window to obtain the chat window. Then, as Figure 5 shown, for the live broadcast window, in an off-screen rendering manner, first render the live broadcast window into a bitmap and output it to the memory to obtain a memory bitmap, and then use the alpha blending technology to perform pixel blending on the memory bitmap and the computer screen to obtain the off-screen rendering result of the live broadcast window. For the chat window, the bitmap can be rendered in a screen rendering manner and directly output to the screen to obtain the screen rendering result of the chat window. Finally, synchronize the live broadcast window and the chat window so that what is seen from the user's perspective is the only window - the undecomposed online classroom window.

[0084] By adopting the embodiment of the present invention, the appearance of the "sandwich window" and the problem of out-of-sync displacement of the two sub-windows can be prevented, thereby improving the smoothness of the window screen display and the performance of window rendering on the premise of ensuring normal display of the target window.

[0085] The method of the embodiment of the present invention is elaborated in detail above, and the related devices of the embodiment of the present invention are provided below.

[0086] Please refer to Figure 6 , Figure 6 which is a schematic structural diagram of a window rendering device provided by an embodiment of the present invention. The device may include:

[0087] An information determination module 601, configured to determine the service function implemented by each functional area in the target window and the function type of the service function.

[0088] In specific implementation, the target window may be a special-shaped window that can be decomposed into multiple sub-windows, such as a "hui" - shaped window, a "tian" - shaped window, etc. At the same time, the target window is also a layer window, and it may include multiple functional areas, and each functional area is used to implement at least one service function. For example, as Figure 2As shown, the online classroom window includes a live broadcast area for teachers to give online lectures, and a chat area for teachers and students to communicate online. Among them, the business functions implemented in the live broadcast area include the brush function required by the teacher in the lecture, the function of loading web pages (such as web pages that teachers search online and show to students) or other program windows (such as PowerPoint windows for displaying courseware), etc., and the business function implemented in the chat area is real-time interaction between multiple users (teachers and students). In the present invention, all business functions implemented by the target window are divided into pixel mixing business functions and non-pixel mixing business functions. As the name implies, pixel mixing business functions are implemented based on pixel mixing technology, for example, brush functions. Among them, commonly used pixel mixing technologies include alpha mixing technology, which refers to covering a window control (frame) on another frame, and the upper frame will not completely cover the lower frame, that is, part of the area in the upper layer of the frame is transparent, wherein the pixels of the two frames are mixed according to the pre-set alpha value. Non-pixel mixing business functions refer to other business functions that do not require the use of pixel mixing technology.

[0089] The window decomposition module 602 is used to decompose the target window according to the function type to obtain a first sub-window and a second sub-window.

[0090] In a specific implementation, the first sub-window and the second sub-window each include at least one functional area. For each functional area, if the business function implemented by the functional area includes a pixel mixing business function, the functional area is determined to belong to the first sub-window, otherwise, the functional area is determined to belong to the second sub-window. Figure 2 As shown, the online classroom window based on the Windows system includes a live broadcast area and a chat area, wherein the brush function implemented in the live broadcast area is a pixel blending type of business function, so the live broadcast area is divided into the first sub-window. The multi-user real-time interactive function implemented in the chat area does not require the use of pixel blending technology and is a non-pixel blending type of business function, so the chat area is divided into the second sub-window. Finally, the first sub-window composed of the live broadcast area - the live broadcast window, and the second sub-window composed of the chat area - the chat window are obtained. In addition, as Figure 2 As shown, the window display areas of the live broadcast window and the chat window can be hollowed out using the interface provided by the Windows system, and the hollowed-out area currently displays the computer's background desktop.

[0091] It should be noted that starting from Windows 2000, the Windows operating system began to adopt a layered window mechanism. Under this mechanism, multiple windows can be stacked to form a stack of stacked windows. The stack of windows is positioned according to an imaginary axis that is perpendicular to the screen and extends outside the screen. This axis is usually called the Z axis. Each window in the stack of stacked windows is stacked in sequence with the Z axis as the reference axis. Each window has a stacking order (z-order) that indicates the layer of the stack of stacked windows that the window is located in, and can also indicate the order in which the windows are displayed. Figure 3 As shown, the stacked window stack includes 3 windows, among which the top window has a z-order of 3, the second window is stacked below the top window along the Z axis, and its z-order is 2, and the third window is also the bottom window, which is stacked below the second window along the Z axis, and its z-order is 1. In addition, there are subordinate relationships such as parent / child relationship and sibling relationship between windows. Among them, the child window is the next level window derived from the parent window, and the sibling window is two windows derived from the same level directory of the same parent window.

[0092] Based on the above description, the present invention further describes the first child window and the second child window: 1) the second child window is a window without a parent window and is usually placed at the top level as a top-level window; 2) the first child window is the logical parent window of the second child window, and the second child window saves the handle of the first child window.

[0093] The window rendering module 603 is used to render the first sub-window in an off-screen rendering manner and to render the second sub-window in an on-screen rendering manner.

[0094] In the specific implementation, in order to achieve the fluency of real-time screen display, the window that needs to realize the business function of pixel mixing should be rendered using the off-screen rendering method. The business function of non-pixel mixing only requires the normal display of the picture, and the requirement for the real-time display of the picture is lower than that of the pixel mixing type, so it is only necessary to use the screen rendering method to render the second sub-window. The fundamental reason is that in the screen rendering method, the rendering operation of the GPU is performed in the current screen buffer, and the off-screen rendering method also opens up another buffer outside the current screen buffer to complete the rendering of the window, so when the screen refresh rate is required to be high, the off-screen rendering can better meet the requirements of the picture fluency. However, it can also be seen from this that compared with screen rendering, off-screen rendering occupies more memory and increases the processing burden of the CPU. Therefore, the present invention first decomposes the target window into multiple sub-windows, and then selects a suitable rendering method according to the actual situation of each sub-window, so as to achieve the purpose of reducing the occupied memory and alleviating the CPU burden.

[0095] Optionally, the device in the embodiment of the present invention further includes a synchronization processing module, configured to perform synchronization processing on the first sub-window and the second sub-window.

[0096] In a specific implementation, the first sub-window is the logical parent window of the second sub-window. When either the first sub-window or the second sub-window moves, the other sub-window needs to be synchronously displaced. On the one hand, the second sub-window can monitor the displacement synchronization message of the first sub-window, wherein the displacement synchronization message may refer to, but is not limited to, the displacement information broadcast or sent to the second sub-window after the first sub-window moves to the surroundings. The displacement information may include the direction and amplitude of the displacement, and the second sub-window may perform synchronous displacement according to the displacement synchronization message. The synchronous displacement may also include synchronously hiding the second sub-window / first sub-window when the first sub-window / second sub-window is hidden.

[0097] On the other hand, the first sub-window can be controlled to monitor the focus message of the second sub-window, and the first sub-window can be controlled to shift according to the monitored focus message. Among them, in the Windows system, the focus determines which input box in that window / control / window receives the keyboard input information. When the user uses the keyboard to input using a certain input method (such as Sogou input method), a candidate box for the input content will appear accordingly. The candidate box comes from the application corresponding to the input method. Therefore, the window receiving the keyboard input needs to be synchronously shifted accordingly to make the candidate box displayed in a suitable place. Corresponding to the present invention, if the second sub-window undergoes a synchronous shift corresponding to the input content candidate box, the first sub-window also needs to be shifted synchronously with the second sub-window.

[0098] After the target window is decomposed into the first sub-window and the second sub-window, although the first sub-window is the logical parent window of the second sub-window, there is no actual parent-child relationship between the first sub-window and the second sub-window. Therefore, the operating system will not synchronize the two, so the first sub-window / second sub-window moves, and the second sub-window / first sub-window cannot be synchronized with it. The problem often occurs, resulting in two separate sub-windows from the user's perspective. Synchronous displacement processing of the first sub-window and the second sub-window can effectively solve this problem.

[0099] Optionally, the stacking order of the first sub-window and the second sub-window in the stacked window stack where they are located can be synchronously adjusted. Specifically, the position information of the first sub-window and the second sub-window can be obtained at a preset time interval, wherein the position information can be but is not limited to z-order. For a stacked window stack, the windows therein can be traversed through the GetTopWindow function and the GetNextWindow function to obtain the z-order of the desired window. Then, based on the position information, determine whether there are other windows between the first sub-window and the second sub-window, that is, whether the first sub-window and the second sub-window are adjacent to each other on the Z axis and whether there are other windows (such as browser windows) sandwiched between them. For example, Figure 3 As shown, the z-orders of the first sub-window and the second sub-window are 1 and 3 respectively, the second sub-window is the top-level window, and the first sub-window is the third-level (bottom-level) window. At this time, there is another window between the first sub-window and the second sub-window, and the window is sandwiched between the first sub-window and the second sub-window, thereby forming a "sandwich window". At this time, the other windows between the first sub-window and the second sub-window make it impossible for the first sub-window and the second sub-window to be effectively synthesized into a target window, that is, the target window cannot achieve the same display effect as the conventional rendering method after being rendered according to the method provided by the present invention. Therefore, if there are other windows between the first sub-window and the second sub-window, the positions of the first sub-window and the second sub-window need to be adjusted, wherein the positions of the two windows can be adjusted at the same time, or the position of one of the sub-windows can be kept unchanged and the position of the other sub-window can be adjusted. For example, when the z-orders of the first sub-window and the second sub-window are 3 and 1 respectively, the z-order of the first sub-window can be adjusted to 2, and it can be adjusted from the third-level window to the second-level window, so that the first sub-window and the second sub-window are close to each other.

[0100] Optionally, the target window corresponds to a main thread and a sub-thread. The window rendering module 603 can also render the first sub-window in an off-screen rendering manner in the main thread, and render the second sub-window in an on-screen rendering manner in the sub-thread. At the same time, the synchronization processing operation of the first sub-window and the second sub-window is also performed in the sub-thread. Considering that if the creation (rendering) of the two sub-windows is placed in the main thread, it is easy to cause thread congestion when processing the synchronization problem of the two sub-windows, the above-mentioned method of decomposing tasks into sub-threads is adopted.

[0101] In the embodiment of the present invention, the business function implemented by each of the multiple functional areas included in the target window and the function type of the business function are first determined; then the target window is decomposed according to the function type to obtain a first sub-window and a second sub-window; then the first sub-window is rendered in an off-screen rendering manner, and the second sub-window is rendered in an on-screen rendering manner; finally, the first sub-window and the second sub-window are synchronously processed. The memory occupied by window rendering, the processing burden of the CPU, and the smoothness of window screen display and the performance of window rendering can be improved.

[0102] See also Figure 7 , Figure 7 The structure diagram of a window rendering device provided by an embodiment of the present invention is shown in FIG.

[0103] Among them, the processor 701 can be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and so on. The communication bus 704 can be a peripheral component interconnect standard PCI bus or an extended industrial standard architecture EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7Only one thick line is used to represent it, but it does not mean that there is only one bus or one type of bus. The communication bus 704 is used to realize the connection and communication between these components. Among them, the communication interface 702 of the device in the embodiment of the present invention is used to communicate signaling or data with other node devices. The memory 703 may include a volatile memory, such as a non-volatile dynamic random access memory (NVRAM), a phase change random access memory (PhaseChange RAM, PRAM), a magnetoresistive random access memory (MRAM), etc., and may also include a non-volatile memory, such as at least one disk storage device, an electronically erasable programmable read-only memory (EEPROM), a flash memory device, such as a NOR flash memory or a NAND flash memory, a semiconductor device, such as a solid state drive (SSD), etc. The memory 703 may also be optionally at least one storage device located away from the aforementioned processor 701. A set of program codes is stored in the memory 703, and the processor 701 executes the program in the memory 703:

[0104] Determine the business function implemented by each of the multiple functional areas included in the target window, and the function type of the business function;

[0105] Decomposing the target window according to the function type to obtain a first sub-window and a second sub-window;

[0106] The first sub-window is rendered in an off-screen rendering manner, and the second sub-window is rendered in an on-screen rendering manner.

[0107] Optionally, the first sub-window and the second sub-window include at least one functional area among the multiple functional areas; the functional type includes a pixel mixing type, and the business function of the pixel mixing type is implemented based on a pixel mixing technology;

[0108] The processor 701 is further configured to execute the following operation steps:

[0109] If the business functions implemented by the functional area include the pixel mixing type business function, it is determined that the functional area belongs to the first sub-window; otherwise, it is determined that the functional area belongs to the second sub-window.

[0110] Optionally, the processor 701 is further configured to perform the following operation steps:

[0111] The first sub-window and the second sub-window are synchronously processed, and the synchronous processing includes synchronous displacement.

[0112] Optionally, the processor 701 is further configured to perform the following operation steps:

[0113] Acquire the position information of the first sub-window and the second sub-window at a preset time interval;

[0114] Determine, according to the position information, whether there is another window between the first sub-window and the second sub-window;

[0115] When other windows are spaced between the first sub-window and the second sub-window, the position of the first sub-window and / or the second sub-window is adjusted.

[0116] Optionally, the processor 701 is further configured to perform the following operation steps:

[0117] controlling the second sub-window to monitor the displacement synchronization message of the first sub-window, and controlling the second sub-window to shift according to the displacement synchronization message; and

[0118] The first sub-window is controlled to monitor the focus message of the second sub-window, and the first sub-window is controlled to move according to the focus message.

[0119] Optionally, the target window corresponds to a main thread and a child thread;

[0120] The processor 701 is further configured to execute the following operation steps:

[0121] The first sub-window is rendered in the main thread according to the off-screen rendering method, and the second sub-window is rendered in the sub-thread according to the screen rendering method.

[0122] Furthermore, the processor may also cooperate with the memory and the communication interface to execute the operations of the window rendering device in the above-mentioned embodiment of the invention.

[0123] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0124] The specific implementations described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims

1. A window rendering method, applied to live broadcast scenes, It is characterized in that The method comprises: Determine the business function implemented by each of the multiple functional areas included in the target window and the function type of the business function; the function type includes a pixel mixing type, and the business function of the pixel mixing type is implemented based on a pixel mixing technology; Decomposing the target window according to the function type to obtain a first sub-window and a second sub-window includes: if the business function implemented by the functional area includes the pixel mixing type business function, determining that the functional area belongs to the first sub-window; otherwise, determining that the functional area belongs to the second sub-window; the first sub-window and the second sub-window include at least one functional area among the multiple functional areas; Rendering the first sub-window in an off-screen rendering manner, and rendering the second sub-window in an on-screen rendering manner; wherein the off-screen rendering is rendering performed in a buffer area opened outside the current screen buffer area, and the on-screen rendering is rendering performed in the current screen buffer area; The first sub-window and the second sub-window are synchronously processed, and the synchronous processing includes synchronous displacement.

2. The method according to claim 1, It is characterized in that The synchronous processing of the first sub-window and the second sub-window comprises: Acquire the position information of the first sub-window and the second sub-window at a preset time interval; Determine, according to the position information, whether there is another window between the first sub-window and the second sub-window; When other windows are spaced between the first sub-window and the second sub-window, the position of the first sub-window and / or the second sub-window is adjusted.

3. The method according to claim 1, It is characterized in that The synchronous processing of the first sub-window and the second sub-window comprises: controlling the second sub-window to monitor the displacement synchronization message of the first sub-window, and controlling the second sub-window to shift according to the displacement synchronization message; and The first sub-window is controlled to monitor the focus message of the second sub-window, and the first sub-window is controlled to move according to the focus message.

4. The method according to claim 1, It is characterized in that The target window corresponds to a main thread and a child thread; The rendering of the first sub-window in an off-screen rendering manner and the rendering of the second sub-window in an on-screen rendering manner include: The first sub-window is rendered in the main thread according to the off-screen rendering method, and the second sub-window is rendered in the sub-thread according to the screen rendering method.

5. The method according to claim 4, It is characterized in that After rendering the first sub-window in an off-screen rendering manner and rendering the second sub-window in an on-screen rendering manner, the method further includes: The first sub-window and the second sub-window are synchronously processed in the sub-thread.

6. The method according to any one of claims 1 to 5, It is characterized in that The first sub-window is the logical parent window of the second sub-window; the second sub-window is a top-level window.

7. A window rendering device, applied to live broadcast scenes, It is characterized in that The device comprises: An information determination module, used to determine the business function implemented by each of the multiple functional areas included in the target window, and the function type of the business function; the function type includes a pixel mixing type, and the business function of the pixel mixing type is implemented based on the pixel mixing technology; a window decomposition module, used for decomposing the target window according to the function type to obtain a first sub-window and a second sub-window, comprising: if the business function implemented by the functional area includes the pixel mixing type business function, determining that the functional area belongs to the first sub-window; otherwise, determining that the functional area belongs to the second sub-window; the first sub-window and the second sub-window include at least one functional area among the multiple functional areas; A window rendering module, configured to render the first sub-window in an off-screen rendering manner, and to render the second sub-window in an on-screen rendering manner; wherein the off-screen rendering is rendering performed in a buffer area opened up outside the current screen buffer area, and the on-screen rendering is rendering performed in the current screen buffer area; The synchronization processing module is used to perform synchronization processing on the first sub-window and the second sub-window, wherein the synchronization processing includes synchronization displacement.

8. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method according to any one of claims 1 to 6.

Images