A window fusion display optimization method based on companion relationship measurement

Abstract

The present invention discloses a window fusion display optimization method based on companion relationship measurement, which obtains the main window of an application started in a virtual machine as a first window, sends a window drawing message to a desktop, checks whether the first window is a companion window of other windows according to whether it receives the window drawing message, then sends a window size change message and a window drawing message to the first window in sequence, and measures the companion relationship between other windows and the first window according to the reception of the window size change message and the window drawing message by other windows and the shadow rule of the window, and finally makes the virtual machine not transmit the screened companion window to the host machine, thereby ensuring the correctness of the application window fusion display process in the virtual machine.

Description

Technical Field

[0001] The invention belongs to the technical field of computer software development, and in particular relates to a window fusion display optimization method based on associated relationship measurement. Background Art

[0002] Existing virtual machines, such as VMware virtual machines, usually provide users with two application usage modes. One is the traditional mode displayed with a complete operating system, that is, the operating system running in the virtual machine displays the complete operating system desktop, and all applications in the operating system are displayed on the desktop; the other is a seamless mode or fusion mode that displays applications separately, that is, the operating system running in the virtual machine does not display the operating system desktop, and each application in the operating system is displayed on the host side in the form of an independent window, bringing users the same experience as the application on the host. Among them, the implementation process of the fusion mode requires enumerating the main windows of all applications running on the virtual machine, and then transmitting the data of all main windows set to visible attributes to the host side for rendering.

[0003] Normally, the operating system provides an API for determining whether a window is visible, such as the IsWindowVisible function under Windows. For transparent or semi-transparent windows, the operating system will determine that such windows are visible, but invisible or partially visible to the user. However, the actual use requirement of the fusion mode is to only display windows that are visible to the user. This results in that when the data of such windows is transferred to the host machine for rendering according to the existing fusion mode, what is actually presented to the user is the desktop background of the operating system in the virtual machine displayed in such windows, which is different from the actual display effect. Such windows are more common in applications with special display effects, such as semi-transparent windows used to display shadow effects, fully transparent windows, etc. Using the existing fusion mode for display will significantly reduce the user experience. Summary of the invention

[0004] In view of this, the present invention provides a window fusion display optimization method based on companion relationship measurement, which screens out companion windows by measuring the companion relationship between windows to determine the way of application window fusion display, thereby realizing the window fusion display of application windows in the virtual machine to the host machine.

[0005] The present invention provides a window fusion display optimization method based on associated relationship measurement, which specifically includes the following steps:

[0006] Step 1: forming a first window set from the main windows of all applications started in the virtual machine, wherein each window includes a window handle and an initial window size; establishing a Hook function for the default processing function of the window message, monitoring the window drawing message and the window size change message, so that the windows in the to-be-filtered window list only execute the reception and setting of the message, wherein the to-be-filtered window list is used to save the first window for which it is not determined whether there is a companion relationship;

[0007] Step 2, select a window in the first window set as the current first window, record the display area of ​​the current first window as the first window area, add the current first window to the list of windows to be filtered, send a message to the desktop to refresh the first window area, if the current first window receives a standard window drawing message, add the current first window to the ignored window list, delete it from the list of windows to be filtered, and then execute step 7; otherwise, execute step 3;

[0008] Step 3: Send a first window size change message to the current first window, and record the window width and height in the first window size change message as the source window size; when there is a window that receives the second window size change message among other windows other than the current first window, record the window width and height in the second window size change message as the window size to be confirmed, record these windows as windows to be confirmed, and add the windows to be confirmed to the list of windows to be screened; calculate whether the relationship between the window size to be confirmed and the source window size of the windows to be confirmed meets the shadow rule, and for windows that meet the shadow rule, treat them as suspected companion windows of the current first window, and add the corresponding relationship between the suspected companion window and the current first window to the list of suspected companion windows;

[0009] Step 4: Send a first window drawing message to the current first window according to the suspected companion window list. If there is a suspected companion window that receives the second window drawing message, execute step 6; otherwise, treat the display area of ​​the suspected companion window as the suspected companion area and execute step 5;

[0010] Step 5, sending a message to refresh the suspected companion area to the desktop. If the suspected companion window receives the second window drawing message, the suspected companion window is added to the ignored window list, and the suspected companion window and the current first window are added to the window companion relationship list, and the suspected companion window is deleted from the list of windows to be filtered and the first window set, and the current first window is deleted from the list of suspected companion windows and the list of windows to be filtered, and then executing step 7; otherwise, executing step 6;

[0011] Step 6, deleting the suspected associated window from the suspected associated window list. If there is no information corresponding to the current first window in the suspected associated window list, then deleting the current first window from the list of windows to be screened and executing step 7; otherwise, executing step 4;

[0012] Step 7, delete the current first window from the first window set, if the first window set is not empty, execute step 2, otherwise execute step 8;

[0013] Step 8. For windows in the ignored window list, the virtual machine does not transmit their data to the host machine; for windows that are not in the ignored window list, the host machine creates a second window for them and renders the received data in the second window. At the same time, the relationship between the windows is determined based on the obtained window companion relationship list to complete the window fusion display of the application in the virtual machine.

[0014] Furthermore, for a window in the suspected associated window list, a calculation is made as to whether a relationship between an initial window size of the window and an initial window size of the current first window complies with a shadow rule, and windows that do not comply with the shadow rule are removed.

[0015] Furthermore, step 3 is performed multiple times, and a list of suspected associated windows is determined according to the results of the multiple executions.

[0016] Furthermore, the default processing function of the window message in step 1 is the DefWindowProc function.

[0017] Furthermore, the shadow rule in step 3 is that the companion window is 16 pixels wider and 8 pixels higher than the current first window. Beneficial Effects

[0018] The present invention obtains the main window of an application that has been started in a virtual machine as the first window, sends a window drawing message to the desktop, checks whether the first window is a companion window of other windows according to whether it receives the window drawing message, then sends a window size change message and a window drawing message to the first window in sequence, and measures the companion relationship between other windows and the first window according to the reception of the window size change message and the window drawing message by other windows and the shadow rule of the window, and finally makes the virtual machine not transmit the screened companion window to the host machine, thereby ensuring the correctness of the application window fusion display process in the virtual machine. DETAILED DESCRIPTION

[0019] The present invention is described in detail with reference to the following embodiments.

[0020] The present invention provides a window fusion display optimization method based on companion relationship measurement, the core idea of ​​which is: obtaining the main window of an application that has been started in a virtual machine as the first window, sending a window drawing message to the desktop, checking whether the first window has a rendering companion relationship with other windows based on whether it receives the window drawing message, and then sending a window size change message and a window drawing message to the first window in sequence, and measuring the size companion relationship and rendering companion relationship between other windows and the first window based on the reception of the window size change message and the window drawing message by other windows and the shadow rules of the windows, and finally preventing the virtual machine from transmitting the screened companion window to the host machine, thereby ensuring the correctness of the application window fusion display process in the virtual machine.

[0021] The present invention provides a window fusion display optimization method based on associated relationship measurement, which specifically includes the following steps:

[0022] Step 1: record the operating system running in the virtual machine as the first operating system, and record the desktop of the first operating system as the first desktop; obtain the main windows of all currently started applications in the virtual machine to form a first window set, and each window in the first window set includes a window handle and an initial window size. Establish a Hook function for the default processing function of the window message in the first operating system to monitor the standard window drawing message and the standard window size change message, and make the windows in the window list to be filtered only perform the reception and setting of the two messages without actually performing the drawing or window size change operation, wherein the window list to be filtered is used to save the first window that has not been determined to be an accompanying shadow window.

[0023] Specifically, in Windows system, a Hook function can be established for the default processing function of window message by hooking the DefWindowProc function in the dynamic library user32.dll through the dynamic library Hook method. Among them, the standard window drawing message and the standard change window size message are standard messages sent by the operating system in the virtual machine.

[0024] Step 2. Select a window in the first window set as the current first window, record the display area of ​​the current first window as the first window area, add the window handle of the current first window to the list of windows to be filtered, obtain the desktop handle of the first desktop, and send a message to the first desktop to refresh the first window area. If the current first window receives a standard window drawing message, it means that the current first window is a companion window of other windows. Then, add the current first window to the ignored window list, delete the current first window from the list of windows to be filtered, and then execute step 7; otherwise, it means that the current first window is not a companion window of other windows, and execute step 3.

[0025] In an actual system, the companion window used to display the shadow effect will also be refreshed when the system desktop is refreshed, while the general application window will not be refreshed along with the desktop. Therefore, based on this attribute, it can be determined whether the window has a shadow effect.

[0026] Step 3, sending a first window size change message to the current first window, recording the window width and height in the first window size change message as the source window size; monitoring other windows in the first window set except the current first window, and when there is a window that receives the second window size change message, recording the window width and height in the second window size change message as the window size to be confirmed, recording these windows as windows to be confirmed, and adding the windows to be confirmed to the list of windows to be screened; calculating whether the relationship between the window size to be confirmed and the source window size of the windows to be confirmed meets the shadow rule, and for windows that meet the shadow rule, regard them as suspected companion windows of the current first window, and add the corresponding relationship between the suspected companion window and the current first window to the suspected companion window list, and the windows in the suspected companion window list are all represented by window handles.

[0027] The first window size change message is a standard window size change message constructed by the present invention for screening windows with associated relationships. The second window size change message is a standard window size change message.

[0028] In order to further improve the accuracy of the suspected associated window list, the present invention calculates whether the relationship between the initial window size of the window in the suspected associated window list and the initial window size of the current first window meets the shadow rule, and removes the windows that do not meet the shadow rule from the suspected associated window list.

[0029] In addition, multiple comparison results may be obtained by changing the current first window to different sizes multiple times, so as to improve the accuracy of the suspected associated window list.

[0030] In Windows systems, in order to achieve a three-dimensional display effect for a window, a companion window is usually constructed for the window to display the window's shadow effect, which is also called a shadow window. The companion window opens, shifts, changes size, closes, and hides with the window. Normally, the shadow window is 16 pixels wider than the window and 8 pixels higher than the window, which forms the shadow rule for windows in Windows systems.

[0031] Step 4. Send a first window drawing message to the current first window according to the list of suspected companion windows. If the suspected companion window of the current first window receives the second window drawing message, it indicates that the window is not a companion window of the current first window, and then execute step 6. Otherwise, it indicates that the window may be a companion window and further judgment is required. Obtain the display area of ​​the suspected companion window and record it as the suspected companion area, and then execute step 5.

[0032] The first window drawing message is a standard window drawing message constructed by the present invention for filtering windows with associated relationships. The second window drawing message is a standard window drawing message.

[0033] Step 5. Get the desktop handle of the first desktop, send a message to the first desktop to refresh the suspected companion area. If the suspected companion window receives the second window drawing message, determine that the suspected companion window is a companion window of the current first window, add the window handle of the suspected companion window to the ignored window list, and add the suspected companion window and the current first window to the window companion relationship list, and delete the suspected companion window from the list of windows to be filtered and the first window set, and delete the current first window from the list of suspected companion windows and the list of windows to be filtered, then execute step 7; otherwise, it means that the suspected companion window is not a companion window of the current first window, and execute step 6.

[0034] Step 6: Delete the suspected associated window from the suspected associated window list. If there is no information corresponding to the current first window in the suspected associated window list, delete the current first window from the list of windows to be screened and then execute step 7; otherwise, execute step 4.

[0035] Step 7: Delete the current first window from the first window set. If the first window set is not empty, execute step 2; otherwise, execute step 8.

[0036] Step 8. For windows in the ignored window list, the virtual machine does not transmit their data to the host machine; for windows that are not in the ignored window list, the host machine creates a second window for them and renders the received data in the second window. At the same time, the relationship between windows is determined based on the obtained window companion relationship list, and the window fusion display of the application in the virtual machine is completed to end this process. Example

[0037] In this embodiment, a window fusion display optimization method based on associated relationship measurement provided by the present invention is used to realize the window fusion display of Windows applications in a virtual machine on a host host, which specifically includes the following steps:

[0038] S1. Hook the DefWindowProc function of the first operating system to create a Hook function MyHookProc function. The calling method of MyHookProc function is:

[0039] LRESULT MyHookProc LRESULT DefWindowProc (HWND hWnd, UINT Msg, WPARAMwParam, LPARAM lParam) {}

[0040] Among them, hWnd is the handle pointing to the window procedure that receives the message, Msg is the specified message type; wParam is used to specify the remaining message-specific information, and its data is related to the Msg parameter value; lParam is used to specify the remaining message-specific information, and its data is related to the Msg parameter value.

[0041] S2. First, determine by the relationship between the window sizes, and judge from the shape and size whether a window may be a companion window of another window, that is, a shadow.

[0042] S2.1. Traverse the first window set including all main windows of all programs, and for the window mainWindow in the first window set, send a window size modification message WM_SIZE to mainWindow. The parameters of the WM_SIZE message generally include:

[0043] wParam: An identifier indicating the reason for the window size change. Common values ​​include SIZE_RESTORED, which means the window is restored; SIZE_MINIMIZED, which means the window is minimized; and SIZE_MAXIMIZED, which means the window is maximized.

[0044] lParam: contains the new size of the window, which is LPARAM and is usually interpreted as a RECT structure, which contains the new size of the window. That is:

[0045] int width=LOWORD(lParam); / / Get window width

[0046] int height=HIWORD(lParam); / / Get the window height

[0047] In Windows, after processing the WM_SIZE message, the system default processing function, namely the DefWindowProc function, will be called. DefWindowProc is a function provided by the Windows API, which actually performs the final operation of changing the window size.

[0048] S2.2. mainWindows processes the message according to the custom processing flow, including correctly sending the message to its own companion window associatedWindow, and notifying the MyHookProc function to add the window handle of mainWindow to the list of windows to be filtered, so that when the message is WM_SIZE and the window handle hWnd is the window handle of mainWindow, the DefWindowProc function does not perform the actual window change operation.

[0049] S2.3. Monitor whether all other windows receive the window size change message, and the change method complies with the shadow rule. For example, the size value is larger than the size of the mainWindow rectangular area and the size difference is a set pixel. If so, it means that there is a size companion relationship between the two, which is recorded as associatedWindow, and associatedWindow and mainWindow are saved as a tuple, and this tuple is added to the mapping array mapShadowWindowToMainWindow.

[0050] S3. Determine through the companion relationship of the rendering process whether the window recorded in mapShadowWindowToMainWindow is actually the content of the desktop or the desktop content with a transparent effect added, rather than the main window with a size companion relationship determined in the table.

[0051] S3.1. Traverse each tuple of mapShadowWindowToMainWindow, and for each tuple, take out the mainWindow and the corresponding associatedWindow in the tuple, and notify the MyHookProc function to start monitoring the window drawing or rendering message of the associatedWindow, that is, the WM_PAINT message of the Windows system.

[0052] S3.2. Send a window drawing message WM_PAINT message to mainWindow, and monitor the drawing message processing of associatedWindow in the MyHookProc function. If associatedWindow also triggers the drawing processing, that is, the window drawing message is received, it means that associatedWindow is not a shadow window, and continue to execute S3.1; if not, it means that it may be a shadow window, and execute S3.3.

[0053] If it is a shadow window, redrawing the main window will not cause the shadow window to be redrawn.

[0054] S3.3. Get the window handle of the desktop and record it as desktopWindow. Under Windows, you can call GetDesktopWindow to get the handle; get the window rectangular area of ​​associatedWindow and record it as rectAssociatedWindow. Send a message to desktopWindow to refresh the rectAssociatedWindow area. At the same time, monitor the drawing message processing of associatedWindow in the MyHookProc function. If associatedWindow exists and the drawing processing is triggered, that is, the window drawing message is received, it means that associatedWindow is a shadow window, and associatedWindow will not be sent to the Host for rendering; if not, it means it is not a shadow window, and then continue to loop and execute S3.1.

[0055] S4. On the host side, a corresponding window is created for each application in the virtual machine, and the window content of the corresponding application transmitted from the virtual machine is rendered in the window.

[0056] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A window fusion display optimization method based on associated relationship measurement, characterized in that: The specific steps include: Step 1: forming a first window set from the main windows of all applications started in the virtual machine, wherein each window includes a window handle and an initial window size; A Hook function is established for the default processing function of the window message to monitor the window drawing message and the window size change message, so that the windows in the to-be-filtered window list only receive and set the message, wherein the to-be-filtered window list is used to save the first window for which it is not determined whether there is a companion relationship; Step 2, select a window in the first window set as the current first window, record the display area of ​​the current first window as the first window area, add the current first window to the list of windows to be filtered, send a message to the desktop to refresh the first window area, if the current first window receives a standard window drawing message, add the current first window to the ignored window list, delete it from the list of windows to be filtered, and then execute step 7; otherwise, execute step 3; Step 3: Send a first window size change message to the current first window, and record the window width and height in the first window size change message as the source window size; when there is a window that receives the second window size change message among other windows other than the current first window, record the window width and height in the second window size change message as the window size to be confirmed, record these windows as windows to be confirmed, and add the windows to be confirmed to the list of windows to be screened; calculate whether the relationship between the window size to be confirmed and the source window size of the windows to be confirmed meets the shadow rule, and for windows that meet the shadow rule, treat them as suspected companion windows of the current first window, and add the corresponding relationship between the suspected companion window and the current first window to the list of suspected companion windows; Step 4: Send a first window drawing message to the current first window according to the suspected companion window list. If there is a suspected companion window that receives the second window drawing message, execute step 6; otherwise, treat the display area of ​​the suspected companion window as the suspected companion area and execute step 5; Step 5, sending a message to refresh the suspected companion area to the desktop. If the suspected companion window receives the second window drawing message, the suspected companion window is added to the ignored window list, and the suspected companion window and the current first window are added to the window companion relationship list, and the suspected companion window is deleted from the list of windows to be filtered and the first window set, and the current first window is deleted from the list of suspected companion windows and the list of windows to be filtered, and then executing step 7; otherwise, executing step 6; Step 6, deleting the suspected associated window from the suspected associated window list. If there is no information corresponding to the current first window in the suspected associated window list, then deleting the current first window from the list of windows to be screened and executing step 7; otherwise, executing step 4; Step 7, delete the current first window from the first window set, if the first window set is not empty, execute step 2, otherwise execute step 8; Step 8. For windows in the ignored window list, the virtual machine does not transmit their data to the host machine; for windows that are not in the ignored window list, the host machine creates a second window for them and renders the received data in the second window. At the same time, the relationship between the windows is determined based on the obtained window companion relationship list to complete the window fusion display of the application in the virtual machine.

2. The window fusion display optimization method according to claim 1, characterized in that: For the windows in the suspected associated window list, a calculation is made as to whether a relationship between an initial window size of the window and an initial window size of the current first window complies with a shadow rule, and windows that do not comply with the shadow rule are removed.

3. The window fusion display optimization method according to claim 1, characterized in that: Execute step 3 multiple times, and determine a suspected associated window list according to the results of the multiple executions.

4. The window fusion display optimization method according to claim 1, characterized in that: The default processing function of the window message in step 1 is the DefWindowProc function.

5. The window fusion display optimization method according to claim 1, characterized in that: The shadow rule in step 3 is that the companion window is 16 pixels wider and 8 pixels higher than the current first window.