Multi-application scenario window layout cross-screen cooperative nonlinear restoration system

Abstract

The application provides a multi-application-scene window layout cross-screen cooperative nonlinear restoration system, and belongs to the technical field of computers.The system comprises a system data acquisition module, an application scene generation module, an intelligent layout strategy module and a scene layout restoration module.The system data acquisition module is responsible for acquiring application-related data.The application scene generation module is in communication connection with the system data acquisition module and the intelligent layout strategy module, and generates a highly correlated application combination scene.The intelligent layout strategy module is in communication connection with the system data acquisition module, the application scene generation module and the scene layout restoration module, and generates multiple sets of window layout snapshots.The scene layout restoration module is in communication connection with the system data acquisition module and the intelligent layout strategy module, and is responsible for scene layout restoration.The application establishes a software combination semantic network, and realizes multi-screen application automatic layout based on application data flow and user interaction.

Description

Technical Field

[0001] This invention relates to window layout restoration for multiple application scenarios, and more particularly to a cross-screen collaborative nonlinear restoration system for window layouts in multiple application scenarios, belonging to the field of computer technology. Background Technology

[0002] With advancements in hardware design and manufacturing processes for computers, graphics cards, and display devices, a single graphics card can support multiple display output devices, leading to a growing demand from users for multi-screen display scenarios with multiple display devices. A wealth of desktop applications provides diverse functionalities for different users, and collaborative work across multiple applications has become the norm.

[0003] Current multi-screen display settings typically only record physical monitor parameters, such as resolution and the arrangement of different monitors, while ignoring the interrelationships between software ecosystems. For example, a user might simultaneously open Photoshop and Lightroom, a common combination in photography and design, which can be identified as a design workflow. Furthermore, dynamic scene transitions cannot be handled, such as switching from a "programming IDE + database tool" to a "video conferencing + document presentation," requiring manual rearrangement of windows across multiple displays. The configuration of development / entertainment scenarios cannot be isolated between different accounts on the same hardware device, leading to confusion across user scenarios. Summary of the Invention

[0004] To address the aforementioned issues, this invention provides a cross-screen collaborative nonlinear restoration system for window layout in multiple application scenarios, enabling automatic layout of multi-screen applications based on application data flow and user interaction.

[0005] To achieve the above objectives, the technical solution of the present invention is: a cross-screen collaborative nonlinear restoration system for window layout in multiple application scenarios, the system comprising a system data acquisition module, an application scenario generation module, an intelligent layout strategy module, and a scenario layout restoration module; wherein,

[0006] The system data acquisition module is communicatively connected to the application scenario generation module, the intelligent layout strategy module, and the scenario layout restoration module, and is responsible for collecting application-related data.

[0007] The application scenario generation module is communicatively connected to the system data acquisition module and the intelligent layout strategy module. By analyzing the data collected by the system data acquisition module, it performs multi-dimensional correlation strength analysis on the application, and finally generates a highly correlated application combination scenario, which is then passed to the intelligent layout strategy module.

[0008] The intelligent layout strategy module is communicatively connected to the system data acquisition module, the application scenario generation module, and the scenario layout restoration module. It generates multiple sets of window layout snapshots for each scenario by combining the window layout data collected by the system data acquisition module with the application scenario combination. At the same time, it dynamically updates the snapshot priority through a priority algorithm so that the scenario layout restoration module can restore the scenario window layout.

[0009] The scene layout restoration module is connected to the system data acquisition module and the intelligent layout strategy module. It is responsible for restoring the scene window layout. When a change in the application scene is detected, the scene is identified, and then the best layout is dynamically recommended and restored to the system data acquisition module according to the snapshot priority transmitted by the intelligent layout strategy module.

[0010] Furthermore, the system data acquisition module collects application-related data, including application usage data and window layout data;

[0011] The application usage data includes the application process ID and process name AppID, application startup timestamp, application shutdown timestamp, application active duration, application foreground and background switching time, application CPU usage, application memory usage, and application network access.

[0012] The window layout data includes the coordinate position (x, y) of the application window, the width (w) and height (h) of the application window, the monitor ID where the application window is located, the stacking order, the window state, the split screen information, and the special area occupancy.

[0013] The system data acquisition module collects application-related data, including application file system access information, system clipboard usage information, and monitor connection information.

[0014] Furthermore, the application scenario generation module's working steps include the following:

[0015] Step 1. Calculate the association strength between each pair of applications and construct the association strength matrix between applications;

[0016] Step 2. Convert the association strength matrix into a distance matrix, perform hierarchical clustering, and organize the clustering results;

[0017] Step 3. Analyze the clustering results, obtain highly correlated application groups, and generate information on highly correlated application combination scenarios;

[0018] Step 4. Store the associated application group data.

[0019] Furthermore, in step 1, the correlation strength between each pair of applications is represented by application A and application B, and is calculated using the following formula: CorrelationStrength(A,B) = w1 T(A,B) + w2 R(A,B) + w3 I(A,B), where: T(A,B) is the time dimension correlation between application A and application B, R(A,B) is the resource dimension correlation between application A and application B, I(A,B) is the interaction dimension correlation between application A and application B, w1, w2, w3 are the corresponding weight coefficients, and w1+w2+w3=1.

[0020] Furthermore, the formula for calculating the time-dimensional correlation between application A and application B is as follows:

[0021] T(A,B) = 0.4 Lifecycle overlap + 0.4 Overlapping activity times + 0.2 Initiate sequence correlation, where,

[0022] Lifecycle overlap = Overlap time / Total time. Overlap time refers to the overlap time between the lifecycles of application A and application B, and the total time is the sum of the lifecycles of application A and application B.

[0023] Overlapping Active Time = Shared Active Time / Total Active Time. Shared active time refers to the overlapping time when application A and application B are both in the foreground or in the background. Total active time refers to the sum of the active time of application A and application B.

[0024] Startup sequence relevance = Adjacent start count / Total start count. Adjacent start count refers to the number of times application A and application B are launched adjacently within a specific time period. Total start count refers to the sum of the number of times application A and application B are launched within a specific time period.

[0025] Furthermore, the formula for calculating the resource dimension correlation between application A and application B is as follows:

[0026] R(A,B) = 0.4 Resource usage relevance +0.3 I / O operation relevance +0.3 Network access relevance, among which,

[0027] Resource usage relevance = pearson(CPU_A, CPU_B) 0.5 + pearson(MEM_A, MEM_B) 0.5, pearson(CPU_A, CPU_B) calculates the Pearson correlation coefficient between two sets of data: CPU utilization of application A and CPU utilization of application B; MEM_A and MEM_B are application memory utilization, and pearson(MEM_A, MEM_B) calculates the Pearson correlation coefficient between the two sets of application memory utilization.

[0028] IO operation correlation = Number of common IO operations / Total number of IO operations. The number of common IO operations refers to the number of times that application A and application B perform IO operations simultaneously within the same time period; the total number of IO operations refers to the sum of the number of IO operations performed by application A and application B respectively.

[0029] Network access relevance = Number of shared network accesses / Total number of network accesses. The number of shared network accesses refers to the number of times application A and application B access the same domain name or the same IP address and port. The total number of network accesses refers to the sum of the number of network accesses by application A and application B.

[0030] Furthermore, the formula for calculating the correlation between application A and application B in terms of interaction dimension is as follows:

[0031] I(A,B) = 0.4 Window switching frequency +0.3 Data exchange frequency + 0.3 User operation relevance, among which...

[0032] Window switching frequency = number of times switching between application A and application B / total number of switching times; data exchange frequency = number of times data is exchanged between application A and application B / total number of data exchanges; user operation relevance = number of relevant operations / total number of operations. Relevant operations include window state adjustment, which includes maximizing, minimizing, resizing the window, and window snapping to the edge for split screen.

[0033] Furthermore, the working steps of the intelligent layout strategy module include the following:

[0034] Step 1, Window State Vector: Based on the window layout data collected by the system data acquisition module, the window state is vectorized to generate a state vector for each window;

[0035] Step 2, Generate Scene Window Layout Snapshot: Based on the application combination scene generated by the application scene generation module, collect all relevant window state vectors in each application combination scene into a layout snapshot, and attach parameters to each layout snapshot;

[0036] Step 3, layout similarity deduplication: Calculate the similarity between different layouts, and merge records when the similarity exceeds 90%;

[0037] Step 4: Save a snapshot of the scene window layout.

[0038] Furthermore, the parameters mentioned in step 2 include: layout ID, scene ID, creation time, update time, cumulative usage count, time period usage count, duration, satisfaction rating, window configuration set, display configuration, and layout priority;

[0039] The layout priority is calculated using a comprehensive score based on the following formula:

[0040] Layout priority score = (a × recent usage frequency + b × cumulative usage duration) × time decay coefficient + c × satisfaction score + d × time matching degree, where,

[0041] Time matching degree is the degree of overlap between the current usage time of the layout snapshot and the historical usage time period of the layout snapshot. Time period usage rate = Time period usage count / Cumulative usage count. Time matching degree = Historical usage rate of the current time period × (1 + Recent usage adjustment factor), where a, b, c, and d are weighting coefficients; Time decay coefficient = e^(-λ × Number of unused days), where λ is the decay rate.

[0042] Furthermore, the scene layout restoration module detects changes in the software scene, identifies the current application combination scene based on the application combination, queries the application combination scene ID and its window layout, matches the best layout based on the layout priority, and obtains the window configuration set to complete the layout restoration.

[0043] The beneficial effects of the cross-screen collaborative nonlinear restoration system for window layout in multiple application scenarios of the present invention are as follows:

[0044] This invention designs and implements a multi-application scenario window layout cross-screen coordination nonlinear restoration system based on a domestic desktop operating system. Through clear module division, it achieves the following key objectives:

[0045] Application scenario generation: Collect system application usage data and related window layout data, combine them with domestic operating systems, and generate highly correlated application combination scenario information through multi-dimensional correlation strength analysis;

[0046] Application window layout strategy: Combine application combination scenario information, capture and store multiple layout states, and provide layout recommendations based on layout priority and different users by using memory data;

[0047] Supports layout restoration during scene transitions: After detecting a scene transition, the optimal layout is obtained and the window layout is smoothly restored;

[0048] Continuous optimization of scenario strategies: Continuously collect system data, optimize application combination scenarios and layout strategies, and provide more intelligent restoration strategies.

[0049] This invention supports creating multiple layout versions for the same group of applications (such as "deep coding mode" vs "code review mode"), and supports seamless integration with the window manager under the Kylin system. Attached Figure Description

[0050] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0051] Figure 1 This is a schematic diagram of the structural composition of the cross-screen collaborative nonlinear restoration system for window layout in multiple application scenarios according to the present invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0053] This invention, based on operating system-level window management and event listening technologies, involves application behavior analysis, scenario-based layout strategies, and user habit modeling. Addressing the limitations of traditional multi-screen display settings, which are restricted to physical parameter settings and memory, it establishes a software combination semantic network (e.g., identifying WPS + WeChat + DingTalk as an office collaboration scenario) to achieve automatic multi-screen application layout based on application data flow and user interaction. Simultaneously, it enables non-linear memory of scene states, allowing the same application combination to save multiple layouts in different scenarios. The following embodiments provide a detailed description of this invention's cross-screen collaborative non-linear restoration system for multi-application scenario window layout.

[0054] Example 1

[0055] Combination Figure 1 As shown, the cross-screen collaborative nonlinear restoration system for window layout in multiple application scenarios of the present invention includes a system data acquisition module, an application scenario generation module, an intelligent layout strategy module, and a scenario layout restoration module, wherein:

[0056] The system data acquisition module is connected to the application scenario generation module, intelligent layout strategy module and scenario layout restoration module, and is responsible for collecting data, including application usage data and window layout data, etc.

[0057] The application scenario generation module communicates with the system data acquisition module and the intelligent layout strategy module. By analyzing the data collected by the system data acquisition module, it performs multi-dimensional correlation strength analysis on the application and finally generates highly correlated application combination scenarios, which are then passed to the intelligent layout strategy module.

[0058] The intelligent layout strategy module communicates with the system data acquisition module, application scenario generation module, and scenario layout restoration module. It generates multiple sets of window layout snapshots for each scenario by combining window layout data collected by the system data acquisition module with application scenario combinations. At the same time, it dynamically updates the snapshot priority through a priority algorithm so that the scenario layout restoration module can restore the scenario window layout.

[0059] The scene layout restoration module communicates with the system data acquisition module and the intelligent layout strategy module. It is responsible for restoring the scene window layout. When a change in the application scene is detected, the scene is identified, and then the best layout is dynamically recommended to the system data acquisition module and restored according to the snapshot priority passed by the intelligent layout strategy module.

[0060] Through the collaboration of these four modules, a window layout restoration solution based on application scenarios is achieved. This automatically restores window layouts for different users and usage scenarios, reducing the tedious manual adjustments required by users. Furthermore, it continuously learns and iterates during user interaction, providing an even more intelligent restoration solution.

[0061] Example 2

[0062] Building upon Example 1, the system data acquisition module will be described in further detail. This module is responsible for collecting user application usage data and window layout data, providing data input for the application scenario generation module and the intelligent layout strategy module.

[0063] The collected application usage data includes:

[0064] Application process ID and process name AppID;

[0065] Application startup timestamp;

[0066] Application close timestamp;

[0067] Application active duration (sum of focus time and background running time);

[0068] Application foreground / background switching time;

[0069] Application CPU utilization;

[0070] Application memory usage;

[0071] Application network access status;

[0072] The collected window layout data includes the following parameters:

[0073] The coordinates (x, y) of the application window;

[0074] The width (w) and height (h) of the application window;

[0075] MonitorID of the display where the application window is located;

[0076] Stacking order: The z-order value of the record window;

[0077] Window state: Records special states such as maximized, minimized, and fullscreen;

[0078] Split-screen information: Records the system's split-screen or the user's manual split-screen status;

[0079] Special area occupancy: Records the positions of system UI elements such as the taskbar and sidebar.

[0080] Other system information collection:

[0081] Application file system access information;

[0082] System clipboard usage;

[0083] Monitor connection status.

[0084] Example 3

[0085] Building upon Example 2, the application scenario generation module is further described in detail. The application scenario generation module acquires data from the system data acquisition module, performs data analysis, and identifies highly relevant application combinations to form highly relevant application scenario combinations. The main working steps of this module include the following:

[0086] Step 1. Calculate the association strength between each pair of applications and construct the association strength matrix between applications;

[0087] Step 2. Convert the association strength matrix into a distance matrix (for subsequent clustering), perform hierarchical clustering, and organize the clustering results;

[0088] Step 3. Analyze the clustering results, obtain highly correlated application groups, and generate highly correlated application combination scenario information (application combination scenario ID, application list, average correlation strength).

[0089] Step 4. Store the associated application group data.

[0090] The above steps, based on various data from the system data acquisition module, complete the application association strength analysis, which is the most important task of the application scenario generation module. The following example illustrates the formula for calculating the association strength between applications A and B:

[0091] CorrelationStrength(A,B) = w1 T(A,B) + w2 R(A,B) + w3 I(A,B)

[0092] in:

[0093] T(A,B): Time-dimension correlation

[0094] R(A,B): Resource-dimensional correlation

[0095] I(A,B): Interaction Dimension Relevance

[0096] w1, w2, w3: Corresponding weight coefficients, and w1 + w2 + w3 = 1, with w1, w2, and w3 taking values ​​of 0.3, 0.2, and 0.5 respectively;

[0097] The formula for calculating the time-dimensional correlation T(A,B) is as follows:

[0098] T(A,B) = 0.4 Lifecycle overlap + 0.4 Overlapping activity times + 0.2 Startup sequence relevance

[0099] Wherein, lifecycle overlap = overlap time / total time, lifecycle is the application shutdown timestamp minus the application startup timestamp; overlap time refers to the overlap time of the lifecycles of application A and application B, that is, the duration of the "time intersection", which is also the duration of the overlapping part of the lifecycles of the two applications; total time is the sum of the lifecycles of application A and application B.

[0100] Active time overlap = Common active time / Total active time. Common active time and total active time are calculated by the application foreground and background switching time obtained in Example 1. Common active time refers to the overlapping time when application A and application B are in the foreground or the overlapping time when they are in the background. Total active time refers to the sum of the application active time of application A and application B.

[0101] Startup sequence relevance = Adjacent start count / Total start count. Adjacent start count refers to the number of times application A and application B are launched adjacently within a specific time period. Total start count refers to the sum of the number of times application A and application B are launched within that specific time period.

[0102] The formula for calculating the resource dimension relevance R(A,B) is as follows:

[0103] R(A,B) = 0.4 Resource usage relevance +0.3 I / O operation relevance +0.3 Network access relevance

[0104] Wherein, resource usage relevance = pearson(CPU_A, CPU_B) 0.5 + pearson(MEM_A,MEM_B) 0.5, pearson(CPU_A, CPU_B) calculates the Pearson correlation coefficient between two sets of data: CPU utilization of application A (CPU_A) and CPU utilization of application B (CPU_B). This coefficient quantifies the linear correlation between the two applications in terms of CPU resource consumption. MEM_A and MEM_B represent application memory utilization, obtained from application memory usage data collected in Example 1. pearson(MEM_A, MEM_B) calculates the Pearson correlation coefficient between the two sets of application memory utilization, quantifying the strength and direction of the linear association between the two applications in terms of memory resource consumption.

[0105] IO operation correlation = number of common IO operations / total number of IO operations. The number of IO operations is obtained based on the application file system access data collected in Example 1. The number of common IO operations refers to the number of times that application A and application B perform IO operations simultaneously within the same time period (such as reading and writing the same file or directory at the same time, or having IO behavior within the same time window). The total number of IO operations refers to the sum of the number of IO operations performed by application A and application B respectively.

[0106] Network access relevance = number of common network accesses / total number of network accesses. The number of network accesses is obtained from the application network access data collected in Example 1. The number of common network accesses refers to the number of times application A and application B access the same domain name or the same IP + port. The total number of network accesses refers to the sum of the number of network accesses by application A and application B.

[0107] The formula for calculating the correlation degree I(A,B) of the above interaction dimensions is as follows:

[0108] I(A,B) = 0.4 Window switching frequency +0.3 Data exchange frequency + 0.3 User operation relevance

[0109] Among them, window switching frequency = number of switching between application A and application B / total number of switching, data exchange frequency = number of data exchanges between application A and application B / total number of data exchanges, user operation relevance = number of relevant operations / total number of operations. Relevant operations mainly refer to window state adjustments, including: maximizing, minimizing, window size adjustment, window snapping to the edge for split screen, etc.

[0110] The user operation relevance mentioned above reflects the proportion of operations between application A and application B in the overall total. The number of related operations is the number of related operations that occur between application A and application B (i.e., operations involving maximizing, minimizing, window snapping, splitting the screen, etc.). The total number of operations refers to the total number of related operations that occur between all applications, used for normalization. Specifically:

[0111] The number of related operations that occur between application A and application B, such as: application A and application B windows snapping together in split screen, switching to application B after application A is minimized, and resizing application A and application B windows, etc.

[0112] The total number of related operations that occur between all applications within the statistical period, such as the total number of related operations between application A and application B, application A and application C, application B and application C, application D and application E, etc.

[0113] That is, the number of related operations is the number of related operations between application A and application B, and the total number of operations is the total number of related operations between all applications.

[0114] Example 4

[0115] Building upon Example 3, the intelligent layout strategy module is further described in detail. The intelligent layout strategy module acquires window layout data from the system data acquisition module, combines it with application scenario combinations generated by the application scenario generation module, analyzes and processes the data to form a layout strategy, and implements a dynamic layout priority adjustment algorithm to provide intelligent layout strategies for application scenario combinations.

[0116] The working steps of the intelligent layout strategy module include the following:

[0117] Step 1, Window State Vector: Based on the window layout data collected by the system data acquisition module, the window state is vectorized, generating a state vector [AppID, MonitorID, x, y, w, h, z-order, state] for each window; where AppID is the application process name, MonitorID refers to the monitor ID where the application window is located, x and y refer to the coordinates of the application window, w is the width of the application window, h is the height of the application window, z-order is the recorded Z-axis order value of the window, and state refers to the window state (including recording special states such as maximized, minimized, and full screen);

[0118] Step 2, Generate Scene Window Layout Snapshots: Based on the application combination scenes generated by the application scene generation module, collect all relevant window state vectors in each application combination scene into a layout snapshot (all relevant windows refer to the window arrangement state associated with the application combination scene, such as opening WPS and a browser, adjusting to a preferred window distribution state, with WPS on the right and the browser on the left, etc.), and attach parameters to each layout snapshot, including:

[0119] a. Layout ID: A unique identifier for the layout scheme;

[0120] b. Scene ID: The associated application combination scene ID (one application combination scene ID can be associated with multiple layout snapshots);

[0121] c. Creation time: The time when it was first recorded;

[0122] d. Update time: The time of the most recent update;

[0123] e. Cumulative usage count: The cumulative number of times the current layout snapshot has been applied and restored by the user, which is the number of times the user chooses to use this window layout corresponding to the associated application combination scenario after it is recommended to the user;

[0124] f. Time Period Usage Count: The number of times the current window layout snapshot is applied and restored by the user during the time periods of morning (9:00 to 12:00), noon (13:00 to 15:00), afternoon (15:00 to 18:00), and evening (18:00 to 21:00);

[0125] g. Duration: Cumulative usage time. Multiple scene window layout snapshots can be stored for a single associated application combination scene. The duration refers to the cumulative time of using this window layout snapshot. When the scene layout restoration module detects a scene change, it selects a different snapshot. This snapshot is strongly correlated with the associated application combination scene. When this snapshot is used, the applications in the corresponding application combination scene are all running.

[0126] h. Satisfaction rating: Satisfaction level (0-100) derived from user modification behavior.

[0127] i. Window configuration set: The set of state vectors for all windows;

[0128] j. Display configuration: Current number of displays, resolution, and relative position;

[0129] k. Layout priority: The priority score of the current layout snapshot;

[0130] Step 3, layout similarity deduplication: Calculate the similarity between different layouts, and merge records when the similarity exceeds 90%;

[0131] Step 4: Save a snapshot of the scene window layout.

[0132] The intelligent layout strategy module employs a dynamic priority adjustment algorithm for layout snapshots, dynamically adjusting their priority scores. The comprehensive scoring formula is as follows:

[0133] Layout priority score = (a × recent usage frequency + b × cumulative usage time) × time decay coefficient + c × satisfaction score + d × time matching degree;

[0134] Among them, time matching degree is the correlation between the frequency of use and time characteristics of collected historical layout snapshots, that is, the degree of overlap between the current usage time of the layout snapshot and the historical usage time period of the layout snapshot. For example, users prefer to use this layout snapshot of this application group in the morning. The usage rate of a time period can be calculated based on the usage count of the time period and the cumulative usage count, and then the time matching degree can be calculated.

[0135] Time-of-use utilization rate = Time-of-use count / Cumulative usage count

[0136] Time matching degree = historical usage rate of the current time period × (1 + recent usage adjustment factor). Recent usage adjustment factor: 0.5 if the layout has been used in the same time period in the last 3 days, otherwise 0.

[0137] Where a, b, c, and d are weighting coefficients, initially set to 0.368, 0.211, 0.263, and 0.158, respectively, and a+b+c+d=1 is maintained.

[0138] Time decay function: Uses an exponential decay function to reduce the priority of layouts that have not been used for a long time.

[0139] Time decay factor = e^(-λ × number of days unused)

[0140] Where λ is the decay rate, which is usually set between 0.1 and 0.3.

[0141] Example 5

[0142] Building upon Example 4, the scene layout restoration module is further described in detail. The scene layout restoration module detects changes in the software scene, identifies the current application combination scene based on the application combination, queries the application combination scene ID and its window layout, matches the best layout based on layout priority (if priorities are the same, the most recently used layout is used), and obtains the window configuration set to complete the layout restoration.

[0143] Specifically, when the scene layout restoration module detects a change in the software scene, it restores the scene according to the optimal layout given by the intelligent layout strategy module. After the user confirms, there may be minor adjustments, such as window layout or adding open applications. This result will be dynamically fed back to the system data acquisition module to collect data and update the application combination scene or layout strategy for subsequent selection. Through this cycle, it gradually learns and gets closer to the user's usage habits, thereby providing a more intelligent restoration.

[0144] Example 6

[0145] This embodiment is a specific application example of the system described in the above embodiments. The following uses a typical user scenario as an example to illustrate the workflow of the system of the present invention, including the following steps:

[0146] Step 1. The user turns on the machine and enters the desktop system, then opens the VSCode application, the terminal application, and the browser application;

[0147] Step 2. Based on the scene window layout stored in the smart layout strategy module, match the best layout for the current application group and execute automatically:

[0148] (1) Fix VSCode to the left half of the screen;

[0149] (2) The browser is fixed to the upper right half of the screen;

[0150] (3) The terminal is fixed on the lower half of the right side of the screen.

[0151] Step 3. The scene layout restoration module detects a new monitor connection, sets the screen to extended mode, and executes automatically:

[0152] (1) Pin VSCode to the first screen;

[0153] (2) The browser is pinned to the upper half of the second screen;

[0154] (3) The terminal is fixed in the lower half of the second screen.

[0155] Step 4. The user closes VS Code and the terminal, and opens WPS;

[0156] Step 5. Currently running WPS and a browser, the scene layout restoration module detects changes in the software scene. Based on the scene window layout stored in the intelligent layout strategy module, it matches the current application combination to obtain the most relevant application combination scene and executes it automatically:

[0157] (1) Pin WPS to the first screen;

[0158] (2) Pin the browser to the second screen;

[0159] (3) Open the file manager and hover it in the middle of the second screen.

[0160] Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

Claims

1. A multi-application scenario oriented window layout cross-screen cooperative nonlinear restoration system, characterized in that, The system comprises a system data collection module, an application scenario generation module, an intelligent layout strategy module and a scenario layout restoration module, wherein The system data collection module is in communication connection with the application scenario generation module, the intelligent layout strategy module and the scenario layout restoration module, and is responsible for collecting application-related data; The application scenario generation module is in communication connection with the system data collection module and the intelligent layout strategy module, analyzes the data collected by the system data collection module, performs multi-dimensional correlation strength analysis on the application, finally generates a high-correlation application combination scenario, and delivers it to the intelligent layout strategy module; The intelligent layout strategy module is in communication connection with the system data collection module, the application scenario generation module and the scenario layout restoration module, generates multiple sets of window layout snapshots for each scenario by combining the window layout data collected by the system data collection module with the application combination scenario; simultaneously, the snapshot priority is dynamically updated by a priority algorithm, so as to provide the scenario layout restoration module with the restoration of the scenario window layout; The scenario layout restoration module is in communication connection with the system data collection module and the intelligent layout strategy module, is responsible for the restoration of the scenario window layout, identifies the scenario when detecting the change of the application scenario, and then dynamically feeds back the recommended best layout to the system data collection module and restores it according to the snapshot priority delivered by the intelligent layout strategy module; The working steps of the application scenario generation module comprise the following: Step 1. Calculate the correlation strength between each pair of applications, and construct the correlation strength matrix between the applications; Step 2. Convert the correlation strength matrix into a distance matrix, perform hierarchical clustering, and arrange the clustering results; Step 3. Analyze the clustering results, obtain a high-correlation application group, and generate high-correlation application combination scenario information; Step 4. Store the correlation application group data; The working steps of the intelligent layout strategy module comprise the following: Step 1, window state vector: according to the window layout data collected by the system data collection module, the window state is vectorized, and a state vector is generated for each window; Step 2, generate scenario window layout snapshots: based on the application combination scenario generated by the application scenario generation module, all the window state vectors related to each application combination scenario are collected as a layout snapshot, and a parameter is attached to each layout snapshot; Step 3, layout similarity deduplication: calculate the similarity between different layouts, and merge the records when the similarity is more than 90%; Step 4, store the scenario window layout snapshots.

2. The multi-application-scene-oriented window layout cross-screen coordination non-linear restoration system according to claim 1, characterized in that, The system data collection module collects application-related data, including application usage data and window layout data; The application usage data includes application process ID and process name AppID, application start timestamp, application close timestamp, application active duration, application foreground and background switching time, application CPU usage, application memory usage, and application network access; The window layout data includes the coordinate position (x, y) of the application window, the width (w) and height (h) of the application window, the display MonitorID where the application window is located, the layering order, the window state, the split-screen information and the special area occupation. The system data collection module collects application-related data, which also includes application file system access, system clipboard usage, and display connection. 3.The multi-application scenario oriented window layout cross-screen cooperative nonlinear reduction system according to claim 1, characterized in that, Step 1 The correlation strength between each pair of applications is denoted as application A and application B, and the calculation formula is as follows: CorrelationStrength(A, B) = w1 T(A, B) + w2 R(A, B) + w3 I(A, B), wherein: T(A, B) is the time dimension correlation degree of application A and application B, R(A, B) is the resource dimension correlation degree of application A and application B, I(A, B) is the interaction dimension correlation degree of application A and application B, w1, w2, w3 are corresponding weight coefficients, and w1+w2+w3=1.

4. The multi-application-scene-oriented window layout cross-screen coordination non-linear restoration system according to claim 3, characterized in that, The time dimension correlation degree of application A and application B is calculated according to the following formula: T(A,B) = 0.4 Life cycle overlap + 0.4 Active time overlap + 0.2 Launch sequence correlation, wherein, The life cycle overlap degree = overlap time / total time, the overlap time refers to the overlap time of the life cycle of application A and application B, and the total time refers to the sum of the life cycle of application A and application B; The active time coincidence degree = common active time / total active time, the common active time refers to the overlap time of application A and application B in the foreground or the overlap time of application A and application B in the background, and the total active time refers to the sum of the application active time of application A and application B; The start sequence correlation degree = adjacent start times / total start times, the adjacent start times refer to the number of times that application A and application B are started adjacently within a certain period of time, and the total start times refer to the sum of the start times of application A and application B within a certain period of time.

5. The multi-application-scene-oriented window layout cross-screen coordination non-linear restoration system according to claim 3, characterized in that, The resource dimension correlation degree of application A and application B is calculated according to the following formula: R(A,B) = 0.4 Resource usage correlation + 0.3 IO operation correlation + 0.3 Network access correlation, wherein, Resource usage correlation = pearson(CPU_A, CPU_B) 0.5 + pearson(MEM_A, MEM_B) 0.5, pearson(CPU_A, CPU_B) is the Pearson correlation coefficient calculated for the CPU usage of application A and the CPU usage of application B; MEM_A, MEM_B are the application memory usage, and pearson(MEM_A, MEM_B) is the Pearson correlation coefficient calculated for the two sets of application memory usage. The IO operation correlation degree = common IO operation times / total IO operation times, the common IO operation times refer to the number of times that IO operations of application A and application B occur simultaneously within the same time period, and the total IO operation times refer to the sum of the IO operation times of application A and application B; The network access correlation degree = common network access times / total network access times, the common network access times refer to the number of times that application A and application B access the same domain name or the same ip+port, and the total network access times refer to the sum of the network access times of application A and application B.

6. The multi-application-scene-oriented window layout cross-screen coordination non-linear restoration system according to claim 3, characterized in that, The interaction dimension correlation degree of application A and application B is calculated according to the following formula: I(A,B) = 0.4 Window switching frequency + 0.3 Data exchange frequency + 0.3 User operation relevance, wherein, The window switching frequency = the number of times of switching between application A and application B / the total number of switching times, the data exchange frequency = the number of times of data exchange between application A and application B / the total number of data exchange times, and the user operation correlation degree = the number of related operations / the total number of operations, the related operations including window state adjustment, window state adjustment including maximizing, minimizing, window size adjustment, and window edge split screen. 7.The multi-application scenario oriented window layout cross-screen cooperative non-linear reduction system according to claim 1, characterized in that, The parameters in step 2 include: layout ID, scene ID, creation time, update time, cumulative use times, time period use count, duration, satisfaction score, window configuration set, display configuration, and layout priority; The layout priority is calculated by the following formula: The layout priority score = (a x recent use frequency + b x cumulative use duration) x time decay coefficient + c x satisfaction score + d x time matching degree, wherein, The time matching degree is the degree of overlap between the current use time of the layout snapshot and the historical use time period of the layout snapshot, the time period use rate = time period use count / cumulative use count, the time matching degree = time period use rate x (1 + recent use adjustment factor), a, b, c, and d are weight coefficients, the time period use rate is the historical use rate of the current time period, and the time decay coefficient = e^(-λ x unused days), wherein λ is the decay rate.

8. The multi-application-scene-oriented window layout cross-screen coordination non-linear restoration system according to claim 7, characterized in that, The scene layout restoring module detects software scene change, identifies current application combination scene according to application combination, queries application combination scene ID and window layout, matches optimal layout according to layout priority, and acquires window configuration set to complete layout restoring.

Images