Remote desktop resource scheduling method and device, electronic device and storage medium

Abstract

The application discloses a remote desktop resource scheduling method and device, electronic equipment and a storage medium, and relates to the technical field of cloud computing or other related technical fields, wherein the method comprises the following steps: collecting screen frame data of a remote desktop, and performing visual analysis on the screen frame data to obtain a task type and an operation activity score of a current task of the remote desktop; based on the task type and the operation activity score of the current task, a pre-constructed task graph is called to query a priority weight, a resource dependency node and an expected duration of the current task, and a task execution portrait containing a priority attribute is generated; according to the task execution portrait, multi-dimensional resource quotas are obtained from a resource scheduling pool to generate a resource scheduling scheme; and the resource scheduling scheme is executed to allocate target resources to the remote desktop. The application solves the technical problem of low stability of remote desktop connection in the related art based on a static and single remote desktop resource configuration strategy.

Description

Technical Field

[0001] This invention relates to the field of cloud computing technology or other related technical fields. Specifically, it relates to a remote desktop resource scheduling method and apparatus, electronic device and storage medium. Background Technology

[0002] Remote desktop technology, as a core supporting means for scenarios such as remote office work, remote operation and maintenance, cloud desktops, and online education, has been widely deployed in enterprise, educational institutions, and individual user environments. Its basic principle is to transmit the screen image and input commands of a remote host over a network, enabling users to control the target computer in real time. To achieve a smooth user experience, remote desktop systems continuously consume various computing and communication resources.

[0003] In related technologies, the allocation and management of remote desktop resources usually adopt a static, isolated, and one-dimensional control strategy. Resource quotas are manually preset or default values, which makes it difficult to quickly respond to the ever-changing network environment requirements, easily causing unstable remote desktop connections and affecting the normal operation of business.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a remote desktop resource scheduling method, apparatus, electronic device, and storage medium to at least solve the technical problem of low remote desktop connection stability in related technologies based on static and single remote desktop resource configuration strategies.

[0006] According to one aspect of the present invention, a remote desktop resource scheduling method is provided, comprising: collecting screen frame data of a remote desktop and performing visual analysis on the screen frame data to obtain the task type and operation activity score of the current task of the remote desktop; based on the task type and operation activity score of the current task, calling a pre-built task graph, querying the priority weight, resource dependency nodes and expected duration of the current task, and generating a task execution profile including priority attributes; obtaining multi-dimensional resource quotas from a resource scheduling pool according to the task execution profile, and generating a resource scheduling scheme, wherein the resource scheduling pool includes at least the following resources: network bandwidth, computing resources and storage resources; and executing the resource scheduling scheme to allocate target resources to the remote desktop.

[0007] Further, after executing the resource scheduling scheme and scheduling resources for the remote desktop, the method further includes: inputting the task execution profile and the historical operation behavior data of the remote desktop into the task prediction model; calculating the task transition probability through the task prediction model; and outputting the predicted task corresponding to the current task based on the task transition probability. The task prediction model is a pre-built machine model for task prediction. A resource reservation instruction is generated based on the predicted task, wherein the resource reservation instruction is used to pre-schedule resources for the predicted task from the resource scheduling pool before the predicted task is executed.

[0008] Further, the step of performing visual analysis on the screen frame data to obtain the task type of the current task of the remote desktop includes: performing image preprocessing on each screen frame in the screen frame data, extracting regional feature maps and filtering out static background pixels to generate a dynamic operation area mask; inputting the dynamic operation area into a classification model, identifying the interface element types in the screen frame through the classification model; and determining the task type of the current task of the remote desktop based on the interface element types.

[0009] Further, the step of performing visual analysis on the screen frame data to obtain the operation activity score of the current task on the remote desktop includes: extracting the dynamic operation area of ​​each screen frame from the screen frame data using an incremental frame difference algorithm; calculating the pixel change rate of each dynamic operation area; identifying the operation motion trajectory for each dynamic operation area using a target tracking algorithm; calculating the event interaction frequency based on the operation motion trajectory; and calculating the operation activity score based on the pixel change rate and the event interaction frequency.

[0010] Further, the step of obtaining multi-dimensional resource quotas from the resource scheduling pool based on the task execution profile and generating a resource scheduling scheme includes: calculating the urgency factor of the current task based on the priority weight and expected duration in the task execution profile; initiating a multi-dimensional resource query request to the resource scheduling pool based on the urgency factor and the resource dependency nodes in the task execution profile to query the available quotas and resource competition heat index of each resource; constructing a resource feasibility matrix based on the available quotas and resource competition heat index of each resource, combined with the resource quantity threshold required by the current task for each resource; determining the resource demand list and resource quotas based on the resource feasibility matrix, and generating the resource scheduling scheme based on the resource demand list and resource quotas.

[0011] Further, the step of determining the resource requirement list and resource quota based on the resource feasibility matrix includes: marking the current task as a resource-constrained task when the resource feasibility matrix indicates that any resource does not meet the current task's requirements; sorting all tasks executed by the remote desktop when the priority of the resource-constrained task is greater than a priority threshold, suspending the resource quota of preset tasks according to the sorting result, and preempting resources for the current task to obtain the preempted resource requirement list and resource quota; or, adjusting the resource quantity threshold required by the current task for each resource when the priority of the resource-constrained task is less than or equal to the priority threshold to obtain the adjusted resource requirement list and resource quota.

[0012] Furthermore, the user device is configured with multiple network interfaces, and the data transmission of the remote desktop is performed in parallel through the multiple network interfaces. The step of allocating target resources to the remote desktop includes: obtaining real-time network quality parameters of each network interface, wherein the real-time network quality parameters include at least one of the following: available bandwidth, network latency, packet loss rate, and billing mode of each network interface; matching the resource requirements in the task execution profile with the real-time parameters of each network interface, and calculating the comprehensive transmission performance score of each network interface; sorting the network interfaces based on the comprehensive transmission performance score to obtain the interface sorting result, and allocating the target resource quota in the resource scheduling scheme to the multiple network interfaces proportionally according to the interface sorting result, wherein the target resource quota includes at least: bandwidth quota.

[0013] According to another aspect of the present invention, a remote desktop resource scheduling device is also provided, comprising: an analysis unit, configured to collect screen frame data of a remote desktop and perform visual analysis on the screen frame data to obtain the task type and operation activity score of the current task of the remote desktop; a generation unit, configured to, based on the task type and operation activity score of the current task, invoke a pre-built task graph, query the priority weight, resource dependency nodes, and expected duration of the current task, and generate a task execution profile containing priority attributes; a scheduling unit, configured to, based on the task execution profile, obtain multi-dimensional resource quotas from a resource scheduling pool and generate a resource scheduling scheme, wherein the resource scheduling pool includes at least the following resources: network bandwidth, computing resources, and storage resources; and an allocation unit, configured to, execute the resource scheduling scheme, and allocate target resources to the remote desktop.

[0014] Furthermore, the remote desktop resource scheduling also includes: a first calculation module, used to input the task execution profile and the historical operation behavior data of the remote desktop into a task prediction model, calculate the task transition probability through the task prediction model, and output the predicted task corresponding to the current task based on the task transition probability, wherein the task prediction model is a pre-built machine model for task prediction; and a first generation module, used to generate a resource reservation instruction based on the predicted task, wherein the resource reservation instruction is used to pre-schedule resources for the predicted task from the resource scheduling pool before the predicted task is executed.

[0015] Furthermore, the analysis unit includes: a first extraction module, used to perform image preprocessing on each screen frame in the screen frame data, extract region feature maps and filter out static background pixels to generate a dynamic operation region mask; a first recognition module, used to input the dynamic operation region into a classification model and identify the interface element type in the screen frame through the classification model; and a first determination module, used to determine the task type of the current task on the desktop based on the interface element type.

[0016] Furthermore, the analysis unit further includes: a second extraction module, used to extract the dynamic operation region of each screen frame from the screen frame data using an incremental frame difference algorithm; a second calculation module, used to calculate the pixel change rate of each dynamic operation region; a second recognition module, used to recognize the operation motion trajectory for each dynamic operation region using a target tracking algorithm; a third calculation module, used to calculate the event interaction frequency based on the operation motion trajectory; and a fourth calculation module, used to calculate the operation activity score based on the pixel change rate and the event interaction frequency.

[0017] Further, the scheduling unit includes: a fifth calculation module, used to calculate the urgency factor of the current task based on the priority weight and expected duration in the task execution profile; a first query module, used to initiate a multi-dimensional resource query request to the resource scheduling pool based on the urgency factor and the resource dependency nodes in the task execution profile, to query the available quota and resource competition heat index of each resource; a first construction module, used to construct a resource feasibility matrix based on the available quota and resource competition heat index of each resource, combined with the resource quantity threshold required by the current task for each resource; and a second determination module, used to determine the resource demand list and resource quota based on the resource feasibility matrix, and generate the resource scheduling scheme based on the resource demand list and resource quota.

[0018] Further, the first determining module includes: a first marking submodule, used to mark the current task as a resource-constrained task when the resource feasibility matrix indicates that any resource does not meet the current task requirements; a first preemption submodule, used to sort all tasks executed by the remote desktop when the priority of the resource-constrained task is greater than a priority threshold, suspend the resource quota of preset tasks according to the sorting result, preempt resources for the current task, and obtain the preempted resource requirement list and the resource quota; and a first adjustment submodule, used to adjust the resource quantity threshold required by the current task for each resource when the priority of the resource-constrained task is less than or equal to the priority threshold, and obtain the adjusted resource requirement list and the resource quota.

[0019] Furthermore, the user device is configured with multiple network interfaces, and the data transmission of the remote desktop is performed in parallel through the multiple network interfaces. The allocation unit includes: a first acquisition module, used to acquire real-time network quality parameters of each network interface, wherein the real-time network quality parameters include at least one of the following: available bandwidth, network latency, packet loss rate, and billing mode of each network interface; a sixth calculation module, used to match the resource requirements in the task execution profile with the real-time parameters of each network interface, and calculate the comprehensive transmission performance score of each network interface; and a first allocation module, used to sort the network interfaces based on the comprehensive transmission performance score, obtain the interface sorting result, and allocate the target resource quota in the resource scheduling scheme to the multiple network interfaces proportionally according to the interface sorting result, wherein the target resource quota includes at least: bandwidth quota.

[0020] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute any of the above-described remote desktop resource scheduling methods.

[0021] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement any of the above-described remote desktop resource scheduling methods.

[0022] In this application, the following steps are performed: screen frame data of the remote desktop is collected, and visual analysis is performed on the screen frame data to obtain the task type and operation activity score of the current task of the remote desktop; based on the task type and operation activity score of the current task, a pre-built task graph is invoked to query the priority weight, resource dependency nodes and expected duration of the current task, and a task execution profile containing priority attributes is generated; multi-dimensional resource quotas are obtained from the resource scheduling pool according to the task execution profile, and a resource scheduling scheme is generated, wherein the resource scheduling pool includes at least the following resources: network bandwidth, computing resources and storage resources; the resource scheduling scheme is executed to allocate target resources to the remote desktop.

[0023] In this application, by real-time sensing of task types and calculation of operation activity scores, combined with a pre-built knowledge graph, the priority of the current task is identified and a task execution profile is generated. Based on the task execution profile, multi-dimensional resource quotas are obtained from the resource scheduling pool to achieve effective scheduling of multi-dimensional resources, ensure the stable operation of the remote desktop, improve the stability of the remote desktop connection, and thus solve the technical problem of low remote desktop connection stability in related technologies based on static and single remote desktop resource configuration strategies. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0025] Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a remote desktop resource scheduling method is shown.

[0026] Figure 2 This is a flowchart of an optional remote desktop resource scheduling method according to an embodiment of the present invention;

[0027] Figure 3 This is a flowchart of an optional visual analysis of screen frame data according to an embodiment of the present invention;

[0028] Figure 4 This is a flowchart of another optional visual analysis of screen frame data according to an embodiment of the present invention;

[0029] Figure 5 This is a schematic diagram of an optional remote desktop resource scheduling device according to an embodiment of the present invention;

[0030] Figure 6 This is a hardware structure block diagram of an electronic device (or mobile device) that performs an optional remote desktop resource scheduling method according to an embodiment of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] It should be noted that the remote desktop resource scheduling method and apparatus in this application can be used in the field of cloud computing technology for resource scheduling of remote desktops, and can also be used in any field other than cloud computing technology for resource scheduling of remote desktops. This application does not limit the application field of the remote desktop resource scheduling method and apparatus.

[0034] It should be noted that the information collected in this application (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of this data all comply with relevant laws, regulations, and standards, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding access points are provided for users to choose to authorize or refuse. For example, interfaces are set up between this system and relevant users or organizations, providing users with corresponding access points to choose to agree to or refuse automated decision-making results; if the user chooses to refuse, the process proceeds to the expert decision-making stage.

[0035] The following embodiments of the present invention can be applied to various remote desktop resource scheduling systems / applications / devices. The present invention collects screen frame data from the remote desktop and, based on visual analysis technology, automatically identifies the type of the current remote desktop task and quantifies its operational activity, thereby forming a fine-grained semantic perception of the user's current operational behavior. Based on this, a pre-built task graph is invoked, which pre-associates priority weights, resource dependency nodes, and expected durations corresponding to different task types and operational activity combinations, thereby generating a task execution profile containing clear resource requirement attributes. Based on this task execution profile, a resource request is initiated to a unified resource scheduling pool, which integrates heterogeneous resources. The system comprehensively calculates and generates a multi-dimensional resource quota scheduling scheme based on the priority, dependency relationships, and duration carried in the profile, achieving effective scheduling of multi-dimensional resources, ensuring the stable operation of the remote desktop, and effectively improving the connection stability of the remote desktop.

[0036] The present invention will now be described in detail with reference to various embodiments.

[0037] Example 1

[0038] According to an embodiment of the present invention, an embodiment of a remote desktop resource scheduling method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0039] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a remote desktop resource scheduling method is shown. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1The different configurations shown.

[0040] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0041] The memory 104 can be used to store software programs and modules of application software, such as the program instruction / data storage device corresponding to the remote desktop resource scheduling method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the aforementioned remote desktop resource scheduling method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0042] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0043] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0044] Under the aforementioned operating environment, this application provides the following: Figure 2 The remote desktop resource scheduling method shown is implemented by a remote desktop resource scheduling system.

[0045] Figure 2This is a flowchart of an optional remote desktop resource scheduling method according to an embodiment of the present invention, such as... Figure 2 As shown, the method includes the following steps:

[0046] Step S201: Collect screen frame data of the remote desktop and perform visual analysis on the screen frame data to obtain the task type and operation activity score of the current task on the remote desktop.

[0047] In step S201 above, remote desktop refers to a service system that displays and interactively controls the desktop environment of a remote host on a local device in real time through network protocols. Its core function is to synchronously transmit the graphical interface, input commands and audio and video streams of the remote computer to the local terminal to realize remote operation across devices.

[0048] Screen frame data refers to a continuous sequence of image frames generated by a remote desktop service within a unit of time. Each frame is a complete pixel snapshot of the remote desktop at a specific moment, containing all visual information about the screen content, such as window layout, text content, cursor position, and animation changes. Screen frame data is collected through the screen capture module of the remote desktop client. This module calls the graphics interface or utilizes a virtual display driver at the operating system level to periodically capture the rendering output of the remote desktop, forming a continuous data stream without interfering with normal user operations. Collecting screen frame data provides a direct visual representation of the remote desktop's running status, rather than relying on indirect information such as system logs, process names, or port traffic. This frame data carries the complete visual semantics of the user's actual actions, serving as the most original and accurate input source for identifying the nature of the current task and the intensity of interaction.

[0049] Visual analytics refers to the computer vision processing of acquired screen frame data sequences, automatically analyzing screen content through image processing algorithms and pattern recognition techniques. Specifically, this can include: calculating the pixel change rate between adjacent frames, detecting the movement trajectory of interface elements (such as window dragging and scroll bar sliding), recognizing the update frequency of text regions, analyzing the dynamic fluctuations of color distribution, and detecting cursor activity density and trajectory complexity. This transforms abstract user actions into quantifiable and comparable visual indicators, thereby enabling automatic identification of task types and objective assessment of activity levels.

[0050] The current task types include, but are not limited to: text editing, graphic design, video playback, 3D modeling and rendering, web browsing, data table manipulation, and silent operation. Each task type has stable and distinguishable visual characteristics: for example, text editing is characterized by local text updates and low inter-frame change rates; 3D modeling presents global view rotation, high-pixel rearrangement, and continuous motion flow. Different task types also have different resource requirements.

[0051] The activity level score is a quantitative indicator of the intensity of user interaction in the current task, calculated by weighting multiple features from visual analysis output. Its calculation factors include: inter-frame pixel change rate, cursor movement frequency, interface element displacement frequency, color fluctuation frequency, text input event frequency, and other event interaction frequencies.

[0052] Task type and activity level score together form the semantic input basis for remote desktop resource scheduling. Task type determines the category and structure of resource dependencies, while activity level score determines the intensity and persistence of resource demand. Combining the two allows for precise inference of the current combined demand for network bandwidth, computing resources, and storage resources for a given task, enabling precise resource control from fixed allocation to on-demand adaptation.

[0053] An alternative embodiment, Figure 3 This is a flowchart of an optional visual analysis of screen frame data according to an embodiment of the present invention, such as... Figure 3 As shown, the steps for visually analyzing screen frame data to obtain the task type of the current task on the remote desktop include: Step S301, performing image preprocessing on each screen frame in the screen frame data, extracting regional feature maps and filtering out static background pixels to generate a dynamic operation area mask; Step S302, inputting the dynamic operation area into a classification model, and identifying the interface element types in the screen frames through the classification model; Step S303, determining the task type of the current task on the remote desktop based on the interface element types.

[0054] Furthermore, after obtaining the screen frame data from the remote desktop, image preprocessing is first performed on each screen frame. This preprocessing includes standardization and noise suppression of the raw screen frame data to improve the accuracy of subsequent analysis. Then, through edge detection and optical flow analysis, the motion gradient and spatial change intensity of pixels in each screen frame are calculated to generate a feature map reflecting the local dynamic activity level. In this feature map, high-response areas correspond to areas of screen content change (such as scroll bar movement, window dragging, and text input), while low-response areas are static backgrounds (such as desktop wallpaper, fixed toolbars, and static icons). Filtering out static background pixels is based on multi-frame cumulative comparison, identifying areas whose position and pixel values ​​have not changed in several consecutive frames, marking them as background, and masking them from the feature map. Finally, a dynamic operation region mask is generated to retain the binary mask image of the currently visually changing area. Its function is to focus on the active areas of actual user interaction, excluding irrelevant background information, and significantly reducing the computational load and risk of misjudgment in subsequent analysis.

[0055] The active region image patches cropped from the above mask are fed into a pre-trained convolutional neural network classifier. This classification model has learned from a large number of labeled samples during the training phase and can identify interface element types, such as text input boxes, buttons, scroll bars, menu bars, chart areas, video playback windows, and 3D rendering viewports. The model output is a set of probability distributions indicating the most likely category of the interface element contained in the current active region. For example, when a high-density rectangular text region and a blinking cursor feature are detected, the model outputs a probability of 0.92 for the text editor category; when continuously rotating 3D meshes and multi-light source shadow changes are detected, the output outputs a probability of 0.89 for the 3D modeling environment category.

[0056] The classification model outputs interface element categories and maps them to a pre-defined task type library. This library consists of a set of manually defined mapping rules. For example, if an interface element is a text input box, a blinking cursor, or has low motion amplitude, the task type is mapped to text editing; if an interface element is a 3D mesh, involves multi-view rotation, or has high texture variation, the task type is mapped to 3D modeling. This mapping process does not rely on a single frame; instead, it combines classification results from multiple consecutive frames through majority voting to improve robustness and achieve fine-grained task type identification.

[0057] An alternative embodiment, Figure 4 This is another optional flowchart for visual analysis of screen frame data according to an embodiment of the present invention, such as... Figure 4 As shown, the steps for visually analyzing screen frame data to obtain the operation activity score of the current task on the remote desktop include: Step S401, using an incremental frame difference algorithm to extract the dynamic operation area of ​​each screen frame from the screen frame data; Step S402, calculating the pixel change rate of each dynamic operation area; Step S403, for each dynamic operation area, using a target tracking algorithm to identify the operation motion trajectory; Step S404, calculating the event interaction frequency based on the operation motion trajectory; Step S405, calculating the operation activity score based on the pixel change rate and the event interaction frequency.

[0058] Furthermore, the calculation of the operation activity score abstracts user operation behavior into measurable and comparable numerical indicators. First, the incremental frame difference algorithm is used to calculate the difference between the corresponding pixel positions of two adjacent frames and retain the area where the pixel value change exceeds the preset threshold. In this way, the visual change part caused by user operation can be efficiently identified from the continuous screen frame sequence, thereby separating the dynamic operation area.

[0059] Subsequently, for each dynamic operation area, the sum of the absolute differences between all pixels and the previous frame is calculated and normalized to the proportion of the total number of pixels in that area, yielding the pixel change rate. This pixel change rate reflects the intensity of content change in that area per unit time. For example, during document scrolling, the entire text area shifts as a whole, resulting in a moderate pixel change rate; however, during 3D model rotation, a large number of pixels undergo non-rigid rearrangement due to perspective transformation, significantly increasing the pixel change rate. The pixel change rate is an important indicator for quantifying visual dynamics; a higher value indicates more drastic changes in interface content and more intensive user operations.

[0060] For each dynamic operation area, within the dynamic area, a pre-defined target tracking algorithm is used to continuously locate and track the movement path of the operated object. This identifies interactive objects with clear spatial forms, such as cursors, window boundaries, scroll bars, and selected text boxes. The role of target tracking is to associate scattered pixel changes into interpretable operational behavior units. For example, the process of moving the cursor from point A to point B in consecutive frames is identified as a drag event or navigation action, rather than meaningless random noise. This allows the evaluation of operation activity to move from the amount of pixel changes to the number of behavioral events, enhancing semantic relevance.

[0061] Based on the identified motion trajectories, the number of independent interaction events occurring per unit time is counted to obtain the event interaction frequency, which is used to measure the rhythm density of user operations. For example, frequent keystrokes accompanied by cursor movement during text editing may generate high-frequency, small-amplitude events; while slow model rotation in 3D modeling may generate low-frequency, large-scale events. This metric reflects the frequency dimension of user input behavior, complementing the intensity dimension of pixel change rate.

[0062] Finally, the two quantitative indicators mentioned above are weighted and fused together to calculate the operation activity score based on pixel change rate and event interaction frequency, generating a comprehensive score between 0 and 1. The weight allocation is set according to the statistical distribution of typical tasks. For example, the pixel change rate weight is set to 0.6 to reflect the physical intensity of visual content changes, and the event interaction frequency weight is set to 0.4 to reflect the rhythm density of user operation behavior.

[0063] Step S202: Based on the task type and operation activity score of the current task, call the pre-built task graph, query the priority weight, resource dependency nodes and expected duration of the current task, and generate a task execution profile containing priority attributes.

[0064] In step S202 above, the pre-constructed task graph is a structured knowledge base that stores the semantic features and resource association rules of various tasks in the remote desktop environment in the form of graph data. Its nodes represent task types, and edges represent resource dependencies and behavioral evolution patterns between tasks. Each node has three attribute fields: priority weight, resource dependent nodes, and expected duration. The priority weight is a normalized value between 0 and 1, used to characterize the priority of the task in occupying system resources in a multi-task concurrent scenario; the resource dependent nodes refer to the main types of resources consumed by the task during runtime and their relative proportions, such as "GPU encoding load accounts for 70%, memory cache usage accounts for 50%, and network bandwidth requirement is 30%"; the expected duration is the typical duration range of this type of task based on historical behavior statistics, such as text editing averaging 27 minutes and 3D modeling averaging 85 minutes.

[0065] After obtaining the current task type and operation activity score, the system searches for matching nodes in the task graph. The task type is used to locate the main node in the graph, and the operation activity score is used to dynamically interpolate the parameter values ​​of that node. For example, when the task type is 3D modeling and the operation activity score is 0.9, the system reads the default priority weight of the 3D modeling node from the graph as 0.85, and the resource-dependent nodes as {GPU: 0.7, Cache: 0.6, Bandwidth: 0.3}, with an expected duration of 60-120 minutes. Subsequently, based on the offset of the activity score, the resource-dependent nodes are scaled up proportionally, such as increasing the GPU load weight to 0.82 and the cache space pre-allocation to 0.75, to reflect the increased resource demand under high-intensity interaction.

[0066] The retrieved priority weights, resource dependency nodes, and expected duration are integrated into a structured data object to form a task execution profile. This profile not only includes static task attributes but also incorporates dynamic adjustment parameters based on current operational activity, creating a task requirement specification that can be directly consumed by the resource scheduling system. For example, a typical task execution profile might be represented as: {Task Type: 3D Modeling, Priority Weight: 0.88, Resource Dependency Nodes: {GPU: 0.82, Cache: 0.75, Bandwidth: 0.35}, Expected Duration: 95 minutes}. The priority attribute specifically refers to the priority weight value carried in this profile, used to compete for ranking with other task profiles in the resource scheduling pool, determining the order of resource allocation.

[0067] Step S203: Obtain multi-dimensional resource quotas from the resource scheduling pool based on the task execution profile, and generate a resource scheduling scheme.

[0068] It should be noted that the resource scheduling pool includes at least the following resources: network bandwidth, computing resources, and storage resources. The resource scheduling pool is a unified resource pool that centrally manages network bandwidth, computing resources, and storage resources. Internally, it uses quantifiable metrics to register the status and track the remaining capacity of various resources. The resource scheduling pool updates the occupancy status of each resource in real time, ensuring that resource allocation is based on the current actual availability.

[0069] In step S203 above, the priority weights, resource dependency nodes, and expected duration contained in the task execution profile are used as input parameters to trigger the resource scheduling algorithm to dynamically allocate resources in the pool. The priority weights determine the task's ranking priority in resource contention, with higher-priority tasks receiving resources first. The proportions of each dimension in the resource dependency nodes (e.g., GPU: 0.82, cache: 0.75, bandwidth: 0.35) guide the resource quota allocation structure, ensuring that the allocation ratio matches the actual needs of the task. The expected duration determines the length of time resources are reserved; the system pre-allocates resources and sets an automatic release time window accordingly to prevent short-term tasks from occupying resources for extended periods. Subsequently, the allocation results are structured into an executable instruction set to generate a resource scheduling scheme.

[0070] Through the above steps, abstract priorities, dependencies, and durations are transformed into quantifiable, competitive, and time-limited resource quotas, breaking through the limitations of traditional static bandwidth limiting or coarse-grained resource reservation.

[0071] An optional embodiment of the step of obtaining multi-dimensional resource quotas from the resource scheduling pool based on the task execution profile and generating a resource scheduling scheme includes: calculating the urgency factor of the current task based on the priority weight and expected duration in the task execution profile; initiating a multi-dimensional resource query request to the resource scheduling pool based on the urgency factor and the resource dependency nodes in the task execution profile to query the available quotas and resource competition heat index of each resource; constructing a resource feasibility matrix based on the available quotas and resource competition heat index of each resource and the resource quantity threshold required by the current task for each resource; determining the resource demand list and resource quotas based on the resource feasibility matrix, and generating a resource scheduling scheme based on the resource demand list and resource quotas.

[0072] Furthermore, the urgency factor is a comprehensive quantitative indicator that measures the urgency of resource acquisition for the current task. It is calculated by combining the priority weight in the task execution profile with the expected duration. The formula is: Urgency Factor = Priority Weight × (1 + 1 / (Expected Duration / Baseline Duration)), where the baseline duration is the system's preset typical task duration. The urgency factor gives higher priority and shorter duration tasks greater urgency, requiring rapid response and resource release as quickly as possible; while high-priority and long-duration tasks, although high-priority, have their urgency moderately reduced due to their longer occupation period, preventing long-term resource locking. The purpose of the urgency factor is to dynamically distinguish between tasks that must be responded to immediately and those that can be scheduled later in resource competition, avoiding resource rigidity caused by simply prioritizing tasks.

[0073] The priority and granularity of queries are determined by the urgency factor. High-urgency tasks trigger real-time synchronous queries, simultaneously requesting the current available quota and resource contention intensity index of resources such as network bandwidth, GPU (Graphics Processing Unit) encoding capacity, and memory cache. Available resource quota refers to the currently unused resource capacity, such as 40Mbps of available bandwidth, 15% GPU load margin, and 800MB of free cache space. The resource contention intensity index is a statistical analysis of the instantaneous request density of each type of resource within the resource scheduling pool, thereby obtaining the resource supply status and the intensity of contention, providing an environmental awareness basis for subsequent decision-making.

[0074] Subsequently, based on the available quota and resource competition intensity index of each resource, combined with the resource quantity threshold required by the current task for each resource, a resource feasibility matrix is ​​constructed. Rows represent resource types (e.g., bandwidth, GPU, cache), and columns represent evaluation dimensions (available quota, competition intensity, demand threshold). Each row is calculated with a feasibility score: Feasibility Score = Available Quota / Demand Threshold × (1 - Competition Intensity Index). This score reflects whether the resource can meet the demand and avoid system-level congestion. Resource items with a score below 0.3 are marked as unsatisfactory and require downgrading.

[0075] The system filters all resource items with feasibility scores higher than a threshold from the matrix to form a list of achievable requirements. Based on the resource dependency ratios in the task execution profile, the system allocates the maximum feasible quota proportionally. For example, if bandwidth is infeasible but GPU and cache are feasible, the system reduces the bandwidth requirement from 45Mbps to 30Mbps (below the minimum guaranteed value of the threshold), while simultaneously increasing the GPU and cache quotas proportionally to ensure that the total resource supply is consistent with the task profile structure. The final resource requirement list includes the target value and minimum guaranteed value for each resource, and the resource quota is the actual allocated value.

[0076] The above results are encapsulated into structured instructions, including: resource type, target quota, minimum guaranteed quota, effective time, and release conditions, thereby generating a resource scheduling scheme. This scheme can directly drive the underlying protocol engine and resource management module to perform dynamic adjustments, achieving coordinated responses such as bandwidth reduction, frame rate control, and cache expansion.

[0077] An optional embodiment of determining a resource requirement list and resource quota based on a resource feasibility matrix includes: marking the current task as a resource-constrained task when the resource feasibility matrix indicates that any resource does not meet the current task's requirements; sorting all tasks executed on the remote desktop when the priority of the resource-constrained task is greater than a priority threshold, suspending the resource quota of preset tasks according to the sorting result, preempting resources for the current task, and obtaining a preempted resource requirement list and resource quota; or, adjusting the resource quantity threshold required by the current task for each resource when the priority of the resource-constrained task is less than or equal to a priority threshold, and obtaining an adjusted resource requirement list and resource quota.

[0078] Furthermore, when the feasibility score of any resource in the resource feasibility matrix falls below a preset threshold, it is determined that the current task cannot fully meet its requirements under the current resource conditions, and the system sets the task status to resource-constrained. This flag is used to trigger subsequent resource coordination mechanisms, distinguishing it from resource-sufficient tasks, indicating that its operation will face performance degradation or waiting risks. Its purpose is to identify high-demand tasks with bottlenecks in the current resource environment, providing a basis for subsequent differentiated processing decisions.

[0079] When a task's priority weight exceeds a system-defined threshold, it is deemed a critical task and eligible for resource preemption. The system first sorts all active remote desktop tasks in descending order of priority weight. Then, it selects pauseable tasks from the lower priority tasks, such as non-core interactive tasks like web browsing and file downloads, and suspends a portion of their resource quota. The pause operation does not terminate task execution; it only temporarily reclaims cache space, reduces bandwidth quota, limits GPU usage, and records the pause duration and resource usage for later recovery. After resource preemption is complete, the system reallocates the reclaimed resources to the current task, ensuring its resource quota meets the original demand threshold, creating a new resource demand list and quota. This mechanism ensures that high-priority tasks maintain a stable experience even under resource constraints, implementing a critical task priority strategy.

[0080] When a task's priority does not meet the preemption criteria, the system does not preempt resources but instead performs a degraded adaptation. Based on the proportion of resource dependency nodes in the task execution profile, the system sequentially reduces the threshold requirements of each resource. For example, if the task is "video playback" with dependencies of {bandwidth: 0.8, cache: 0.5, GPU: 0.2}, but the available bandwidth quota is insufficient, the bandwidth threshold is prioritized to be reduced (e.g., from 50Mbps to 30Mbps), while the cache and GPU requirements are proportionally reduced (e.g., to 30MB and 0.12 respectively), ensuring that the resource quota maintains the structural consistency of the task profile. The adjusted resource requirement list reflects the minimum acceptable requirements, and the resource quota corresponds to the maximum compliant value currently allocable by the scheduling pool, forming a degraded but still workable resource configuration scheme.

[0081] Step S204: Execute the resource scheduling scheme to allocate target resources to the remote desktop.

[0082] In step S204 above, the structured instructions included in the generated resource scheduling scheme, such as resource type, target quota, minimum guaranteed quota, effective time, and release conditions, are transformed item by item into control commands for the underlying protocol stack and system resource management module, and executed synchronously. Based on the parameters specified in the resource scheduling scheme, the resources upon which the remote desktop service depends are precisely configured. Network bandwidth allocation is achieved through a network traffic shaper. The system sets token bucket or leaky bucket rate limits to control the transmission rate of the remote desktop data stream within the target quota range, ensuring that a basic frame rate is maintained even during network fluctuations. Computational resource allocation is achieved by the scheduler calling the GPU memory management interface and CPU thread pool control module to dynamically adjust the number of encoder threads, the size of the image rendering frame buffer, and the complexity of the compression algorithm. Storage resource allocation is achieved by the memory manager reserving a specified amount of cache space for the remote desktop process to store frequently updated screen differential frames and retransmission queues, reducing the retransmission overhead caused by network packet loss.

[0083] During execution, the system continuously monitors the actual resource usage. If there is a deviation between the target quota and the current operating status, a compensation mechanism is immediately triggered. For example, when the actual bandwidth usage is detected to be close to the minimum guaranteed value but still lower than the target quota, the system will slowly increase the transmission rate and restore the image resolution and frame rate. If the actual available amount of a resource is lower than the guaranteed value due to external interference (such as sudden traffic from other applications), the system will initiate an early warning and trigger a reassessment process for the resource scheduling scheme.

[0084] In one optional embodiment, multiple network interfaces are configured on the user device, and data transmission of the remote desktop is performed in parallel through multiple network interfaces. The step of allocating target resources to the remote desktop includes: obtaining real-time network quality parameters of each network interface, wherein the real-time network quality parameters include at least one of the following: available bandwidth, network latency, packet loss rate, and billing mode of each network interface; matching the resource requirements in the task execution profile with the real-time parameters of each network interface, and calculating the comprehensive transmission performance score of each network interface; sorting each network interface based on the comprehensive transmission performance score to obtain the interface sorting result; and allocating the target resource quota in the resource scheduling scheme to multiple network interfaces proportionally according to the interface sorting result, wherein the target resource quota includes at least: bandwidth quota.

[0085] It should be noted that user terminal devices (such as laptops, tablets, or smart terminals) can simultaneously possess two or more independent network connectivity capabilities, such as Wi-Fi, 4G, 5G, and Ethernet. Each interface can independently access different network links. Based on this, when transmitting data, the remote desktop service is not limited to using a single network channel, but instead splits the data stream into multiple sub-streams and sends them synchronously through multiple network interfaces to achieve multi-path concurrent transmission.

[0086] Furthermore, through the network monitoring module, dynamic performance indicators of each available network interface (such as Wi-Fi, 4G, 5G, and Ethernet) on the user equipment are collected in real time. Specifically, these indicators can include: available bandwidth, network latency, packet loss rate, and billing mode for each network interface. Available bandwidth refers to the maximum downlink transmission capacity that the current interface can stably use without causing congestion; network latency reflects the one-way time from when a data packet is sent from the device to when it responds to the remote server; packet loss rate indicates the proportion of data packets that fail to reach their destination within a certain sampling period, reflecting link stability; and the billing mode indicates whether the interface is billed by traffic or free, used for cost-sensitive resource allocation decisions. This establishes a real-time awareness capability of the multi-path network environment, providing an objective basis for subsequent resource allocation.

[0087] The bandwidth requirement threshold, latency tolerance range, and stability requirements defined in the task execution profile are quantitatively compared with the real-time parameters of each interface. The adaptability of each parameter is weighted and synthesized according to the weight of the resource-dependent nodes in the task execution profile to form the comprehensive transmission performance score of the interface. All interfaces are ranked from high to low according to their comprehensive transmission performance scores. Based on the ranking results, the bandwidth quota specified in the resource scheduling scheme is dynamically allocated according to the transmission performance scores and proportions. The allocation principle is that high-scoring interfaces are given priority to receive a larger share, while low-scoring interfaces assume a supplementary or backup role. At the same time, the billing mode is used as an adjustment factor. If a high-scoring interface is billed based on traffic, the system can appropriately reduce its allocation ratio and migrate some traffic to free interfaces to achieve dual optimization of performance and cost.

[0088] By employing a multi-interface collaborative scheduling mechanism, the allocation of bandwidth resources for remote desktops is upgraded from a single-link constraint to a multi-path collaborative optimization. The comprehensive transmission performance score integrates performance and cost dimensions, ensuring that resource allocation no longer relies on static strategies or manual selection, but rather dynamically selects the best option based on the real-time environment.

[0089] In one optional embodiment, after executing the resource scheduling scheme and scheduling resources for the remote desktop, the method further includes: inputting the task execution profile and historical operation behavior data of the remote desktop into the task prediction model; calculating the task transition probability through the task prediction model; and outputting the predicted task corresponding to the current task based on the task transition probability. The task prediction model is a pre-built machine model for task prediction. A resource reservation instruction is generated based on the predicted task. The resource reservation instruction is used to pre-schedule resources for the predicted task from the resource scheduling pool before the predicted task is executed.

[0090] Furthermore, this embodiment of the invention introduces a task prediction mechanism to reserve resources for the next task corresponding to the current task. Specifically, the structured features of the current task (such as priority weights, resource dependency nodes, expected duration, and operation type) and the user's remote desktop operation behavior sequence over a past period are used as input features and fed into a pre-trained task prediction model. This task prediction model is a machine learning model based on sequence learning, which can employ a long short-term memory network. Its training data comes from historical remote desktop session logs, including task switching timestamps, resource consumption changes, user interaction frequency, and operation intent labels. After the input data is standardized, the model extracts temporal dependencies and identifies the transition patterns between the current task and subsequent possible tasks.

[0091] The task prediction model outputs a conditional probability distribution of the transition from the current task to other potential tasks, transforming vague user behavior trends into quantifiable future task expectations and providing a forward-looking basis for resource scheduling. Several potential tasks with the highest probabilities are selected as prediction results for subsequent tiered resource reservation. Based on the resource dependency nodes of the predicted tasks and their predicted probabilities, priority-based resource pre-scheduling instructions are generated. These instructions differ from the formal resource scheduling scheme; they reserve rather than lock resources. The resource reservation instructions are sent to the resource scheduling pool, which reserves the corresponding resource capacity for the predicted tasks in the background without interrupting the current task's execution. After the predicted task starts, the system activates the pre-allocated resources, achieving zero-wait switching; if not triggered, the reserved resources will be automatically returned after a timeout, avoiding resource waste.

[0092] Another optional embodiment involves obtaining the current usage scenario information, network quality parameters, and user historical behavior data of the remote desktop, and combining the usage scenario information, network quality parameters, and user historical behavior data to assist in the allocation of bandwidth resources.

[0093] Furthermore, based on usage scenario information and network quality parameters, the protocol bandwidth is set. Specifically, this includes: pre-establishing a correspondence table between usage scenario information, network quality parameters, and protocol bandwidth; querying the correspondence table based on the acquired usage scenario information and network quality parameters to determine the initial protocol bandwidth; using a machine learning model to analyze historical user behavior data and network performance data to predict changes in user bandwidth requirements under specific scenarios, adjusting the bandwidth usage of the remote desktop service in advance, and adjusting the initial protocol bandwidth based on the prediction results to obtain the final protocol bandwidth. Adjusting the initial protocol bandwidth can be achieved by adjusting the resolution or color quality of the desktop image and adjusting the data transmission rate of the remote desktop service. In addition, the correspondence table and the prediction parameters of the machine learning model are updated and optimized periodically or when the network environment changes. Network quality parameters can include: real-time network bandwidth values, network latency values, and packet loss rate.

[0094] Through the above steps, screen frame data of the remote desktop is collected and visually analyzed to obtain the task type and operation activity score of the current task on the remote desktop. Based on the task type and operation activity score of the current task, a pre-built task graph is invoked to query the priority weight, resource dependency nodes, and expected duration of the current task, generating a task execution profile containing priority attributes. Based on the task execution profile, multi-dimensional resource quotas are obtained from the resource scheduling pool to generate a resource scheduling plan. The resource scheduling pool includes at least the following resources: network bandwidth, computing resources, and storage resources. The resource scheduling plan is executed to allocate target resources to the remote desktop.

[0095] In this embodiment, by real-time sensing of task types and calculation of operation activity scores, combined with a pre-built knowledge graph, the priority of the current task is identified and a task execution profile is generated. Based on the task execution profile, multi-dimensional resource quotas are obtained from the resource scheduling pool to achieve effective scheduling of multi-dimensional resources, ensure the stable operation of the remote desktop, improve the stability of the remote desktop connection, and thus solve the technical problem of low remote desktop connection stability in related technologies based on static and single remote desktop resource configuration strategies.

[0096] The following is a detailed description with reference to another embodiment.

[0097] Example 2

[0098] The remote desktop resource scheduling device provided in this embodiment includes multiple implementation units, each of which corresponds to a specific implementation step in the above embodiment one. The specific implementation method and beneficial effects can be referred to the foregoing method embodiment, and will not be repeated here.

[0099] Figure 5 is a schematic diagram of an optional remote desktop resource scheduling device according to an embodiment of the present invention, such as... Figure 5 As shown, the remote desktop resource scheduling device may include: an analysis unit 51, a generation unit 52, a scheduling unit 53, and an allocation unit 54, wherein,

[0100] Analysis unit 51 is used to collect screen frame data of the remote desktop and perform visual analysis on the screen frame data to obtain the task type and operation activity score of the current task of the remote desktop.

[0101] The generation unit 52 is used to generate a task execution profile containing priority attributes by calling a pre-built task graph based on the task type and operation activity score of the current task, querying the priority weight, resource dependency nodes and expected duration of the current task;

[0102] The scheduling unit 53 is used to obtain multi-dimensional resource quotas from the resource scheduling pool based on the task execution profile and generate a resource scheduling scheme. The resource scheduling pool includes at least the following resources: network bandwidth, computing resources and storage resources.

[0103] The allocation unit 54 is used to execute a resource scheduling scheme and allocate target resources to the remote desktop.

[0104] The aforementioned remote desktop resource scheduling device collects screen frame data from the remote desktop through the analysis unit 51 and performs visual analysis on the screen frame data to obtain the task type and operation activity score of the current task on the remote desktop. Based on the task type and operation activity score of the current task, the generation unit 52 calls a pre-built task graph to query the priority weight, resource dependency nodes, and expected duration of the current task, generating a task execution profile containing priority attributes. Based on the task execution profile, the scheduling unit 53 obtains multi-dimensional resource quotas from the resource scheduling pool and generates a resource scheduling scheme. The resource scheduling pool includes at least the following resources: network bandwidth, computing resources, and storage resources. The allocation unit 54 executes the resource scheduling scheme to allocate target resources to the remote desktop.

[0105] In this embodiment, by real-time sensing of task types and calculation of operation activity scores, combined with a pre-built knowledge graph, the priority of the current task is identified and a task execution profile is generated. Based on the task execution profile, multi-dimensional resource quotas are obtained from the resource scheduling pool to achieve effective scheduling of multi-dimensional resources, ensure the stable operation of the remote desktop, improve the stability of the remote desktop connection, and thus solve the technical problem of low remote desktop connection stability in related technologies based on static and single remote desktop resource configuration strategies.

[0106] Furthermore, remote desktop resource scheduling also includes: a first calculation module, used to input task execution profiles and historical operation behavior data of the remote desktop into a task prediction model, calculate the task transition probability through the task prediction model, and output the predicted task corresponding to the current task based on the task transition probability, wherein the task prediction model is a pre-built machine model for task prediction; and a first generation module, used to generate resource reservation instructions based on the predicted task, wherein the resource reservation instructions are used to pre-schedule resources for the predicted task from the resource scheduling pool before the predicted task is executed.

[0107] Furthermore, the analysis unit includes: a first extraction module, used to perform image preprocessing on each screen frame in the screen frame data, extract regional feature maps and filter out static background pixels to generate a dynamic operation area mask; a first recognition module, used to input the dynamic operation area into a classification model and identify the interface element type in the screen frame through the classification model; and a first determination module, used to determine the task type of the current task on the desktop based on the interface element type.

[0108] Furthermore, the analysis unit also includes: a second extraction module, used to extract the dynamic operation area of ​​each screen frame from the screen frame data using an incremental frame difference algorithm; a second calculation module, used to calculate the pixel change rate of each dynamic operation area; a second recognition module, used to recognize the operation motion trajectory for each dynamic operation area using a target tracking algorithm; a third calculation module, used to calculate the event interaction frequency based on the operation motion trajectory; and a fourth calculation module, used to calculate the operation activity score based on the pixel change rate and the event interaction frequency.

[0109] Furthermore, the scheduling unit includes: a fifth calculation module, used to calculate the urgency factor of the current task based on the priority weight and expected duration in the task execution profile; a first query module, used to initiate a multi-dimensional resource query request to the resource scheduling pool based on the urgency factor and the resource dependency nodes in the task execution profile, to query the available quota and resource competition heat index of each resource; a first construction module, used to construct a resource feasibility matrix based on the available quota and resource competition heat index of each resource, combined with the resource quantity threshold required by the current task for each resource; and a second determination module, used to determine the resource demand list and resource quota based on the resource feasibility matrix, and generate a resource scheduling scheme based on the resource demand list and resource quota.

[0110] Further, the first determining module includes: a first marking submodule, used to mark the current task as a resource-constrained task when the resource feasibility matrix indicates that any resource does not meet the current task requirements; a first preemption submodule, used to sort all tasks executed on the remote desktop when the priority of the resource-constrained task is greater than the priority threshold, suspend the resource quota of the preset task according to the sorting result, preempt resources for the current task, and obtain a resource requirement list and resource quota after preemption; and a first adjustment submodule, used to adjust the resource quantity threshold required by the current task for each resource when the priority of the resource-constrained task is less than or equal to the priority threshold, and obtain an adjusted resource requirement list and resource quota.

[0111] Furthermore, the user device is configured with multiple network interfaces, and data transmission for the remote desktop is performed in parallel through these multiple network interfaces. The allocation unit includes: a first acquisition module, used to acquire real-time network quality parameters of each network interface, wherein the real-time network quality parameters include at least one of the following: available bandwidth, network latency, packet loss rate, and billing mode of each network interface; a sixth calculation module, used to match the resource requirements in the task execution profile with the real-time parameters of each network interface, and calculate the comprehensive transmission performance score of each network interface; and a first allocation module, used to sort each network interface based on the comprehensive transmission performance score, obtain the interface sorting result, and allocate the target resource quota in the resource scheduling scheme to multiple network interfaces proportionally according to the interface sorting result, wherein the target resource quota includes at least: bandwidth quota.

[0112] It should be noted that the analysis unit 51, generation unit 52, scheduling unit 53, and allocation unit 54 mentioned above correspond to steps S201 to S204 in Embodiment 1. The instances and application scenarios implemented by the above units and corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules or units can be hardware or software components stored in memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above modules or units can also run as part of a device in the computer terminal 10 provided in Embodiment 1.

[0113] The invention will now be described in conjunction with another alternative embodiment.

[0114] Example 3

[0115] The present invention can also provide an electronic device. Figure 6 This is a hardware structure block diagram of an electronic device (or mobile device) for executing a remote desktop resource scheduling method according to an embodiment of the present invention, such as... Figure 6 As shown, the electronic device may include: one or more ( Figure 6 (Only one is shown) Processor 602, memory 606, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0116] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the above-described methods. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0117] The processor can access information and applications stored in memory via a transmission device to perform the following steps: Collect screen frame data from the remote desktop and perform visual analysis on the screen frame data to obtain the task type and activity score of the current task on the remote desktop; based on the task type and activity score of the current task, call a pre-built task graph to query the priority weight, resource dependency nodes, and expected duration of the current task, generating a task execution profile containing priority attributes; obtain multi-dimensional resource quotas from the resource scheduling pool based on the task execution profile, and generate a resource scheduling scheme, wherein the resource scheduling pool includes at least the following resources: network bandwidth, computing resources, and storage resources; execute the resource scheduling scheme to allocate target resources to the remote desktop.

[0118] The processor can access information and applications stored in memory via a transmission device to perform the following steps: inputting task execution profiles and historical operation behavior data of the remote desktop into the task prediction model; calculating the task transition probability through the task prediction model; and outputting the predicted task corresponding to the current task based on the task transition probability. Here, the task prediction model is a pre-built machine model for task prediction; and generating resource reservation instructions based on the predicted task. These resource reservation instructions are used to pre-schedule resources for the predicted task from the resource scheduling pool before the predicted task is executed.

[0119] The processor can access information and applications stored in memory via a transmission device to perform the following steps: preprocessing images of each screen frame in the screen frame data, extracting regional feature maps and filtering out static background pixels to generate a dynamic operation area mask; inputting the dynamic operation area into a classification model to identify the interface element types in the screen frames; and determining the task type of the current task on the desktop based on the interface element types.

[0120] The processor can access information and applications stored in memory via a transmission device to perform the following steps: extracting the dynamic operation region of each screen frame from the screen frame data using an incremental frame difference algorithm; calculating the pixel change rate of each dynamic operation region; identifying the operation motion trajectory for each dynamic operation region using a target tracking algorithm; calculating the event interaction frequency based on the operation motion trajectory; and calculating the operation activity score based on the pixel change rate and the event interaction frequency.

[0121] The processor can access information and applications stored in memory via a transmission device to perform the following steps: Calculate the urgency factor of the current task based on the priority weight and expected duration in the task execution profile; Based on the urgency factor and the resource dependency nodes in the task execution profile, initiate a multi-dimensional resource query request to the resource scheduling pool to query the available quota and resource competition intensity index of each resource; Based on the available quota and resource competition intensity index of each resource, and combined with the resource quantity threshold required by the current task for each resource, construct a resource feasibility matrix; Determine the resource demand list and resource quota based on the resource feasibility matrix, and generate a resource scheduling scheme based on the resource demand list and resource quota.

[0122] The processor can access information and applications stored in memory via a transmission device to perform the following steps: if the resource feasibility matrix indicates that any resource does not meet the current task's requirements, mark the current task as a resource-constrained task; if the priority of the resource-constrained task is greater than the priority threshold, sort all tasks executed on the remote desktop, suspend the resource quota of preset tasks according to the sorting result, preempt resources for the current task, and obtain a preempted resource requirement list and resource quota; or, if the priority of the resource-constrained task is less than or equal to the priority threshold, adjust the resource requirement threshold for each resource for the current task, and obtain an adjusted resource requirement list and resource quota.

[0123] The processor can access information and applications stored in memory via a transmission device to perform the following steps: obtaining real-time network quality parameters for each network interface, wherein the real-time network quality parameters include at least one of the following: available bandwidth, network latency, packet loss rate, and billing mode for each network interface; matching the resource requirements in the task execution profile with the real-time parameters of each network interface, and calculating the comprehensive transmission performance score for each network interface; sorting each network interface based on the comprehensive transmission performance score to obtain the interface sorting result; and allocating the target resource quota in the resource scheduling scheme proportionally to multiple network interfaces according to the interface sorting result, wherein the target resource quota includes at least: bandwidth quota.

[0124] This invention provides a remote desktop resource scheduling scheme. By real-time sensing of task types and calculation of operation activity scores, combined with a pre-constructed knowledge graph, the priority of the current task is identified, and a task execution profile is generated. Based on the task execution profile, multi-dimensional resource quotas are obtained from the resource scheduling pool, achieving effective scheduling of multi-dimensional resources, ensuring the stable operation of the remote desktop, and improving the stability of the remote desktop connection. This solves the technical problem of low remote desktop connection stability in related technologies based on static and single remote desktop resource configuration strategies.

[0125] Those skilled in the art will understand that Figure 6 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones, tablets, handheld computers, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 6 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 6 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 6 The different configurations shown.

[0126] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0127] The invention will now be described in conjunction with another alternative embodiment.

[0128] Example 4

[0129] This invention also provides a computer-readable storage medium. Optionally, in this invention, the computer-readable storage medium can be used to store the program code executed by the remote desktop resource scheduling method provided in Embodiment 1.

[0130] Optionally, in this embodiment of the invention, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0131] This invention also provides a computer program product, which, when executed on a data processing device, is suitable for performing the steps of a remote desktop resource scheduling method: collecting screen frame data of the remote desktop and performing visual analysis on the screen frame data to obtain the task type and operation activity score of the current task on the remote desktop; based on the task type and operation activity score of the current task, calling a pre-built task graph to query the priority weight, resource dependency nodes, and expected duration of the current task, generating a task execution profile containing priority attributes; obtaining multi-dimensional resource quotas from a resource scheduling pool according to the task execution profile, generating a resource scheduling scheme, wherein the resource scheduling pool includes at least the following resources: network bandwidth, computing resources, and storage resources; and executing the resource scheduling scheme to allocate target resources to the remote desktop.

[0132] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0133] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0134] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0135] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0136] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0137] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0138] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A remote desktop resource scheduling method, characterized in that, include: Collect screen frame data of the remote desktop and perform visual analysis on the screen frame data to obtain the task type and operation activity score of the current task of the remote desktop. Based on the task type of the current task and the operation activity score, a pre-built task graph is invoked to query the priority weight, resource dependency nodes and expected duration of the current task, and a task execution profile containing priority attributes is generated. Based on the task execution profile, multi-dimensional resource quotas are obtained from the resource scheduling pool to generate a resource scheduling scheme. The resource scheduling pool includes at least the following resources: network bandwidth, computing resources, and storage resources. The resource scheduling scheme is executed to allocate target resources to the remote desktop.

2. The method according to claim 1, characterized in that, After executing the resource scheduling scheme and performing resource scheduling for the remote desktop, the method further includes: The task execution profile and the historical operation behavior data of the remote desktop are input into the task prediction model. The task transfer probability is calculated through the task prediction model, and the predicted task corresponding to the current task is output based on the task transfer probability. The task prediction model is a pre-built machine model for task prediction. Based on the prediction task, a resource reservation instruction is generated, wherein the resource reservation instruction is used to pre-schedule resources for the prediction task from the resource scheduling pool before the prediction task is executed.

3. The method according to claim 1, characterized in that, The step of performing visual analysis on the screen frame data to obtain the task type of the current task on the remote desktop includes: Image preprocessing is performed on each screen frame in the screen frame data to extract regional feature maps and filter out static background pixels to generate a dynamic operation region mask. The dynamic operation area is input into the classification model, and the classification model identifies the type of interface elements in the screen frame. The task type of the current task on the application desktop is determined based on the interface element type.

4. The method according to claim 1, characterized in that, The step of performing visual analysis on the screen frame data to obtain the operation activity score of the current task on the remote desktop includes: The incremental frame differential algorithm is used to extract the dynamic operation area of ​​each screen frame from the screen frame data; Calculate the pixel change rate of each of the aforementioned dynamic operation regions; For each of the aforementioned dynamic operation areas, a target tracking algorithm is used to identify the operation motion trajectory; The event interaction frequency is calculated based on the operation motion trajectory; The operation activity score is calculated based on the pixel change rate and the event interaction frequency.

5. The method according to claim 1, characterized in that, The steps of obtaining multi-dimensional resource quotas from the resource scheduling pool based on the task execution profile and generating a resource scheduling scheme include: The urgency factor of the current task is calculated based on the priority weight and expected duration in the task execution profile. Based on the urgency factor and the resource dependency nodes in the task execution profile, a multi-dimensional resource query request is initiated to the resource scheduling pool to query the available quota and resource competition heat index of each resource. Based on the available quotas and resource competition intensity index of each resource, and combined with the resource quantity threshold required by the current task for each resource, a resource feasibility matrix is ​​constructed. Based on the resource feasibility matrix, a resource demand list and resource quota are determined, and a resource scheduling scheme is generated based on the resource demand list and resource quota.

6. The method according to claim 5, characterized in that, The steps for determining the resource demand list and resource quotas based on the resource feasibility matrix include: If the resource feasibility matrix indicates that any resource does not meet the requirements of the current task, the current task is marked as a resource-constrained task. If the priority of the resource-constrained task is greater than a priority threshold, all tasks executed on the remote desktop are sorted, and the resource quotas of preset tasks are paused based on the sorting results. Resources are then preempted for the current task, resulting in the preempted resource requirement list and resource quota; or... If the priority of the resource-constrained task is less than or equal to the priority threshold, adjust the resource requirement threshold for each resource for the current task to obtain the adjusted resource requirement list and resource quota.

7. The method according to claim 1, characterized in that, The user device is configured with multiple network interfaces, and the data transmission of the remote desktop is performed in parallel through these multiple network interfaces. The step of allocating target resources to the remote desktop includes: Obtain real-time network quality parameters for each of the network interfaces, wherein the real-time network quality parameters include at least one of the following: available bandwidth, network latency, packet loss rate, and billing mode for each of the network interfaces; The resource requirements in the task execution profile are matched with the real-time parameters of each network interface, and the comprehensive transmission performance score of each network interface is calculated. Based on the comprehensive transmission performance score, each of the network interfaces is sorted to obtain an interface sorting result. According to the interface sorting result, the target resource quota in the resource scheduling scheme is proportionally allocated to multiple network interfaces, wherein the target resource quota includes at least: bandwidth quota.

8. A remote desktop resource scheduling device, characterized in that, include: The analysis unit is used to collect screen frame data of the remote desktop and perform visual analysis on the screen frame data to obtain the task type and operation activity score of the current task of the remote desktop. The generation unit is used to, based on the task type of the current task and the operation activity score, call the pre-built task graph, query the priority weight, resource dependency nodes and expected duration of the current task, and generate a task execution profile containing priority attributes. The scheduling unit is used to obtain multi-dimensional resource quotas from the resource scheduling pool based on the task execution profile and generate a resource scheduling scheme, wherein the resource scheduling pool includes at least the following resources: network bandwidth, computing resources and storage resources; The allocation unit is used to execute the resource scheduling scheme and allocate target resources to the remote desktop.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the remote desktop resource scheduling method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the remote desktop resource scheduling method according to any one of claims 1 to 7.

Images