User desktop interaction control device and method based on AI

Abstract

The invention relates to the technical field of communication, in particular to an AI-based user desktop interaction control device and method, and the method comprises the following steps: obtaining an original interaction data sequence, carrying out the abnormal value detection and timestamp sorting preprocessing of the data, and generating an ordered original interaction data sequence; time domain and frequency domain transformation is carried out on the original interaction data sequence, multi-scale frequency features are extracted to construct a feature vector space for obtaining feature data, and the feature data are used for representing time-frequency features of interaction signals; performing self-attention mechanism processing on the feature data, and performing weighted calculation through a multi-head attention weight matrix and a position coding parameter to obtain weighted feature data of context sensing; and inputting the weighted feature data into a long short-term memory network, and outputting time sequence associated interactive behavior prediction parameters in combination with parameters of a forgetting gate, an input gate and an output gate of a gating circulation unit.

Description

Technical Field

[0001] The present invention relates to the field of communication technologies, and in particular, to an AI-based user desktop interaction control device and method thereof. Background Art

[0002] With the development of computer technologies, the interaction mode between users and desktop devices has gradually evolved from traditional mouse and keyboard operations towards the direction of intelligence and naturalness.

[0003] Currently, the mainstream desktop interaction control technologies mainly rely on preset instruction sets or simple pattern matching, making it difficult to efficiently understand the complex operation intentions of users, and there are problems such as rigid interaction responses and insufficient adaptability. At the same time, traditional interaction methods have obvious limitations at the data processing level: on the one hand, the processing of user operation data (such as click coordinates, sliding trajectories, etc.) mostly stays in the original time-domain analysis, lacking the mining of the frequency-domain characteristics of signals, resulting in the inability to comprehensively represent the time-frequency characteristics of interaction behaviors; on the other hand, in the face of interaction data with strong temporality, traditional models are difficult to capture long-distance dependence relationships. For example, when identifying the deep intentions of users in a continuous operation sequence, misjudgments often occur due to insufficient extraction of temporal features.

[0004] In the intention recognition and decision-making control link, existing technologies usually adopt fixed rules or simple machine learning algorithms, and cannot dynamically optimize strategies according to user behaviors. For example, when user operation habits change or new interaction scenarios appear, traditional systems are difficult to adjust the response mode through autonomous learning, resulting in a decline in interaction efficiency. In addition, most solutions lack a closed-loop optimization mechanism based on user feedback, and are unable to continuously improve the interaction experience in actual use, making it difficult to meet the requirements of scenarios such as intelligent office and efficient human-machine collaboration. Summary of the Invention

[0005] Based on this, it is necessary to provide an AI-based user desktop interaction control device and method for the above technical problems, effectively solving problems such as shallow data processing, rigid intention understanding, and inability to autonomously learn strategies in traditional technologies, and improving the interaction efficiency and user experience in complex scenarios.

[0006] The present invention provides an AI-based user desktop interaction control method, and the method includes: Obtain the original interaction data sequence, perform preprocessing of outlier detection and timestamp sorting on the data, and generate an ordered original interaction data sequence; Perform time-domain and frequency-domain transformations on the original interaction data sequence, and extract multi-scale frequency features to construct a feature vector space for obtaining feature data, where the feature data is used to characterize the time-frequency characteristics of the interaction signal; Perform self-attention mechanism processing on the feature data, and perform weighted calculation through the multi-head attention weight matrix and the position encoding parameters to obtain context-aware weighted feature data; Input the weighted feature data into a long short-term memory network, and combine the forgetting gate, input gate, and output gate parameters of the gated recurrent unit to output the time-series related interaction behavior prediction parameters; Based on the preset intention classifier parameters, map the interaction behavior prediction parameters to a predefined intention space, calculate the intention probability distribution through the softmax function, and take the category corresponding to the maximum probability as the intention category data; Call the historical operation data to construct a decision-making process model, combine the intention category data and the current system state, and calculate the optimal control parameter set through a reinforcement learning algorithm; Execute the desktop interaction control action according to the optimal control parameter set, and perform policy gradient update according to the user feedback to continuously optimize the interaction response policy.

[0007] In one embodiment, the obtaining the original interaction data sequence, performing outlier detection and timestamp sorting preprocessing on the data, and generating an ordered original interaction data sequence includes: Collect the operation data of the user on the desktop device, including click coordinates, sliding trajectories, and pressing times, and construct a three-dimensional spatio-temporal data matrix; Perform outlier detection on the collected original data, and eliminate the outlier data with a deviation exceeding the preset threshold by calculating the deviation between the data points and the local mean; Sort the processed data according to the timestamp to generate an ordered original interaction data sequence.

[0008] In one embodiment, the performing time-domain and frequency-domain transformation on the original interaction data sequence includes: Perform segmentation processing on the original interaction data sequence, and calculate the mean, variance, and standard deviation within the time window of each segment of data; Apply a difference operator to calculate the change rate between adjacent data points to generate a first-order difference sequence; Construct a sliding time window, calculate the cumulative sum and cumulative difference of the data within the window, and extract time-domain features.

[0009] In one embodiment, the extracting multi-scale frequency features to construct a feature vector space includes: Calculate the frequency features at different scales, including peak frequency, center frequency, and bandwidth; Construct a feature correlation matrix to analyze the correlation between different frequency features; Apply the principal component analysis algorithm to project the high-dimensional feature vectors into a low-dimensional space to generate a feature vector space.

[0010] In one embodiment, the self-attention mechanism is applied to the feature data, and weighted calculation is performed through the multi-head attention weight matrix and the position encoding parameters to obtain context-aware weighted feature data, including: Construct a feature similarity matrix and calculate the cosine similarity between feature vectors; Apply the weight normalization algorithm to convert the similarity matrix into an attention weight matrix; Perform weighted summation on the feature data according to the attention weight matrix to generate context-aware weighted feature data.

[0011] In one embodiment, the weighted feature data is input into a long short-term memory network, and combined with the forgetting gate, input gate, and output gate parameters of the gated recurrent unit, and the time-series related interaction behavior prediction parameters are output, including: Normalize the weighted feature data and map the data to the interval [0, 1]; Construct a time series prediction model and calculate the hidden state at the current time step; Apply the forgetting gate, input gate, and output gate mechanisms to control the flow and update of information, and generate time-series related interaction behavior prediction parameters.

[0012] In one embodiment, based on the preset intention classifier parameters, the interaction behavior prediction parameters are mapped to a predefined intention space, including: Load a pre-trained intention classifier model, and the classifier includes multi-layer fully connected neural network and activation function parameters; Perform dimension alignment processing on the interaction behavior prediction parameters to match them with the dimension of the input layer of the classifier; Through the forward propagation algorithm, input the processed prediction parameters into the classifier, and calculate the intention space mapping vector using the preset weight matrix and bias parameters.

[0013] In one embodiment, the intention probability distribution is calculated through the softmax function, and the category corresponding to the maximum probability is taken as the intention category data, including: Input the intention space mapping vector into the softmax function to calculate the exponential value of each predefined intention category; Normalize all exponential values to generate a probability distribution vector in the range of [0, 1]; Traverse the probability distribution vector, extract the index position with the maximum probability value, and match the predefined intention category mapping table to obtain the intention category data.

[0014] In one embodiment, the historical operation data is called to construct a decision-making process model, and combined with the intention category data and the current system state, the optimal control parameter set is calculated through the reinforcement learning algorithm, including: Extract the operation sequence matching the current intention from the historical operation database to construct a state-action pair training set; Define the state space of the reinforcement learning environment, and encode the current system state parameters and intention category data into an environmental state vector; Iteratively calculate the optimal action value function based on the state space and the training set to generate a set of control parameters.

[0015] The present invention also provides an AI-based user desktop interaction control device, which is applied to the AI-based user desktop interaction control method described in any one of the above embodiments, and includes: A first data processing module, configured to obtain an original interaction data sequence, perform outlier detection and timestamp sorting preprocessing on the data, and generate an ordered original interaction data sequence; A second data processing module, configured to perform time-domain and frequency-domain transformations on the original interaction data sequence, and extract multi-scale frequency features to construct a feature vector space for obtaining feature data, where the feature data is used to characterize the time-frequency characteristics of the interaction signal; A third data processing module, configured to perform self-attention mechanism processing on the feature data, and perform weighted calculation through a multi-head attention weight matrix and position encoding parameters to obtain context-aware weighted feature data; An input-output module, configured to input the weighted feature data into a long short-term memory network, and combine the forgetting gate, input gate, and output gate parameters of the gated recurrent unit to output time-series related interaction behavior prediction parameters; A mapping processing module, configured to map the interaction behavior prediction parameters to a predefined intention space based on preset intention classifier parameters, calculate an intention probability distribution through a softmax function, and take the category corresponding to the maximum probability as the intention category data; A calculation decision module, configured to call historical operation data to construct a decision process model, and calculate an optimal control parameter set through a reinforcement learning algorithm in combination with the intention category data and the current system state; An optimization module, configured to execute desktop interaction control actions according to the optimal control parameter set, and perform policy gradient update according to user feedback to continuously optimize the interaction response policy.

[0016] The above AI-based user desktop interaction control device and method preprocess the original interaction data sequence by performing outlier detection and timestamp sorting to generate ordered data, ensuring the reliability of interaction signals and eliminating interference such as device noise. Then, the original data is transformed in the time domain and frequency domain, and multi-scale frequency features are extracted to construct a feature vector space, strengthening the characterization of the time-frequency characteristics of interaction behaviors and capturing dynamic features and periodic patterns. Next, the feature data is processed by a self-attention mechanism, and the long short-term memory network is combined to output prediction parameters related to time series, enhancing the modeling of long-distance dependencies in interaction data, focusing on key operations, and memorizing historical patterns. Furthermore, the prediction parameters are mapped to the intention space based on an intention classifier and the softmax function to achieve accurate intention recognition. Then, the historical operation data is called, combined with the current intention and system state, and the optimal control parameter set is calculated through a reinforcement learning algorithm. Finally, control actions are executed according to the optimal parameters, and the policy gradient is updated based on user feedback to form a closed-loop optimization mechanism, effectively solving problems such as shallow data processing, rigid intention understanding, and inability to autonomously learn strategies in traditional technologies, and improving the interaction efficiency and user experience in complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0018] Figure 1 It is a flowchart of the AI-based user desktop interaction control method provided by the present invention; Figure 2 It is a module diagram of the AI-based user desktop interaction control device provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0020] As Figure 1 shown, an AI-based user desktop interaction control method includes the following steps: Obtain the original interaction data sequence, perform outlier detection and timestamp sorting preprocessing on the data, and generate an ordered original interaction data sequence; Perform time-domain and frequency-domain transformations on the original interaction data sequence, extract multi-scale frequency features to construct a feature vector space for obtaining feature data, where the feature data is used to characterize the time-frequency characteristics of the interaction signal; Perform self-attention mechanism processing on the feature data, and perform weighted calculation through the multi-head attention weight matrix and position encoding parameters to obtain context-aware weighted feature data; Input the weighted feature data into a long short-term memory network, combine the forgetting gate, input gate, and output gate parameters of the gated recurrent unit, and output the interaction behavior prediction parameters related to time series; Based on the preset intention classifier parameters, map the interaction behavior prediction parameters to a predefined intention space, calculate the intention probability distribution through the softmax function, and take the category corresponding to the maximum probability as the intention category data; Call historical operation data to construct a decision-making process model, combine the intention category data and the current system state, and calculate the optimal control parameter set through a reinforcement learning algorithm; Execute desktop interaction control actions according to the optimal control parameter set, and perform policy gradient updates based on user feedback to continuously optimize the interaction response strategy.

[0021] The above AI-based user desktop interaction control method preprocesses by obtaining the original interaction data sequence and performing outlier detection and timestamp sorting to generate ordered data to ensure the reliability of the interaction signal and eliminate interference such as device noise; then performs time-domain and frequency-domain transformations on the original data and extracts multi-scale frequency features to construct a feature vector space to strengthen the characterization of the time-frequency characteristics of the interaction behavior, capture dynamic features and periodic patterns; then perform self-attention mechanism processing on the feature data, combine the long short-term memory network to output time-series related prediction parameters, enhance the modeling of the long-distance dependence relationship of the interaction data, focus on key operations and remember historical patterns; furthermore, map the prediction parameters to the intention space based on the intention classifier and the softmax function to achieve accurate intention recognition; then call historical operation data, combine the current intention and system state, and calculate the optimal control parameter set through a reinforcement learning algorithm; finally, execute control actions according to the optimal parameters and perform policy gradient updates based on user feedback to form a closed-loop optimization mechanism, effectively solving problems such as shallow data processing, rigid intention understanding, and inability to autonomously learn strategies in traditional technologies, and improving the interaction efficiency and user experience in complex scenarios.

[0022] In one embodiment, obtaining the original interaction data sequence and performing preprocessing of outlier detection and timestamp sorting on the data to generate an ordered original interaction data sequence includes the following steps: Collect the operation data of the user on the desktop device, including click coordinates, sliding trajectories, and pressing times, and construct a three-dimensional spatio-temporal data matrix.

[0023] Specifically, the acquisition device includes a mouse, a touch screen, etc., and the sampling frequency is set to 100 Hz (i.e., 100 data points per second).

[0024] Construct a three-dimensional spatio-temporal data matrix , where is the number of samples, and the three dimensions are the click coordinates and the pressing time , is the length of the time series.

[0025] If the user clicks 3 times continuously, the collected data can be expressed as , corresponding to the coordinates and pressing time at 3 time points.

[0026] Perform outlier detection on the collected raw data. By calculating the deviation between the data point and the local mean, the outlier data with a deviation exceeding the preset threshold is removed.

[0027] Specifically, for the data point , calculate its deviation from the mean of the local neighborhood points .

[0028] , where Among them, is the th data point (such as click coordinates, pressing time, etc.), is the mean of the local neighborhood of the data point points, is the number of points in the local neighborhood (such as the number of nearest neighbor points), is the nearest neighbor points set of the data point , is the deviation between the data point and the local mean.

[0029] In this embodiment, the preset threshold , where, is the standard deviation of the local neighborhood data. When , remove this point.

[0030] If a click coordinate , its local neighborhood mean is , and the standard deviation , then , far exceeding , it is determined as an outlier.

[0031] Sort the processed data according to the timestamp to generate an ordered sequence of original interaction data.

[0032] Specifically, sort in ascending order according to the data acquisition timestamp to generate an ordered sequence , where .

[0033] In one embodiment, perform time-domain and frequency-domain transformations on the original interaction data sequence, including the following steps: Perform segmentation processing on the original interaction data sequence, and calculate the mean, variance, and standard deviation within the time window of each segment of data.

[0034] Specifically, set the time window , with an overlap rate of 50%, divide the sequence into M segments, and calculate for each segment: Mean: , where is the th data point within the time window, is the number of data points within the window, is the mean of the data within the window.

[0035] Variance: Standard deviation: , where is the standard deviation Apply a difference operator to calculate the change rate between adjacent data points to generate a first-order difference sequence.

[0036] Specifically, the first-order difference , where is the change rate between adjacent data points.

[0037] Construct a sliding time window, calculate the cumulative sum and cumulative difference of the data within the window, and extract time-domain features.

[0038] Specifically, calculate the cumulative sum and the cumulative difference In one embodiment, extract multi-scale frequency features to construct a feature vector space, including the following steps: Calculate frequency features at different scales, including peak frequency, center frequency, and bandwidth.

[0039] Specifically, define short-scale (100 ms), medium-scale (500 ms), and long-scale (1 s) windows, and extract the , , at each scale respectively to form a feature vector: , with a dimension of 3 3 = 9 dimensions.

[0040] Perform a fast Fourier transform on the time-domain sequence to obtain the frequency-domain representation .

[0041] Power spectral density , where is the frequency variable, is the time variable, is the length of the time-domain sequence, i.e., the total number of data points in the time series, is the signal value at the -th time point in the time-domain sequence, which reflects the specific characteristics of the original interaction data at time , is the imaginary unit.

[0042] Peak frequency: Center frequency: Bandwidth: , where is the frequency at which the power drops to half of the peak.

[0043] Construct a feature correlation matrix to analyze the correlation between different frequency features.

[0044] Specifically, construct the correlation matrix , where to analyze the linear correlation between features, such as the correlation between the peak frequency and the center frequency.

[0045] Among them, is the correlation coefficient between feature and feature , and are different frequency features, is the covariance between feature and feature , is the standard deviation of feature and feature .

[0046] Apply the principal component analysis algorithm to project the high-dimensional feature vector into a low-dimensional space to generate a feature vector space.

[0047] Specifically, perform principal component analysis on the feature vector, retain 95% of the variance, and the dimensionality reduction formula is: , where is the first principal component matrices, obtained by eigenvalue decomposition , satisfies If the original feature is 100-dimensional, it may be reduced to 20 dimensions after dimensionality reduction.

[0048] In one embodiment, the feature data is processed by a self-attention mechanism, and weighted calculation is performed through a multi-head attention weight matrix and position encoding parameters to obtain context-aware weighted feature data, including the following steps: Construct a feature similarity matrix and calculate the cosine similarity between feature vectors.

[0049] Specifically, construct a query matrix , key matrix , value matrix , all with dimensions , where is the feature dimension, is the sequence length.

[0050] Cosine similarity matrix: Where , is the th row of matrix , and is the th row vector of matrix , and and is the cosine similarity between the

[0051] Apply the weight normalization algorithm to convert the similarity matrix into an attention weight matrix.

[0052] Specifically, apply the softmax function: , to obtain the attention weight matrix , where .

[0053] Where is the attention weight of the rd feature to the th feature.

[0054] Weighted sum the feature data according to the attention weight matrix to generate context-aware weighted feature data.

[0055] Specifically, context-aware feature: , integrating the feature dependency relationships at each position in the sequence.

[0056] Where is the weighted feature after integrating the sequence dependency relationships.

[0057] Set 8 heads, each head calculates independently, and finally splices them into In one embodiment, the weighted feature data is input into a long short-term memory network, and the forgetting gate, input gate, and output gate parameters of the gated recurrent unit are combined to output the interaction behavior prediction parameters related to time series, including the following steps: Normalize the weighted feature data and map the data to the interval [0, 1].

[0058] Specifically, the min-max normalization method is adopted to map the weighted feature data to the interval [0, 1]. The specific operation is: calculate the minimum value min_val and the maximum value max_val of the feature data, and perform (x - min_val) / (max_val - min_val) on each data point x. For example, if a certain feature value range is [-10, 20], it will be converted to [0, 1] after normalization, ensuring the unity of the data scale and avoiding the instability of the LSTM network training caused by too large differences in input values.

[0059] Where x is the original feature value, and min_val and max_val are the minimum and maximum values of the feature respectively.

[0060] Build a time series prediction model and calculate the hidden state at the current time step.

[0061] Specifically, initialize the dimension of the hidden layer of the LSTM network (such as 128 dimensions), and set the sequence length (such as the interaction data of the last 50 time steps).

[0062] For the current time step t, use the normalized weighted feature data as the input, combine the hidden state h(t - 1) and the cell state c(t - 1) of the previous time step, and calculate the current hidden state h(t) through the LSTM unit.

[0063] The calculation process of the hidden state will capture the time series dependence relationship of the interaction data, such as the action sequence pattern of continuous clicks and drags by the user.

[0064] Apply the forgetting gate, input gate, and output gate mechanisms to control the flow and update of information, and generate the interaction behavior prediction parameters related to time series.

[0065] Specifically, the forgetting gate: generate a weight between 0 and 1 according to the current input and the previous hidden state, and determine which historical information to discard. For example, if the user has not performed keyboard input for a long time, the forgetting gate will reduce the weight of the early keyboard operation data.

[0066] The input gate: screen the valid information in the current input, combine it with the candidate cell state, and update the cell state c(t). For example, when a rapid mouse slide is detected, the input gate will strengthen the feature weight of this action.

[0067] Output Gate: Based on the updated cell state, generate the hidden state h(t) at the current time step, which is the interactive behavior prediction parameter. This parameter contains the sequential correlation prediction of the user's next operation, such as predicting the coordinate range of a mouse click or the type of keyboard input.

[0068] Among them, h(t) is the hidden state (interactive behavior prediction parameter) at the current time step, and c(t) is the cell state at the current time step.

[0069] In one embodiment, based on the preset intention classifier parameters, map the interactive behavior prediction parameter to the predefined intention space, including the following steps: Load the pre-trained intention classifier model, and the classifier includes a multi-layer fully connected neural network and activation function parameters.

[0070] Specifically, the pre-trained model adopts a three-layer fully connected neural network structure: the input layer dimension is the same as the dimension of the prediction parameter output by the LSTM (such as 128 dimensions), the middle layer contains 256 neurons and uses the ReLU activation function, and the output layer dimension matches the number of predefined intention categories (such as 10 types of intentions).

[0071] The model is trained through historical interaction data, and the training data includes the user operation sequence and its corresponding true intention label (such as "copy text", "switch window", "save file", etc.).

[0072] Perform dimension alignment processing on the interactive behavior prediction parameter to make it match the classifier input layer dimension.

[0073] Specifically, if the dimension of the prediction parameter output by the LSTM (such as 128 dimensions) does not match the dimension of the classifier input layer (such as 256 dimensions), it is processed in the following way: If the output dimension is lower than the input dimension, pad zeros after the feature vector or increase the dimension through a linear transformation of the fully connected layer.

[0074] If the output dimension is higher than the input dimension, use principal component analysis (PCA) for dimensionality reduction or directly intercept the key feature dimensions to ensure consistency with the input layer dimension.

[0075] Through the forward propagation algorithm, input the processed prediction parameter into the classifier, and calculate the intention space mapping vector using the preset weight matrix and bias parameters.

[0076] Specifically, input the dimension-aligned prediction parameter into the classifier: The first fully connected layer calculates W1×input + b1 and outputs non-linear features through the ReLU activation function.

[0077] The second fully connected layer calculates W2×hidden1 + b2 in the same way and passes through ReLU again.

[0078] The output layer calculates W3×hidden2 + b3 to generate a mapping vector with the same dimension as the number of intent categories (e.g., 10 - dimensional), and each element corresponds to the raw score (logits) of an intent category.

[0079] Among them, W is the weight matrix, b is the bias vector, and input is the input feature (such as the prediction parameter output by the LSTM).

[0080] In one embodiment, the intent probability distribution is calculated through the softmax function, and the category corresponding to the maximum probability is taken as the intent category data, including the following steps: Input the mapping vector of the intent space into the softmax function to calculate the exponential value of each predefined intent category.

[0081] Specifically, input the mapping vector of the intent space (such as [2.1, -0.5, 3.2,...]) into the softmax function, and calculate the exponential value exp(logit) for each element. For example, calculate exp(3.2)≈24.53 for the value 3.2.

[0082] Sum all the exponential values, and then divide each exponential value by the sum to obtain a probability distribution vector ranging from [0, 1]. For example, if the total sum of the exponents is 50 and the exponential value of a certain element is 24.53, then the corresponding probability is 24.53 / 50≈0.49.

[0083] Normalize all the exponential values to generate a probability distribution vector ranging from [0, 1].

[0084] Specifically, the predefined intent category mapping table adopts a dictionary structure, such as {0: "Copy text", 1: "Open file", 2: "Save document",...}.

[0085] Traverse the probability distribution vector, extract the index position with the maximum probability value, and match the predefined intent category mapping table to obtain the intent category data.

[0086] Specifically, traverse the probability distribution vector, find the index of the element with the maximum probability value (such as index 2 corresponding to the probability 0.49), and match through the mapping table to obtain the intent category data (such as "Save document"). This process realizes the conversion from numerical features to semantic intents, providing a decision - making basis for subsequent desktop control actions.

[0087] In one embodiment, call the historical operation data to construct a decision - making process model, combine the intent category data and the current system state, and calculate the optimal control parameter set through the reinforcement learning algorithm, including the following steps: Extract the operation sequence that matches the current intention from the historical operation database, and construct the state-action pair training set.

[0088] Specifically, extract the operation sequence that matches the current intention from the historical database. For example, when the intention is "copy text", extract actions such as mouse dragging and keyboard Ctrl+C under this intention in the history to form state-action pairs. , where the state includes: System state: the current window position , the mouse position , and the keyboard state (key press situation).

[0089] Intention encoding: one-hot vector of the intention category (e.g., the corresponding position is 1 in a 10-dimensional vector).

[0090] Action : such as the mouse click coordinates , key input, etc., parameterized as a vector (click type: left button / right button).

[0091] Define the state space of the reinforcement learning environment, and encode the current system state parameters and intention category data into the environmental state vector.

[0092] Specifically, the state space : Concatenate the system state and the intention encoding into a 100-dimensional vector Action space A: a discrete action set, such as {left click, right click, drag, keyboard input}, and each action corresponds to a set of control parameters.

[0093] Iteratively calculate the optimal action value function based on the state space and the training set to generate a set of control parameters.

[0094] Specifically, iteratively update where is the value function for executing action in state , is the current state (system state + intention encoding), is the current action, is the next state, is the next action, (learning rate), (discount factor), is the immediate reward after executing action (e.g., +1 for correct execution of the intention, -1 for incorrect).

[0095] Generate a set of control parameters: through a greedy strategy Select the optimal action, and its parameters such as click coordinates and drag distance constitute the set of control parameters.

[0096] Such as Figure 2 As shown, an AI-based user desktop interaction control device, which is applied to the AI-based user desktop interaction control method of any of the above embodiments, includes a first data processing module, a second data processing module, a third data processing module, an input / output module, a mapping processing module, a calculation and decision-making module, and an optimization module: The first data processing module is used to obtain the original interaction data sequence, perform outlier detection and timestamp sorting preprocessing on the data, and generate an ordered original interaction data sequence; The second data processing module is used to perform time-domain and frequency-domain transformations on the original interaction data sequence, extract multi-scale frequency features to construct a feature vector space, and obtain feature data, which is used to characterize the time-frequency characteristics of the interaction signal; The third data processing module is used to perform self-attention mechanism processing on the feature data, perform weighted calculations through the multi-head attention weight matrix and position encoding parameters, and obtain context-aware weighted feature data; The input / output module is used to input the weighted feature data into a long short-term memory network, combine the forgetting gate, input gate, and output gate parameters of the gated recurrent unit, and output the interaction behavior prediction parameters related to time series; The mapping processing module is used to map the interaction behavior prediction parameters to a predefined intention space based on the preset intention classifier parameters, calculate the intention probability distribution through the softmax function, and take the category corresponding to the maximum probability as the intention category data; The calculation and decision-making module is used to call the historical operation data to construct a decision-making process model, combine the intention category data and the current system state, and calculate the optimal control parameter set through a reinforcement learning algorithm; The optimization module is used to execute the desktop interaction control action according to the optimal control parameter set, and perform policy gradient update according to the user feedback to continuously optimize the interaction response policy.

[0097] The above-mentioned AI-based user desktop interaction control device preprocesses by obtaining the original interaction data sequence and performing outlier detection and timestamp sorting to generate ordered data to ensure the reliability of interaction signals and eliminate interference such as device noise. Then, the original data is transformed in the time domain and frequency domain, and multi-scale frequency features are extracted to construct a feature vector space, strengthening the characterization of the time-frequency characteristics of interaction behaviors and capturing dynamic features and periodic patterns. Then, the feature data is processed by a self-attention mechanism, and the prediction parameters related to time series are output in combination with a long short-term memory network, enhancing the modeling of the long-distance dependence relationship of interaction data, focusing on key operations and memorizing historical patterns. Furthermore, the prediction parameters are mapped to the intention space based on an intention classifier and the softmax function to achieve accurate intention recognition. Then, the historical operation data is called, combined with the current intention and system state, and the optimal control parameter set is calculated through a reinforcement learning algorithm. Finally, control actions are executed according to the optimal parameters, and policy gradient updates are performed based on user feedback to form a closed-loop optimization mechanism, effectively solving problems such as shallow data processing, rigid intention understanding, and inability to autonomously learn strategies in traditional technologies, and improving the interaction efficiency and user experience in complex scenarios.

[0098] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0099] The above embodiments only represent several implementation manners of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as a limitation to the scope of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the appended claims.

Claims

1. An AI-based user desktop interaction control method, characterized in that, The method includes: Obtaining an original interaction data sequence, performing outlier detection and timestamp sorting preprocessing on the data to generate an ordered original interaction data sequence; Performing time-domain and frequency-domain transformations on the original interaction data sequence, and extracting multi-scale frequency features to construct a feature vector space for obtaining feature data, where the feature data is used to characterize the time-frequency characteristics of the interaction signal; Performing self-attention mechanism processing on the feature data, and performing weighted calculation through a multi-head attention weight matrix and position encoding parameters to obtain context-aware weighted feature data; Inputting the weighted feature data into a long short-term memory network, and combining the forgetting gate, input gate, and output gate parameters of the gated recurrent unit to output time-series related interaction behavior prediction parameters; Based on preset intention classifier parameters, mapping the interaction behavior prediction parameters to a predefined intention space, calculating an intention probability distribution through a softmax function, and taking the category corresponding to the maximum probability as the intention category data; Invoking historical operation data to construct a decision-making process model, combining the intention category data and the current system state, and calculating an optimal control parameter set through a reinforcement learning algorithm; Performing desktop interaction control actions according to the optimal control parameter set, and performing policy gradient update according to user feedback to continuously optimize the interaction response policy.

2. The AI-based user desktop interaction control method according to claim 1, wherein, The obtaining of the original interaction data sequence, performing outlier detection and timestamp sorting preprocessing on the data to generate an ordered original interaction data sequence includes: Collecting operation data of the user on the desktop device, including click coordinates, sliding trajectories, and pressing times, to construct a three-dimensional spatio-temporal data matrix; Performing outlier detection on the collected original data, and removing abnormal data with a deviation exceeding a preset threshold by calculating the deviation between the data points and the local mean; Sorting the processed data according to timestamps to generate an ordered original interaction data sequence.

3. The AI-based user desktop interaction control method according to claim 2, wherein, The performing of time-domain and frequency-domain transformations on the original interaction data sequence includes: Performing segmented processing on the original interaction data sequence, and calculating the mean, variance, and standard deviation within the time window of each segment of data; Applying a difference operator to calculate the change rate between adjacent data points to generate a first-order difference sequence; Constructing a sliding time window, calculating the cumulative sum and cumulative difference of the data within the window, and extracting time-domain features.

4. The AI-based user desktop interaction control method according to claim 3, wherein The extracting of multi-scale frequency features to construct a feature vector space includes: Calculating frequency features at different scales, including peak frequency, center frequency, and bandwidth; Constructing a feature correlation matrix to analyze the correlation between different frequency features; Applying a principal component analysis algorithm to project the high-dimensional feature vectors into a low-dimensional space to generate a feature vector space.

5. The AI-based user desktop interaction control method according to claim 4, wherein The performing of self-attention mechanism processing on the feature data, and performing weighted calculation through a multi-head attention weight matrix and position encoding parameters to obtain context-aware weighted feature data includes: Constructing a feature similarity matrix and calculating the cosine similarity between feature vectors; Applying a weight normalization algorithm to convert the similarity matrix into an attention weight matrix; Performing weighted summation on the feature data according to the attention weight matrix to generate context-aware weighted feature data.

6. The AI-based user desktop interaction control method according to claim 5, wherein Inputting the weighted feature data into a long short-term memory network, and combining the parameters of the forget gate, input gate, and output gate of the gated recurrent unit to output the interaction behavior prediction parameters related to time series, including: Performing normalization processing on the weighted feature data to map the data to the interval [0, 1]; Constructing a time series prediction model to calculate the hidden state at the current time step; Applying the forget gate, input gate, and output gate mechanisms to control the flow and update of information, and generating the interaction behavior prediction parameters related to time series.

7. The AI-based user desktop interaction control method according to claim 6, wherein Based on the preset intention classifier parameters, mapping the interaction behavior prediction parameters to a predefined intention space, including: Loading a pre-trained intention classifier model, where the classifier includes a multi-layer fully connected neural network and activation function parameters; Performing dimension alignment processing on the interaction behavior prediction parameters to make them match the dimension of the input layer of the classifier; Through the forward propagation algorithm, inputting the processed prediction parameters into the classifier, and calculating the intention space mapping vector using the preset weight matrix and bias parameters.

8. The AI-based user desktop interaction control method according to claim 7, wherein Calculating the intention probability distribution through the softmax function, and taking the category corresponding to the maximum probability as the intention category data, including: Inputting the intention space mapping vector into the softmax function to calculate the exponential value of each predefined intention category; Performing normalization processing on all exponential values to generate a probability distribution vector ranging from [0, 1]; Traversing the probability distribution vector, extracting the index position with the maximum probability value, and matching the predefined intention category mapping table to obtain the intention category data.

9. The AI-based user desktop interaction control method according to claim 8, wherein, Invoking the historical operation data to construct a decision-making process model, combining the intention category data and the current system state, and calculating the optimal control parameter set through the reinforcement learning algorithm, including: Extracting the operation sequence matching the current intention from the historical operation database to construct a state-action pair training set; Defining the state space of the reinforcement learning environment, and encoding the current system state parameters and intention category data into an environment state vector; Iteratively calculating the optimal action value function based on the state space and the training set to generate a control parameter set.

10. An AI-based user desktop interaction control device, which is applied to the AI-based user desktop interaction control method according to any one of claims 1 to 9 above, and is characterized in that, Including: The first data processing module is used to obtain the original interaction data sequence, perform outlier detection and timestamp sorting preprocessing on the data, and generate an ordered original interaction data sequence; The second data processing module is used to perform time domain and frequency domain transformations on the original interaction data sequence, and extract multi-scale frequency features to construct a feature vector space for obtaining feature data, where the feature data is used to characterize the time-frequency characteristics of the interaction signal; The third data processing module is used to perform self-attention mechanism processing on the feature data, and perform weighted calculation through the multi-head attention weight matrix and position encoding parameters to obtain context-aware weighted feature data; The input-output module is used to input the weighted feature data into a long short-term memory network, and combine the parameters of the forget gate, input gate, and output gate of the gated recurrent unit to output the interaction behavior prediction parameters related to time series; The mapping processing module is used to map the interaction behavior prediction parameters to a predefined intention space based on preset intention classifier parameters, calculate the intention probability distribution through the softmax function, and take the category corresponding to the maximum probability as the intention category data; The calculation and decision-making module is used to call historical operation data to construct a decision-making process model, combine the intention category data and the current system state, and calculate the optimal control parameter set through a reinforcement learning algorithm; The optimization module is used to execute desktop interaction control actions according to the optimal control parameter set, and perform policy gradient updates based on user feedback to continuously optimize the interaction response policy.

Images