Human-computer interaction method and device, computer equipment and storage medium

Abstract

The invention discloses a man-machine interaction method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining interaction data of the man-machine interaction device and a user operation portrait corresponding to the interaction data, and the user operation portrait is determined according to historical interaction data of the man-machine interaction device; generating a candidate interaction instruction based on the interaction data; generating a mapping rule set based on the user operation portrait; converting the candidate interaction instruction into a target interaction instruction based on the mapping rule set; and based on the target interaction instruction, controlling target equipment corresponding to the man-machine interaction device, so that the target equipment executes the corresponding operation. The user operation portrait is constructed based on the historical interaction data, the mapping rule set is generated according to the user operation portrait, and the candidate interaction instruction can be automatically converted into the target interaction instruction more conforming to the operation habit of the target user, so that the response of the target equipment is more accurate and natural.

Description

Technical Field

[0001] This invention relates to the field of equipment control, and more particularly to a human-computer interaction method, device, computer equipment, and storage medium. Background Technology

[0002] With the widespread adoption of portable computing devices, touch mice, as pointer control devices based on touch panels, have been widely used in laptops, tablets, and some desktop computers. Users can move the cursor, scroll the page, and zoom by sliding, clicking, and dragging with one or more fingers. Compared to traditional mechanical mice, touch mice have advantages such as small size, no need for external cables, and support for multiple gestures.

[0003] However, in actual use, different users have differences in finger size, touch pressure, swipe speed, and operating habits. Traditional fixed-parameter recognition algorithms are prone to causing action recognition deviations or delays. For users with insufficient hand dexterity or special touch habits, existing products lack personalized recognition optimization and error correction functions.

[0004] Therefore, there is an urgent need for an interaction optimization solution for touch mice that can improve interaction accuracy, thereby increasing the breadth of the target audience and the interactive experience, so that users with different hand shapes, operating habits and touch characteristics can obtain stable and accurate usage results. Summary of the Invention

[0005] Therefore, it is necessary to provide a human-computer interaction method, device, computer equipment, and storage medium to address the aforementioned technical problems, in order to solve the problem that traditional human-computer interaction methods cannot accurately adapt to users with different hand shapes, operating habits, and touch characteristics.

[0006] A human-computer interaction method, applied to a human-computer interaction device, comprising: The interaction data of the human-computer interaction device and the user operation profile corresponding to the interaction data are obtained, and the user operation profile is determined based on the historical interaction data of the human-computer interaction device. Based on the interaction data, candidate interaction instructions are generated; Based on the user operation profile, a mapping rule set is generated; Based on the mapping rule set, the candidate interaction instructions are converted into target interaction instructions; Based on the target interaction command, the target device corresponding to the human-computer interaction device is controlled so that the target device performs the corresponding operation.

[0007] Optionally, the human-computer interaction device includes a touch array and a pressure array, the interaction data includes touch data corresponding to the touch array and pressure data corresponding to the pressure array, and the step of generating candidate interaction commands based on the interaction data includes: Based on the touch data and the pressure data, feature extraction processing is performed to obtain the corresponding temporal and spatial features; The temporal features and spatial features are fused to obtain fused features; The fused features are provided to a preset temporal reasoning model to obtain the candidate interaction instructions.

[0008] Optionally, generating a mapping rule set based on the user operation profile includes: Based on the distribution of historical interaction data corresponding to different operation modes in the user operation profile, the correspondence between candidate interaction commands and target interaction commands is determined. A mapping rule set is constructed based on the correspondence, and the mapping rules from the candidate interaction instruction to the target interaction instruction are stored in the mapping rule set.

[0009] Optionally, controlling the target device corresponding to the human-computer interaction device based on the target interaction command includes: Obtain the current operating status parameters and historical trajectory data of the target device; Based on the current operating status parameters and the historical trajectory data, the target interaction command is subjected to sliding compensation to obtain the compensated target interaction command. Based on the compensated target interaction command, the target device corresponding to the human-computer interaction device is controlled.

[0010] Optionally, the step of performing slip compensation on the target interaction command based on the current running state parameters and the historical trajectory data to obtain the compensated target interaction command includes: Based on the current operating status parameters and the historical trajectory data, predict the operating status of the target device within a preset prediction period; The predicted running state is compared with the target interaction command to generate a correction amount; The target interaction command is adjusted according to the correction amount to obtain the compensated target interaction command.

[0011] Optionally, controlling the target device corresponding to the human-computer interaction device based on the compensated target interaction command includes: Determine if there is an interactive instruction to be executed; If a conflict exists, a conflict detection is performed to determine whether there is a conflict between the compensated target interaction instruction and the currently executed interaction instruction. If a conflict exists, the conflict type is determined, and the compensated target interaction instruction and the currently executed interaction instruction are prioritized according to the conflict type to generate an interaction instruction sorting table. Based on the interaction instruction sorting table, the corresponding interaction instructions are executed in sequence.

[0012] Optionally, the conflict detection includes determining whether the objects targeted by the instructions are the same, whether the execution time intervals overlap, and whether the execution effects affect each other.

[0013] A human-computer interaction device, comprising: The first acquisition module is used to acquire the interaction data of the human-computer interaction device and the user operation profile corresponding to the interaction data. The user operation profile is determined based on the historical interaction data of the human-computer interaction device. The first generation module is used to generate candidate interaction instructions based on the interaction data; The second generation module is used to generate a mapping rule set based on the user operation profile; The first conversion module is used to convert the candidate interaction instruction into a target interaction instruction based on the mapping rule set; The first control module is used to control the target device corresponding to the human-computer interaction device based on the target interaction command, so that the target device performs the corresponding operation.

[0014] A computer device includes a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein the processor implements the above-described human-computer interaction method when executing the computer-readable instructions.

[0015] A readable storage medium having computer-readable instructions stored thereon, which, when executed by a processor, implement the human-computer interaction method.

[0016] The aforementioned human-computer interaction method, apparatus, computer device, and storage medium acquire interaction data from the human-computer interaction device and a user operation profile corresponding to the interaction data, wherein the user operation profile is determined based on historical interaction data of the human-computer interaction device; generate candidate interaction commands based on the interaction data; generate a mapping rule set based on the user operation profile; convert the candidate interaction commands into target interaction commands based on the mapping rule set; and control a target device corresponding to the human-computer interaction device based on the target interaction command, so that the target device performs the corresponding operation. By constructing a user operation profile based on historical interaction data and generating a mapping rule set accordingly, candidate interaction commands can be automatically converted into target interaction commands that better suit the target user's operating habits, thereby making the response of the target device more accurate and natural. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a human-computer interaction method in one embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a human-computer interaction device according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] In one embodiment, such as Figure 1 As shown, a human-computer interaction method is provided, including the following steps: 101. Obtain the interaction data of the human-computer interaction device, and the user operation profile corresponding to the interaction data.

[0021] In this embodiment, the above-described human-computer interaction method can be applied to a human-computer interaction device, specifically, the control system of the human-computer interaction device, which can be a touch-screen mouse. The user operation profile is determined based on the historical interaction data of the human-computer interaction device.

[0022] Specifically, the user operation profile refers to a user behavior model built based on historical interaction data, used to characterize the user's operating habits. Candidate interaction commands refer to initially identified device control commands. The mapping rule set refers to a set of command conversion logic.

[0023] Specifically, the system first acquires the user's current interaction data using the touch-sensitive mouse, including but not limited to touch position coordinates, swipe trajectory, click frequency, pressure applied, and multi-finger gesture information. To improve interaction adaptability, the system maintains historical interaction data records for the touch-sensitive mouse in the background. By analyzing features such as operation frequency, gesture preferences, and click area distribution in the historical data, a corresponding user operation profile is generated. This user operation profile may include parameters such as commonly used operation modes (e.g., single-finger swipe, multi-finger zoom), touch area heatmap, and typical response latency, which are used in subsequent steps to generate interaction commands highly matched to the user's habits.

[0024] 102. Generate candidate interaction instructions based on interaction data.

[0025] In this embodiment, after generating a user operation profile, the system further selects multiple instructions with the highest matching degree to the current interaction data from a preset interaction instruction library based on the acquired interaction data and the user's operating habits, as candidate interaction instructions. For example, when a user performs a long press and slide right on a touch mouse, the operation is matched with multiple instructions in the preset instruction library such as "move right page", "fast forward playback", and "drag and select". Several instructions are selected as candidate interaction instructions based on the similarity score, and relevant confidence parameters are attached for subsequent selection and optimization.

[0026] In one possible embodiment, the generation of the aforementioned candidate interaction commands can be dynamically generated based on a machine learning model, rather than relying on a fixed preset command library. Specifically, interaction data can be input into a deep neural network model trained on historical interaction data. The model encodes and extracts features from the interaction data in vector space, and directly generates multiple candidate interaction commands based on the probability distribution of the feature vectors on the multi-classification output layer. For example, for a multi-mode screen interaction device, after recognizing the user's swipe trajectory, the number of gestures, and the device posture, the model can simultaneously generate multiple candidate interaction commands such as "scroll list," "zoom map," and "flip image," and score each command to support subsequent optimization or combined execution.

[0027] 103. Generate a mapping rule set based on user operation profiles.

[0028] In this embodiment, after obtaining candidate interaction commands, a mapping rule set is generated based on the user operation profile. Specifically, the operation habit parameters recorded in the user operation profile (such as frequently used functions, command execution frequency, historical operation scenarios, etc.) can be associated and matched with the candidate interaction commands to generate a set of "user features - interaction commands" mapping rules.

[0029] This mapping rule set can include not only a one-to-one correspondence between instructions and user characteristics, but also applicable conditions and priority parameters. For example, for users who frequently use the two-finger zoom operation in image editing software, the mapping rule can prioritize mapping the "two-finger zoom" operation as "zoom in on the image" rather than "zoom in on the canvas," and record the priority in the rule to support fast execution.

[0030] In one possible embodiment, the mapping rule set can be automatically generated using association rule mining algorithms or pattern recognition methods, without relying on manually set priority logic. Algorithms such as Apriori and FP-Growth can be used to automatically extract high-frequency association patterns from large amounts of historical interaction data and transform these patterns into mapping rules.

[0031] For example, if the algorithm analyzes the pattern that "in a document editing scenario, there is an 80% probability that the user will execute 'Insert Comment' after right-clicking," it will automatically generate a mapping rule: when a user right-clicks and is in a document editing scenario, the "Insert Comment" function should be popped up first. This method can dynamically adapt to changes in user behavior and continuously optimize the accuracy and coverage of the rule set without manual intervention.

[0032] 104. Convert candidate interaction instructions into target interaction instructions based on the mapping rule set.

[0033] In this embodiment, after generating the mapping rule set, the instruction conversion module is invoked to convert candidate interaction instructions into target interaction instructions based on the mapping rule set. Specifically, the candidate interaction instruction list is first traversed. For each candidate interaction instruction, a matching rule item in the mapping rule set is retrieved. If a match is found, the conversion is performed according to the target instruction and its execution parameters set in the rule item. For example, when the candidate interaction instruction is "slide two fingers outwards", and the rule item in the mapping rule set corresponding to "zoom in on the image" is matched, the candidate instruction is directly converted into the target instruction "zoom in on the image" and output, thereby improving the accuracy and response speed of execution.

[0034] 105. Based on the target interaction command, control the target device corresponding to the human-computer interaction device so that the target device performs the corresponding operation.

[0035] In this embodiment, the target device can be a host, laptop computer or other smart device. The target device associated with the human-computer interaction device can be controlled based on the target interaction command so that the target device can perform the corresponding operation.

[0036] Specifically, when the target interaction command is "zoom in on the image", an image scaling control signal will be sent to the host via Bluetooth or wired communication interface, causing the host's display screen to be zoomed in immediately; when the target interaction command is "play video", a playback control signal will be sent to the multimedia player, causing it to load and play the corresponding video file, thereby achieving an instant response to the user's intent.

[0037] Furthermore, the control of the target device can be mapped using a composite gesture recognition algorithm: when the touch mouse is stationary (S0), it enters standby mode; when a single-finger swipe is detected (S1), page scrolling is achieved according to the calculation formula of speed = 0.5ΔX + 0.3ΔY; when a two-finger pinch is detected (S2), content scaling is performed according to the ratio = 1 + 0.1ΔR; when a three-finger rotation is detected (S3), the viewing angle is rotated according to the angle = Δθ × 0.8; when a four-finger pan is detected (S4), the canvas is panned according to the step size = ΔX / 10, thereby achieving multi-dimensional and refined interactive control of the target device.

[0038] In this embodiment of the invention, interaction data of the human-computer interaction device and a user operation profile corresponding to the interaction data are acquired. The user operation profile is determined based on historical interaction data of the human-computer interaction device. Based on the interaction data, candidate interaction instructions are generated. Based on the user operation profile, a mapping rule set is generated. Based on the mapping rule set, the candidate interaction instructions are converted into target interaction instructions. Based on the target interaction instructions, a target device corresponding to the human-computer interaction device is controlled to enable the target device to perform the corresponding operation. By constructing a user operation profile based on historical interaction data and generating a mapping rule set accordingly, candidate interaction instructions can be automatically converted into target interaction instructions that better suit the target user's operating habits, thereby making the response of the target device more accurate and natural.

[0039] It is understood that in the specific implementation of this application, data such as interaction data, user operation profiles, and historical interaction data are involved. When the embodiments in this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0040] Optionally, the human-computer interaction device includes a touch array and a pressure array. The interaction data includes touch data corresponding to the touch array and pressure data corresponding to the pressure array. In the step of generating candidate interaction instructions based on the interaction data, feature extraction processing can also be performed based on the touch data and pressure data to obtain corresponding temporal features and spatial features; the temporal features and spatial features are fused to obtain fused features; and the fused features are provided to a preset temporal inference model to obtain candidate interaction instructions.

[0041] In this embodiment, touch data refers to the touch point coordinates and sliding trajectory information generated when the user operates through the touch array. Specifically, it can be implemented by using a capacitive touch sensor to collect the touch point coordinate change sequence, which is used to characterize the spatial distribution and dynamic path of the user's operation. Pressure data refers to the force change information applied to the pressure array by the user during operation. Specifically, it can be implemented by using a piezoelectric thin film sensor or a strain gauge sensor to collect the pressure value sequence, which is used to reflect the touch intensity characteristics during user operation.

[0042] Specifically, temporal features refer to the dynamic patterns that change over time extracted from touch and pressure data. This can be achieved by using sliding window statistical methods to calculate parameter sequences such as touch point speed, acceleration, and pressure change gradients, used to capture the rhythmic characteristics of user operations. Spatial features refer to the touch point distribution morphology and trajectory geometry extracted from touch data. This can be achieved by using convolutional neural networks to extract texture features or trajectory curvature features from the touch point distribution map, used to characterize the spatial layout characteristics of user operations.

[0043] Specifically, the fusion feature can be a fusion feature sequence or a fusion feature vector. The fusion feature sequence refers to the joint representation after integrating temporal and spatial features into multi-dimensional data. Specifically, it can be generated by feature splicing or attention mechanism weighted fusion to construct operational feature expressions that include spatiotemporal correlation.

[0044] Specifically, a temporal reasoning model refers to a machine learning model used to process serialized data and generate instruction prediction results. It can be constructed using a long short-term memory network or a Transformer architecture, and is used to infer the interactive instructions corresponding to the user's intent based on the fused feature sequence.

[0045] Specifically, the touch data can be obtained from the five-finger touch coordinates output by a 5x5 capacitive plate array, and the pressure data can be obtained from the five-finger pressure values ​​output by a 5-dot matrix thin-film sensor. After obtaining the five-finger touch coordinates and pressure values, noise filtering can be performed on them to obtain filtered five-finger touch coordinates and pressure values. Kalman filtering can be used to eliminate touch signal jitter. Furthermore, the filtered five-finger touch coordinates and pressure values ​​can be normalized to obtain normalized five-finger touch coordinates and pressure values. The normalized five-finger touch coordinates are used as the aforementioned touch data, and the normalized five-finger pressure values ​​are used as the aforementioned pressure data.

[0046] When extracting features, for spatial features, the K-means clustering algorithm can be used to cluster the spatial distribution of the input data to obtain spatial feature vectors reflecting the differences between local and global spatial distributions. For temporal feature extraction, the displacement trend between time segments can be calculated based on the sliding direction vector, and the temporal feature vector reflecting the dynamic process can be extracted by combining the changing law of pressure gradient. After extraction, the spatial and temporal features are aligned according to timestamps or sampling order, and the numerical ranges are normalized to eliminate the dimensional differences between different features. Subsequently, according to the preset fusion strategy, multiple sets of feature vectors can be weighted, concatenated, or dimensionality reduced and fused using methods such as principal component analysis (PCA) to generate a comprehensive feature vector that can simultaneously represent spatial distribution information and dynamic change information for subsequent recognition or decision processing.

[0047] Specifically, the above feature fusion can be achieved through weighted fusion using an attention mechanism. The process is as follows: First, pressure features are used as the query vector Q, touch features as the key vector K, and joint features as the value vector V. The fusion result is then calculated using the following formula:

[0048] Where Q represents pressure characteristics, K represents touch characteristics, and V represents combined characteristics. Let be the dimension of the key vector. This is the transpose of the touch feature matrix. This calculation process adaptively assigns weights to different features, enabling feature correlation modeling and information filtering. The final output is a 1*64 dimension fused feature vector, used for subsequent generation of candidate interaction commands.

[0049] In one possible embodiment, after the touch array and pressure array synchronously collect user operation data, the touch point coordinate sequence from the touch data and the pressure value sequence from the pressure data are respectively input into the feature extraction module. The temporal feature extraction module generates a time-dimensional parameter sequence reflecting the dynamic characteristics of the operation by calculating the touch point movement speed, acceleration, and pressure change rate; the spatial feature extraction module generates a set of geometric parameters reflecting the spatial distribution characteristics of the operation by analyzing the touch point distribution density and trajectory curvature. Subsequently, the spatiotemporal feature fusion module normalizes the temporal parameter sequence and spatial geometric parameters, and generates a fused feature sequence through feature concatenation. This sequence is input into a pre-trained temporal inference model, which outputs a set of candidate interaction commands based on the mapping relationship learned from historical operation data. For example, when a user performs a two-finger swipe operation, the temporal inference model can identify zoom commands or page scrolling commands based on the fusion result of the swipe speed abrupt change feature and the touch point spacing change feature.

[0050] Optionally, in the step of generating a mapping rule set based on user operation profile, the correspondence between candidate interaction instructions and target interaction instructions can be determined according to the distribution of historical interaction data corresponding to different operation modes in the user operation profile; a mapping rule set can be constructed based on the correspondence, and the mapping rules from candidate interaction instructions to target interaction instructions can be stored in the mapping rule set.

[0051] In this embodiment, the distribution of historical interaction data refers to the statistical patterns of touch data generated by users under different operating modes in terms of time and space. Specifically, it can be quantitatively analyzed through indicators such as average sliding speed, pressure peak range, and touch area heat map to identify typical characteristics of user operating modes.

[0052] The correspondence refers to the logical matching rules between candidate interaction instructions and target interaction instructions. Specifically, it can be implemented using a conditional judgment tree or a probability mapping matrix, which is used to dynamically adjust the instruction conversion strategy based on user operation characteristics.

[0053] The mapping rule set refers to the database that stores instruction conversion rules. Specifically, it can be implemented using a key-value pair storage structure or a relational data table to support real-time query and update operations.

[0054] Specifically, during user interaction, by analyzing the distribution patterns of touch features under different operating modes in historical interaction data, a dynamic matching relationship is established between candidate interaction commands and target interaction commands. For example, for users who habitually swipe quickly, their touch data shows a right-skewed distribution in the speed dimension; swipe commands in this mode are mapped to control commands that increase cursor movement speed. For users who habitually use light pressure, their pressure data shows a low-value clustering in the intensity dimension; click commands in this mode are mapped to confirmation commands that lower the trigger threshold. The mapping rule set continuously records the command conversion rules under different operating modes to achieve iterative optimization of personalized interaction strategies.

[0055] Optionally, in the step of controlling the target device corresponding to the human-machine interaction device based on the target interaction command, the current operating status parameters and historical trajectory data of the target device can also be obtained; based on the current operating status parameters and historical trajectory data, the target interaction command is subjected to sliding compensation to obtain the compensated target interaction command; and based on the compensated target interaction command, the target device corresponding to the human-machine interaction device is controlled.

[0056] In this embodiment, the current operating status parameter refers to the operating status information collected in real time by the target device during the execution of the operation, which is used to reflect the dynamic changes of the target device during the instruction execution phase.

[0057] Historical trajectory data refers to the trajectory record of the target device's operations within a preset time period, which can be implemented using data such as timestamps, coordinate sequences, and movement directions. Slip compensation refers to the process of predicting the target device's operating status within a preset prediction period and correcting command parameters. It can be implemented using Kalman filtering, trajectory extrapolation algorithms, or neural network prediction models to eliminate user interaction control errors.

[0058] Specifically, the current operating status parameters and historical trajectory data mentioned above can be input into the sliding compensation model. This model is used to predict the possible offset or lag under the current control conditions based on these two types of data and generate a correction value. The sliding compensation model can employ a Kalman filter-based prediction method, a time series prediction method based on a deep neural network, or traditional algorithms such as linear regression or multinomial regression; the specific implementation method is not limited. Next, the original target interaction command is corrected based on the correction value to obtain the compensated target interaction command.

[0059] Optionally, in the step of performing sliding compensation on the target interaction command based on the current operating status parameters and historical trajectory data to obtain the compensated target interaction command, the operating status of the target device within a preset prediction period can also be predicted based on the current operating status parameters and historical trajectory data; the predicted operating status can be compared with the target interaction command to generate a correction amount; and the target interaction command can be adjusted according to the correction amount to obtain the compensated target interaction command.

[0060] In this embodiment of the invention, the preset prediction period refers to the length of the time window used for state prediction, and the compensated target interaction command can be specifically calculated using the following formula:

[0061] in, This is expressed as the aforementioned correction amount. This is represented as a sliding compensation coefficient, which can be adaptively adjusted according to the smoothness of user operation; for example, it could be 0.2. It is expressed as the derivative of the compensated quantity x with respect to time.

[0062] It is understandable that some users may experience discrepancies between their commands and actual needs due to hand tremors, uneven force, or slow reaction times. The correction amount is used to "calibrate" these discrepancies. By introducing a mechanism for predicting operating status and generating correction amounts based on sliding compensation using current operating parameters and historical trajectory data, the dynamic change trend of the target device within a preset prediction period can be predicted in advance. This is then compared with the target interaction commands to generate accurate correction amounts. These correction amounts can be specifically adjusted to the original target interaction commands, ensuring that the compensated commands not only meet the operating characteristics of the target device but also fully adapt to the operating habits and capability limits of special user groups (such as people with limited physical function, slow movements, or insufficient control precision).

[0063] Optionally, in the step of controlling the target device corresponding to the human-computer interaction device based on the compensated target interaction instruction, it can also be determined whether there is a currently pending interaction instruction; if there is, conflict detection is performed to determine whether there is a conflict between the compensated target interaction instruction and the currently pending interaction instruction; if there is a conflict, the conflict type is determined, and the compensated target interaction instruction and the currently pending interaction instruction are prioritized according to the conflict type to generate an interaction instruction sorting table; based on the interaction instruction sorting table, the corresponding interaction instructions are executed sequentially.

[0064] Among them, the currently pending interactive instructions refer to instructions that have been generated but have not yet been executed by the target device. They can be managed through instruction queues or buffers, and their function is to identify potential scenarios of concurrent execution of multiple instructions.

[0065] Conflict detection refers to determining whether two instructions interfere with each other in terms of their target, execution time, or execution effect. This can be achieved through instruction parameter comparison or logical operations, and its purpose is to avoid resource competition or operational conflicts during instruction execution.

[0066] Among them, conflict type refers to the specific manifestation of conflict between instructions, including the same target, overlapping execution time intervals, or mutual influence of execution effects. Its role is to provide a classification basis for priority ranking.

[0067] Priority sorting refers to determining the order of instruction execution based on preset rules or dynamic strategies. Specifically, it can be achieved through weight calculation or rule matching, and its function is to ensure that critical instructions are executed first.

[0068] The interactive instruction sorting table refers to the sequence of instructions to be executed according to priority. It can be stored using a linked list or array data structure, and its function is to clarify the logical order of instruction execution.

[0069] Specifically, after the compensated target interaction instruction is generated, the system first checks whether there are any currently pending interaction instructions that have not yet been executed. If so, it compares the target identifiers, execution time window parameters, and expected operation results of the two instructions to determine whether there is a conflict between them in terms of target, time interval, or execution effect. For example, if two instructions both require controlling the same interactive interface of the same smart computer using a touch-based mouse, but the expected operation direction or touch event type (such as dragging and closing a window) is opposite, then it is determined that the target is the same and the execution effect is conflicting.

[0070] Upon detecting a conflict, the system classifies conflicts according to preset conflict type rules, such as input device resource occupancy, time overlap, or logical contradiction, and then applies the corresponding priority strategy. For example, in input device resource occupancy conflicts, instructions involving security protection or accidental touch prevention are executed first; in time overlap conflicts, instructions with more urgent time windows are executed first. The final generated interactive instruction sorting table arranges the instructions from highest to lowest priority, and the execution modules are called sequentially to process the instructions in the sorting table.

[0071] Optionally, conflict detection includes determining whether the objects targeted by the instructions are the same, whether the execution time intervals overlap, and whether the execution effects affect each other.

[0072] Among them, whether the target of the instruction is the same refers to determining whether the interactive instructions generated by the touch mouse control the same entity in the smart computer or its associated resources. This can be achieved by matching the unique identifier of the target device (such as device ID, MAC address, etc.) or comparing the resource access path, in order to avoid duplicate control or resource occupation conflicts of the same device or the same software resource.

[0073] Whether the execution time intervals overlap refers to determining whether the expected execution time periods of interactive instructions overlap. This can be achieved by comparing instruction timestamps or by calculating based on time windows, in order to avoid abnormal responses or erratic operations on the smart computer due to concurrent execution of touch input.

[0074] Whether the execution results affect each other refers to determining whether the execution results of interactive instructions are logically mutually exclusive or physically interfering. This can be achieved through logical rule base matching or execution result deduction based on state machine models, in order to avoid target device action failure or security risks caused by conflicts.

[0075] In the specific implementation, when multiple interactive instructions to be executed are detected, the target object identifier of each instruction is first extracted. If the same identifier exists, it is determined to be an object conflict. Secondly, the start time and duration parameters of each instruction are parsed. If there is an intersection of time intervals, it is determined to be a time conflict. Finally, the target device state change after execution is deduced through a predefined execution effect rule base. If there is mutual exclusion of states, it is determined to be an effect conflict.

[0076] For example, if there are two cursor movement commands targeting the same smart computer application window, and the execution times of the two commands overlap and the movement directions are opposite, it can be determined as a multiple conflict type and processed according to the preset priority rules.

[0077] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0078] In one embodiment, a human-computer interaction device is provided, which corresponds one-to-one with the human-computer interaction methods described in the above embodiments. For example... Figure 2 As shown, the human-computer interaction device includes a first acquisition module 201, a first generation module 202, a second generation module 203, a first conversion module 204, and a first control module 205. Detailed descriptions of each functional module are as follows: The first acquisition module 201 is used to acquire the interaction data of the human-computer interaction device and the user operation profile corresponding to the interaction data. The user operation profile is determined based on the historical interaction data of the human-computer interaction device. The first generation module 202 is used to generate candidate interaction instructions based on the interaction data; The second generation module 203 is used to generate a mapping rule set based on the user operation profile; The first conversion module 204 is used to convert the candidate interaction instruction into a target interaction instruction based on the mapping rule set; The first control module 205 is used to control the target device corresponding to the human-computer interaction device based on the target interaction command, so that the target device performs the corresponding operation.

[0079] Optionally, the human-computer interaction device includes a touch array and a pressure array, the interaction data includes touch data corresponding to the touch array and pressure data corresponding to the pressure array, and the first generation module 202 is further configured to: Based on the touch data and the pressure data, feature extraction processing is performed to obtain the corresponding temporal and spatial features; The temporal features and spatial features are fused to obtain fused features; The fused features are provided to a preset temporal reasoning model to obtain the candidate interaction instructions.

[0080] Optionally, the second generation module 203 is further configured to: Based on the distribution of historical interaction data corresponding to different operation modes in the user operation profile, the correspondence between candidate interaction commands and target interaction commands is determined. A mapping rule set is constructed based on the correspondence, and the mapping rules from the candidate interaction instruction to the target interaction instruction are stored in the mapping rule set.

[0081] Optionally, the first control module 205 is further configured to: Obtain the current operating status parameters and historical trajectory data of the target device; Based on the current operating status parameters and the historical trajectory data, the target interaction command is subjected to sliding compensation to obtain the compensated target interaction command. Based on the compensated target interaction command, the target device corresponding to the human-computer interaction device is controlled.

[0082] Optionally, the first control module 205 is further configured to: Based on the current operating status parameters and the historical trajectory data, predict the operating status of the target device within a preset prediction period; The predicted running state is compared with the target interaction command to generate a correction amount; The target interaction command is adjusted according to the correction amount to obtain the compensated target interaction command.

[0083] Optionally, the first control module 205 is further configured to: Determine if there is an interactive instruction to be executed; If a conflict exists, a conflict detection is performed to determine whether there is a conflict between the compensated target interaction instruction and the currently executed interaction instruction. If a conflict exists, the conflict type is determined, and the compensated target interaction instruction and the currently executed interaction instruction are prioritized according to the conflict type to generate an interaction instruction sorting table. Based on the interaction instruction sorting table, the corresponding interaction instructions are executed in sequence.

[0084] Optionally, the conflict detection includes determining whether the objects targeted by the instructions are the same, whether the execution time intervals overlap, and whether the execution effects affect each other.

[0085] For specific limitations regarding the human-computer interaction device, please refer to the limitations on the human-computer interaction method above, which will not be repeated here. Each module in the aforementioned human-computer interaction device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0086] In one embodiment, a computer device is provided, which may be a terminal device, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a readable storage medium storing computer-readable instructions. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer-readable instructions implement a human-computer interaction method. The readable storage medium provided in this embodiment includes both non-volatile and volatile readable storage media.

[0087] In this application embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, it implements the steps of the human-computer interaction method described above.

[0088] In one embodiment of the application, a readable storage medium is provided, which stores computer-readable instructions. When the computer-readable instructions are executed by a processor, they implement the steps of the human-computer interaction method described above.

[0089] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware with computer-readable instructions. These computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When executed, these computer-readable instructions can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0090] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0091] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A human-computer interaction method, characterized in that, Applied to human-computer interaction devices, the method includes: The interaction data of the human-computer interaction device and the user operation profile corresponding to the interaction data are obtained, and the user operation profile is determined based on the historical interaction data of the human-computer interaction device. Based on the interaction data, candidate interaction instructions are generated; Based on the user operation profile, a mapping rule set is generated; Based on the mapping rule set, the candidate interaction instructions are converted into target interaction instructions; Based on the target interaction command, the target device corresponding to the human-computer interaction device is controlled so that the target device performs the corresponding operation.

2. The human-computer interaction method as described in claim 1, characterized in that, The human-computer interaction device includes a touch array and a pressure array. The interaction data includes touch data corresponding to the touch array and pressure data corresponding to the pressure array. The step of generating candidate interaction commands based on the interaction data includes: Based on the touch data and the pressure data, feature extraction processing is performed to obtain the corresponding temporal and spatial features; The temporal features and spatial features are fused to obtain fused features; The fused features are provided to a preset temporal reasoning model to obtain the candidate interaction instructions.

3. The human-computer interaction method as described in claim 1 or 2, characterized in that, The generation of the mapping rule set based on the user operation profile includes: Based on the distribution of historical interaction data corresponding to different operation modes in the user operation profile, the correspondence between candidate interaction commands and target interaction commands is determined. A mapping rule set is constructed based on the correspondence, and the mapping rules from the candidate interaction instruction to the target interaction instruction are stored in the mapping rule set.

4. The human-computer interaction method as described in claim 1, characterized in that, The step of controlling the target device corresponding to the human-computer interaction device based on the target interaction command includes: Obtain the current operating status parameters and historical trajectory data of the target device; Based on the current operating status parameters and the historical trajectory data, the target interaction command is subjected to sliding compensation to obtain the compensated target interaction command. Based on the compensated target interaction command, the target device corresponding to the human-computer interaction device is controlled.

5. The human-computer interaction method as described in claim 4, characterized in that, The step of performing slide compensation on the target interaction command based on the current operating status parameters and the historical trajectory data to obtain the compensated target interaction command includes: Based on the current operating status parameters and the historical trajectory data, predict the operating status of the target device within a preset prediction period; The predicted running state is compared with the target interaction command to generate a correction amount; The target interaction command is adjusted according to the correction amount to obtain the compensated target interaction command.

6. The human-computer interaction method as described in claim 4, characterized in that, The step of controlling the target device corresponding to the human-computer interaction device based on the compensated target interaction command includes: Determine if there is an interactive instruction to be executed; If a conflict exists, a conflict detection is performed to determine whether there is a conflict between the compensated target interaction instruction and the currently executed interaction instruction. If a conflict exists, the conflict type is determined, and the compensated target interaction instruction and the currently executed interaction instruction are prioritized according to the conflict type to generate an interaction instruction sorting table. Based on the interaction instruction sorting table, the corresponding interaction instructions are executed in sequence.

7. The human-computer interaction method as described in claim 6, characterized in that, The conflict detection includes determining whether the instructions affect the same object, whether the execution time intervals overlap, and whether the execution effects affect each other.

8. A human-computer interaction device, characterized in that, include: The first acquisition module is used to acquire the interaction data of the human-computer interaction device and the user operation profile corresponding to the interaction data. The user operation profile is determined based on the historical interaction data of the human-computer interaction device. The first generation module is used to generate candidate interaction instructions based on the interaction data; The second generation module is used to generate a mapping rule set based on the user operation profile; The first conversion module is used to convert the candidate interaction instruction into a target interaction instruction based on the mapping rule set; The first control module is used to control the target device corresponding to the human-computer interaction device based on the target interaction command, so that the target device performs the corresponding operation.

9. A computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and running on the processor, characterized in that, When the processor executes the computer-readable instructions, it implements the human-computer interaction method as described in any one of claims 1 to 7.

10. A readable storage medium having computer-readable instructions stored thereon, characterized in that, When the computer-readable instructions are executed by the processor, they implement the human-computer interaction method as described in any one of claims 1 to 7.

Images