Information processing method and device, equipment, storage medium and computer program product

Abstract

The invention provides an information processing method and device, equipment, a storage medium and a computer program product, and the information processing method comprises the steps: obtaining a gesture sequence of at least one user according to the point cloud data of at least two visual angles; acquiring concurrent gesture information according to the gesture sequence; acquiring a priority coefficient of each gesture instruction according to priority factor information of each gesture instruction corresponding to the concurrent gesture information; the priority factor information comprises a device level of a device which the gesture instruction aims at, a current load of the device which the gesture instruction aims at, an alarm level of a service and / or the device which the gesture instruction aims at, and an operation and maintenance window occupancy rate of the device which the gesture instruction aims at; the gesture instruction corresponds to at least one of an operator role of the user and isolation domain information corresponding to the gesture instruction; and executing a gesture instruction response operation according to the priority coefficient. According to the scheme, the problem that in the prior art, an information processing scheme for the gesture instruction is poor in response accuracy can be well solved.

Description

Information processing methods, apparatus, equipment, storage media and computer program products Technical Field

[0001] This application relates to the field of information processing technology, and in particular to an information processing method, apparatus, device, storage medium, and computer program product. Background Technology

[0002] While existing multi-user depth camera interaction systems can achieve basic 3D gesture recognition, they still rely on static or empirical values ​​to determine conflict relationships in high-concurrency scenarios. Therefore, when the number of gestures increases sharply or the status of backend resources fluctuates rapidly, they often cannot provide timely and precise queuing and rate limiting strategies, resulting in inaccurate command responses.

[0003] As shown above, existing information processing solutions for gesture commands suffer from problems such as poor response accuracy. Summary of the Invention

[0004] The purpose of this application is to provide an information processing method, apparatus, device, storage medium, and computer program product to solve the problem of poor response accuracy in existing information processing schemes for gesture commands.

[0005] To address the aforementioned technical problems, this application provides an information processing method, comprising: acquiring a gesture sequence of at least one user based on point cloud data from at least two perspectives; acquiring concurrent gesture information based on the gesture sequence; acquiring a priority coefficient for each gesture instruction based on priority factor information of each gesture instruction corresponding to the concurrent gesture information; the priority factor information including at least one of the following: device level of the device targeted by the gesture instruction, current load of the device targeted by the gesture instruction, alarm level of the service and / or device targeted by the gesture instruction, occupancy rate of the operation and maintenance window of the device targeted by the gesture instruction, operator role of the user corresponding to the gesture instruction, and isolation domain information corresponding to the gesture instruction; and executing a gesture instruction response operation based on the priority coefficient.

[0006] Optionally, obtaining the gesture sequence of at least one user based on point cloud data from at least two perspectives includes: fusing and filtering the point cloud data from at least two perspectives to obtain the gesture trajectory of at least one user; determining the segmentation window length corresponding to each gesture trajectory based on the average velocity of at least two frames in each gesture trajectory; segmenting the corresponding gesture trajectory based on the segmentation window length to obtain candidate gesture segments for each user; and obtaining the gesture sequence of the at least one user based on the candidate gesture segments for each user.

[0007] Optionally, the step of segmenting the corresponding gesture trajectory according to the segmentation window length of each gesture trajectory to obtain candidate gesture segments for each user includes: obtaining the first derivative sign change information of the velocity curve corresponding to the first gesture trajectory according to the first segmentation window length of the first user's first gesture trajectory; determining the segmentation boundary according to the first derivative sign change information; and segmenting the first gesture trajectory according to the segmentation boundary to obtain candidate gesture segments for the first user.

[0008] Optionally, obtaining concurrent gesture information based on the gesture sequence includes: constructing a gesture concurrency relationship matrix based on a first condition and the gesture sequence; the first condition is determined based on two adjacent sliding windows; for each element in the gesture concurrency relationship matrix, the corresponding skeletal trajectory sequence, duration, and semantic information are concatenated and embedded to obtain a multi-dimensional feature vector; the semantic information includes at least one of function tags, associated devices, and service levels; obtaining a mutual exclusion score between any two elements in the gesture concurrency relationship matrix based on the multi-dimensional feature vector; and obtaining concurrent gesture information based on the elements corresponding to the mutual exclusion scores that are higher than a first threshold.

[0009] Optionally, obtaining the priority coefficient of each gesture instruction based on the priority factor information of each gesture instruction corresponding to the concurrent gesture information includes: obtaining the priority coefficient of each gesture instruction based on the window heat coefficient and the priority factor information of each gesture instruction corresponding to the concurrent gesture information.

[0010] Optionally, it also includes: obtaining the window heat coefficient based on the alarm density and traffic fluctuation rate within the first time period.

[0011] Optionally, the step of performing a gesture command response operation based on the priority coefficient includes: obtaining the difference between the priority coefficients of two mutually exclusive gesture commands; setting a lock delay for the gesture command corresponding to a target priority coefficient; the lock delay is determined based on the difference corresponding to the target priority coefficient and the device level; the target priority coefficient refers to the lower priority coefficient among the two priority coefficients corresponding to the difference greater than a second threshold; and performing a gesture command response operation based on the priority coefficient and the lock delay.

[0012] Optionally, the step of executing the gesture command response operation according to the priority coefficient includes: obtaining gesture command intent information according to the priority coefficient; executing the gesture command response operation corresponding to the priority coefficient using a digital twin network according to the gesture command intent information to obtain a prediction result; determining a hierarchical risk score based on the prediction result, intent interruption rate, vortex center parameter information, and reduction number; the vortex center parameter information includes: vortex center radius, lifetime, and blockage increment; and executing the gesture command response operation for gesture commands corresponding to the hierarchical risk score that is below a third threshold.

[0013] Optionally, the step of using a digital twin network to execute the gesture command response operation corresponding to the priority coefficient based on the gesture command intent information to obtain the prediction result includes: constructing a resource matrix corresponding to the resources required by the command based on the gesture command intent information; constructing a potential field on the resource matrix using a conflict vortex engine; determining vortex center parameter information based on the permission coverage tensor, link saturation tensor, lock wait tensor, and the potential field; and executing the gesture command response operation corresponding to the priority coefficient using the digital twin network based on the vortex center parameter information to obtain the prediction result.

[0014] Optionally, it further includes: determining a conflict coverage rate for a target gesture instruction; the target gesture instruction is a gesture instruction corresponding to the hierarchical risk score that is greater than or equal to the third threshold; performing a first operation for the target gesture instruction based on the conflict coverage rate; the first operation includes: increasing the delay duration and unlocking conditions; or, re-acquiring the priority coefficient.

[0015] Optionally, determining the conflict coverage rate for a target gesture instruction includes: acquiring category parameter information of the target gesture instruction; the category parameter information includes at least one of trajectory, context, and telemetry signature; if the category parameter information indicates that the target gesture instruction belongs to a target category, increasing the sampling rate of the original sensor stream for the target gesture instruction; and determining the conflict coverage rate based on the sampling rate.

[0016] Optionally, it also includes: determining the delay duration based on real-time power consumption information and dynamic power consumption limit.

[0017] This application embodiment also provides an information processing device, including: a first acquisition module, configured to acquire a gesture sequence of at least one user based on point cloud data from at least two perspectives; a second acquisition module, configured to acquire concurrent gesture information based on the gesture sequence; a third acquisition module, configured to acquire a priority coefficient of each gesture instruction based on priority factor information of each gesture instruction corresponding to the concurrent gesture information; the priority factor information includes at least one of the following: device level of the device targeted by the gesture instruction, current load of the device targeted by the gesture instruction, alarm level of the service and / or device targeted by the gesture instruction, occupancy rate of the operation and maintenance window of the device targeted by the gesture instruction, operator role of the user corresponding to the gesture instruction, and isolation domain information corresponding to the gesture instruction; and a first execution module, configured to execute a gesture instruction response operation based on the priority coefficient.

[0018] Optionally, obtaining the gesture sequence of at least one user based on point cloud data from at least two perspectives includes: fusing and filtering the point cloud data from at least two perspectives to obtain the gesture trajectory of at least one user; determining the segmentation window length corresponding to each gesture trajectory based on the average velocity of at least two frames in each gesture trajectory; segmenting the corresponding gesture trajectory based on the segmentation window length to obtain candidate gesture segments for each user; and obtaining the gesture sequence of the at least one user based on the candidate gesture segments for each user.

[0019] Optionally, the step of segmenting the corresponding gesture trajectory according to the segmentation window length of each gesture trajectory to obtain candidate gesture segments for each user includes: obtaining the first derivative sign change information of the velocity curve corresponding to the first gesture trajectory according to the first segmentation window length of the first user's first gesture trajectory; determining the segmentation boundary according to the first derivative sign change information; and segmenting the first gesture trajectory according to the segmentation boundary to obtain candidate gesture segments for the first user.

[0020] Optionally, obtaining concurrent gesture information based on the gesture sequence includes: constructing a gesture concurrency relationship matrix based on a first condition and the gesture sequence; the first condition is determined based on two adjacent sliding windows; for each element in the gesture concurrency relationship matrix, the corresponding skeletal trajectory sequence, duration, and semantic information are concatenated and embedded to obtain a multi-dimensional feature vector; the semantic information includes at least one of function tags, associated devices, and service levels; obtaining a mutual exclusion score between any two elements in the gesture concurrency relationship matrix based on the multi-dimensional feature vector; and obtaining concurrent gesture information based on the elements corresponding to the mutual exclusion scores that are higher than a first threshold.

[0021] Optionally, obtaining the priority coefficient of each gesture instruction based on the priority factor information of each gesture instruction corresponding to the concurrent gesture information includes: obtaining the priority coefficient of each gesture instruction based on the window heat coefficient and the priority factor information of each gesture instruction corresponding to the concurrent gesture information.

[0022] Optionally, it also includes: a fourth acquisition module, used to acquire the window heat coefficient based on the alarm density and traffic fluctuation rate within the first time period.

[0023] Optionally, the step of performing a gesture command response operation based on the priority coefficient includes: obtaining the difference between the priority coefficients of two mutually exclusive gesture commands; setting a lock delay for the gesture command corresponding to a target priority coefficient; the lock delay is determined based on the difference corresponding to the target priority coefficient and the device level; the target priority coefficient refers to the lower priority coefficient among the two priority coefficients corresponding to the difference greater than a second threshold; and performing a gesture command response operation based on the priority coefficient and the lock delay.

[0024] Optionally, the step of executing the gesture command response operation according to the priority coefficient includes: obtaining gesture command intent information according to the priority coefficient; executing the gesture command response operation corresponding to the priority coefficient using a digital twin network according to the gesture command intent information to obtain a prediction result; determining a hierarchical risk score based on the prediction result, intent interruption rate, vortex center parameter information, and reduction number; the vortex center parameter information includes: vortex center radius, lifetime, and blockage increment; and executing the gesture command response operation for gesture commands corresponding to the hierarchical risk score that is below a third threshold.

[0025] Optionally, the step of using a digital twin network to execute the gesture command response operation corresponding to the priority coefficient based on the gesture command intent information to obtain the prediction result includes: constructing a resource matrix corresponding to the resources required by the command based on the gesture command intent information; constructing a potential field on the resource matrix using a conflict vortex engine; determining vortex center parameter information based on the permission coverage tensor, link saturation tensor, lock wait tensor, and the potential field; and executing the gesture command response operation corresponding to the priority coefficient using the digital twin network based on the vortex center parameter information to obtain the prediction result.

[0026] Optionally, it further includes: a first determining module, used to determine a conflict coverage rate for a target gesture instruction; the target gesture instruction is a gesture instruction corresponding to the hierarchical risk score that is greater than or equal to the third threshold; a second execution module, used to perform a first operation for the target gesture instruction based on the conflict coverage rate; the first operation includes: increasing the delay duration and unlocking conditions; or, re-acquiring the priority coefficient.

[0027] Optionally, determining the conflict coverage rate for a target gesture instruction includes: acquiring category parameter information of the target gesture instruction; the category parameter information includes at least one of trajectory, context, and telemetry signature; if the category parameter information indicates that the target gesture instruction belongs to a target category, increasing the sampling rate of the original sensor stream for the target gesture instruction; and determining the conflict coverage rate based on the sampling rate.

[0028] Optionally, it also includes: a second determining module, used to determine the delay duration based on real-time power consumption information and dynamic power consumption limit.

[0029] This application embodiment also provides an information processing device, including: a processor; the processor is configured to: acquire a gesture sequence of at least one user based on point cloud data from at least two perspectives; acquire concurrent gesture information based on the gesture sequence; acquire a priority coefficient for each gesture instruction based on priority factor information of each gesture instruction corresponding to the concurrent gesture information; the priority factor information includes at least one of the following: device level of the device targeted by the gesture instruction, current load of the device targeted by the gesture instruction, alarm level of the service and / or device targeted by the gesture instruction, occupancy rate of the operation and maintenance window of the device targeted by the gesture instruction, operator role of the user corresponding to the gesture instruction, and isolation domain information corresponding to the gesture instruction; and execute a gesture instruction response operation based on the priority coefficient.

[0030] Optionally, obtaining the gesture sequence of at least one user based on point cloud data from at least two perspectives includes: fusing and filtering the point cloud data from at least two perspectives to obtain the gesture trajectory of at least one user; determining the segmentation window length corresponding to each gesture trajectory based on the average velocity of at least two frames in each gesture trajectory; segmenting the corresponding gesture trajectory based on the segmentation window length to obtain candidate gesture segments for each user; and obtaining the gesture sequence of the at least one user based on the candidate gesture segments for each user.

[0031] Optionally, the step of segmenting the corresponding gesture trajectory according to the segmentation window length of each gesture trajectory to obtain candidate gesture segments for each user includes: obtaining the first derivative sign change information of the velocity curve corresponding to the first gesture trajectory according to the first segmentation window length of the first user's first gesture trajectory; determining the segmentation boundary according to the first derivative sign change information; and segmenting the first gesture trajectory according to the segmentation boundary to obtain candidate gesture segments for the first user.

[0032] Optionally, obtaining concurrent gesture information based on the gesture sequence includes: constructing a gesture concurrency relationship matrix based on a first condition and the gesture sequence; the first condition is determined based on two adjacent sliding windows; for each element in the gesture concurrency relationship matrix, the corresponding skeletal trajectory sequence, duration, and semantic information are concatenated and embedded to obtain a multi-dimensional feature vector; the semantic information includes at least one of function tags, associated devices, and service levels; obtaining a mutual exclusion score between any two elements in the gesture concurrency relationship matrix based on the multi-dimensional feature vector; and obtaining concurrent gesture information based on the elements corresponding to the mutual exclusion scores that are higher than a first threshold.

[0033] Optionally, obtaining the priority coefficient of each gesture instruction based on the priority factor information of each gesture instruction corresponding to the concurrent gesture information includes: obtaining the priority coefficient of each gesture instruction based on the window heat coefficient and the priority factor information of each gesture instruction corresponding to the concurrent gesture information.

[0034] Optionally, the processor is further configured to: obtain the window heat coefficient based on the alarm density and traffic fluctuation rate within a first time period.

[0035] Optionally, the step of performing a gesture command response operation based on the priority coefficient includes: obtaining the difference between the priority coefficients of two mutually exclusive gesture commands; setting a lock delay for the gesture command corresponding to a target priority coefficient; the lock delay is determined based on the difference corresponding to the target priority coefficient and the device level; the target priority coefficient refers to the lower priority coefficient among the two priority coefficients corresponding to the difference greater than a second threshold; and performing a gesture command response operation based on the priority coefficient and the lock delay.

[0036] Optionally, the step of executing the gesture command response operation according to the priority coefficient includes: obtaining gesture command intent information according to the priority coefficient; executing the gesture command response operation corresponding to the priority coefficient using a digital twin network according to the gesture command intent information to obtain a prediction result; determining a hierarchical risk score based on the prediction result, intent interruption rate, vortex center parameter information, and reduction number; the vortex center parameter information includes: vortex center radius, lifetime, and blockage increment; and executing the gesture command response operation for gesture commands corresponding to the hierarchical risk score that is below a third threshold.

[0037] Optionally, the step of using a digital twin network to execute the gesture command response operation corresponding to the priority coefficient based on the gesture command intent information to obtain the prediction result includes: constructing a resource matrix corresponding to the resources required by the command based on the gesture command intent information; constructing a potential field on the resource matrix using a conflict vortex engine; determining vortex center parameter information based on the permission coverage tensor, link saturation tensor, lock wait tensor, and the potential field; and executing the gesture command response operation corresponding to the priority coefficient using the digital twin network based on the vortex center parameter information to obtain the prediction result.

[0038] Optionally, the processor is further configured to: determine a conflict coverage rate for a target gesture instruction; the target gesture instruction is a gesture instruction corresponding to the hierarchical risk score that is greater than or equal to the third threshold; perform a first operation for the target gesture instruction based on the conflict coverage rate; the first operation includes: increasing the delay duration and unlocking conditions; or, re-acquiring the priority coefficient.

[0039] Optionally, determining the conflict coverage rate for a target gesture instruction includes: acquiring category parameter information of the target gesture instruction; the category parameter information includes at least one of trajectory, context, and telemetry signature; if the category parameter information indicates that the target gesture instruction belongs to a target category, increasing the sampling rate of the original sensor stream for the target gesture instruction; and determining the conflict coverage rate based on the sampling rate.

[0040] Optionally, the processor is further configured to: determine the delay duration based on real-time power consumption information and dynamic power consumption limit.

[0041] This application also provides an information processing device, including a memory, a processor, and a program stored in the memory and executable on the processor; when the processor executes the program, it implements the above-described information processing method.

[0042] This application also provides a readable storage medium storing a program thereon, which, when executed by a processor, implements the steps in the information processing method described above.

[0043] This application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the information processing method described above.

[0044] The beneficial effects of the above technical solution of this application are as follows: In the above solution, the information processing method obtains the gesture sequence of at least one user based on point cloud data from at least two perspectives; obtains concurrent gesture information based on the gesture sequence; obtains the priority coefficient of each gesture instruction based on the priority factor information of each gesture instruction corresponding to the concurrent gesture information; the priority factor information includes at least one of the following: the device level of the device targeted by the gesture instruction, the current load of the device targeted by the gesture instruction, the alarm level of the service and / or device targeted by the gesture instruction, the occupancy rate of the operation and maintenance window of the device targeted by the gesture instruction, the operator role of the user corresponding to the gesture instruction, and the isolation domain information corresponding to the gesture instruction; and executes the gesture instruction response operation based on the priority coefficient; it can support the dynamic generation of gesture instruction priorities in high-concurrency scenarios to achieve fine-grained conflict judgment and instruction sorting, and also supports timely and fine-grained queuing and rate limiting strategies when the number of gestures increases sharply or the status of backend resources fluctuates rapidly, thereby improving the accuracy of instruction response and effectively solving the problem of poor response accuracy in the information processing schemes for gesture instructions in the prior art. Attached Figure Description

[0045] Figure 1 is a schematic flowchart of the information processing method according to an embodiment of this application; Figure 2 is a comparative schematic diagram of information processing according to an embodiment of this application; Figure 3 is a schematic diagram of the specific implementation of the information processing method according to an embodiment of this application (first); Figure 4 is a schematic diagram of the specific implementation of the information processing method according to an embodiment of this application (second); Figure 5 is a schematic diagram of the specific implementation of the information processing method according to an embodiment of this application (third); Figure 6 is a schematic diagram of the structure of the information processing device according to an embodiment of this application; Figure 7 is a schematic diagram of the structure of the information processing equipment according to an embodiment of this application. Detailed Implementation

[0046] To make the technical problems, technical solutions and advantages of this application clearer, a detailed description will be provided below in conjunction with the accompanying drawings and specific embodiments.

[0047] This application addresses the problem of poor response accuracy in existing information processing schemes for gesture commands by providing an information processing method, as shown in Figure 1, including: Step 11: Obtaining a gesture sequence of at least one user based on point cloud data from at least two perspectives; the user corresponds to the point cloud data (e.g., the point cloud data is collected for the user).

[0048] The "point cloud data from at least two perspectives" can be obtained from acquisition devices (such as cameras) positioned in multiple locations. Gestures in this solution can also be other types of commands, such as other limb movements (e.g., arm movements, head movements). In other words, other forms of command flows are also applicable to this solution. Correspondingly, the skeletal flow mentioned below can be replaced with a multimodal interactive trajectory flow; the resource touchpoint 3D matrix can be replaced with a resource-channel-time tensor matrix; furthermore, the conflict vortex engine can be replaced with other general-purpose risk simulation engines and digital twin interfaces. That is, any general-purpose risk simulation engine and digital twin interface capable of achieving the following information acquisition functions of this solution is acceptable.

[0049] Step 12: Obtain concurrent gesture information based on the gesture sequence.

[0050] The concurrent gesture information may include relevant information for gestures that are mutually exclusive.

[0051] Step 13: Obtain the priority coefficient of each gesture command based on the priority factor information of each gesture command corresponding to the concurrent gesture information; the priority factor information includes at least one of the following: the device level of the device targeted by the gesture command, the current load of the device targeted by the gesture command, the alarm level of the service and / or device targeted by the gesture command, the maintenance window occupancy rate of the device targeted by the gesture command, the operator role of the user corresponding to the gesture command, and the isolation domain information corresponding to the gesture command.

[0052] The priority factor information can be real-time or pre-collected, and is not limited here. The isolation domain information can be used to indicate whether the corresponding gesture command crosses a network region or a geographical region.

[0053] Step 14: Execute the gesture command response operation according to the priority coefficient.

[0054] Optionally, predictions can be made based on priority coefficients, and the response can be executed only if the prediction results meet the execution conditions.

[0055] The information processing method provided in this application embodiment obtains a gesture sequence of at least one user based on point cloud data from at least two perspectives; the user corresponds to the point cloud data; concurrent gesture information is obtained based on the gesture sequence; priority coefficients of each gesture command are obtained based on priority factor information of each gesture command corresponding to the concurrent gesture information; the priority factor information includes at least one of the following: device level of the device targeted by the gesture command, current load of the device targeted by the gesture command, alarm level of the service and / or device targeted by the gesture command, occupancy rate of the operation and maintenance window of the device targeted by the gesture command, operator role of the user corresponding to the gesture command, and isolation domain information corresponding to the gesture command; and a gesture command response operation is executed based on the priority coefficient; it can support the dynamic generation of gesture command priorities in high-concurrency scenarios to achieve fine-grained conflict determination and command sorting, and also supports timely and fine-grained queuing and rate limiting strategies when the number of gestures increases sharply or the status of backend resources fluctuates rapidly, improving the accuracy of command response, and effectively solving the problem of poor response accuracy in existing information processing schemes for gesture commands.

[0056] The step of obtaining the gesture sequence of at least one user based on point cloud data from at least two perspectives includes: fusing and filtering the point cloud data from at least two perspectives to obtain the gesture trajectory of at least one user; determining the segmentation window length corresponding to each gesture trajectory based on the average velocity of at least two frames in each gesture trajectory; segmenting the corresponding gesture trajectory according to the segmentation window length to obtain candidate gesture segments for each user; and obtaining the gesture sequence of the at least one user based on the candidate gesture segments for each user. This enables adaptive determination of window boundaries, reduces identity drift and over-segmentation or under-segmentation, and accurately obtains the gesture sequence. The "data filtering" may include at least one of operations such as jitter frame removal and duplicate click merging, which is not limited here.

[0057] In this embodiment, the step of segmenting the corresponding gesture trajectory according to the segmentation window length to obtain candidate gesture segments for each user includes: obtaining the first derivative sign change information of the velocity curve corresponding to the first gesture trajectory based on the first segmentation window length of the first user's first gesture trajectory; determining the segmentation boundary based on the first derivative sign change information; and segmenting the first gesture trajectory according to the segmentation boundary to obtain candidate gesture segments for the first user. This allows for adaptive boundary determination of the window based on the first derivative sign change of the velocity curve, thereby accurately obtaining the candidate gesture segments through trajectory segmentation.

[0058] The step of obtaining concurrent gesture information based on the gesture sequence includes: constructing a gesture concurrency relationship matrix based on a first condition and the gesture sequence; the first condition is determined based on two adjacent sliding windows; for each element in the gesture concurrency relationship matrix, the corresponding skeletal trajectory sequence, duration, and semantic information are concatenated and embedded to obtain a multi-dimensional feature vector; the semantic information includes at least one of function tags, associated devices, and service levels; obtaining the mutual exclusion score between any two elements in the gesture concurrency relationship matrix based on the multi-dimensional feature vector; and obtaining concurrent gesture information based on the elements corresponding to the mutual exclusion scores that are higher than a first threshold. This allows for accurate acquisition of the concurrent gesture information. The "construction of the gesture concurrency relationship matrix" considers both concurrent window operations and adjacent window cross-triggers simultaneously through the first condition. "Embedding processing" can be scenario-based embedding, and / or "concurrent gesture information" can include relevant conflict index information, but is not limited to these.

[0059] In this embodiment, obtaining the priority coefficient of each gesture instruction based on the priority factor information of each gesture instruction corresponding to the concurrent gesture information includes: obtaining the priority coefficient of each gesture instruction based on the window heat coefficient and the priority factor information of each gesture instruction corresponding to the concurrent gesture information. This allows for accurate determination of the scene-adaptive priority coefficient.

[0060] Furthermore, the information processing method further includes: obtaining the window heat coefficient based on the alarm density and traffic fluctuation rate within a first time period. This enables real-time updates of the window heat coefficient.

[0061] The step of executing the gesture command response operation based on the priority coefficient includes: obtaining the difference between the priority coefficients of two mutually exclusive gesture commands; setting a lock delay for the gesture command corresponding to a target priority coefficient; the lock delay is determined based on the difference corresponding to the target priority coefficient and the device level; the target priority coefficient refers to the lower of the two priority coefficients corresponding to the difference greater than a second threshold; and executing the gesture command response operation based on the priority coefficient and the lock delay. This achieves differential threshold control and delay locking, ensuring accurate execution of the gesture command response operation.

[0062] In this embodiment, the step of executing a gesture command response operation based on the priority coefficient includes: obtaining gesture command intent information based on the priority coefficient; executing the gesture command response operation corresponding to the priority coefficient using a digital twin network based on the gesture command intent information to obtain a prediction result; determining a hierarchical risk score based on the prediction result, intent interruption rate, vortex center parameter information, and the number of reductions; the vortex center parameter information includes: vortex center radius, lifetime, and blocking increment; and executing a gesture command response operation for gesture commands corresponding to the hierarchical risk score below a third threshold. This supports resource conflict prediction and ensures accurate and normal execution of commands as much as possible. The gesture command intent information can be a real-time intent graph. "Obtaining gesture command intent information based on the priority coefficient" may include: remapping the gesture trajectory corresponding to the priority coefficient to a unified time base to form a two-dimensional gesture-clock grid based on the priority coefficient; obtaining intent micro-units based on the two-dimensional gesture-clock grid; and disassembling the intent micro-units to generate a real-time intent graph, but is not limited to this.

[0063] The step of executing the gesture command response operation corresponding to the priority coefficient using a digital twin network based on the gesture command intent information to obtain the prediction result includes: constructing a resource matrix corresponding to the resources required by the command based on the gesture command intent information; constructing a potential field on the resource matrix using a conflict vortex engine; determining vortex center parameter information based on the permission coverage tensor, link saturation tensor, lock wait tensor, and the potential field; and executing the gesture command response operation corresponding to the priority coefficient using the digital twin network based on the vortex center parameter information to obtain the prediction result. This allows for accurate acquisition of the prediction result. The "resource matrix" may include three-dimensional elements: CPU cores, network segments, and power circuits.

[0064] Furthermore, the information processing method further includes: determining a conflict coverage rate for a target gesture instruction; the target gesture instruction is a gesture instruction corresponding to the hierarchical risk score that is greater than or equal to the third threshold; and performing a first operation on the target gesture instruction based on the conflict coverage rate; the first operation includes: increasing the delay duration and unlocking conditions; or, re-acquiring the priority coefficient. This enables conflict handling and executable resolution.

[0065] The step of determining the conflict coverage rate for a target gesture command includes: acquiring category parameter information of the target gesture command; the category parameter information includes at least one of trajectory, context, and telemetry signature; if the category parameter information indicates that the target gesture command belongs to a target category, increasing the sampling rate of the original sensor stream for the target gesture command; and determining the conflict coverage rate based on the sampling rate. This allows for accurate determination of the conflict coverage rate. The "target category" can be, for example, a category labeled "suspected" or "unknown" based on a corresponding threshold, but is not limited to this.

[0066] Furthermore, the information processing method further includes: determining the delay duration based on real-time power consumption information and a dynamic power consumption limit. This allows for accurate determination of the delay duration. The "real-time power consumption information" can be the real-time power consumption information corresponding to the target gesture command, such as the instantaneous power consumption information of the target gesture command since its inception.

[0067] The information processing method provided in the embodiments of this application will be illustrated with examples below.

[0068] To address the aforementioned technical issues, and considering that current architectures mostly rely on single-layer matching or simple priority tables to drive scheduling, failing to establish a closed-loop mechanism spanning perception, analysis, deduction, and execution, and lacking quantification and feedback on the potential impact of subsequent instructions, leading to difficulties in timely exposure of resource contention, hidden deadlocks, or device overload; furthermore, since the maintenance of identity consistency for multi-view skeletal sequences often employs frame-by-frame matching or distance heuristics, identity drift accumulates once occlusion, rapid crossover, or background interference occurs, thus affecting the accuracy of conflict detection and the reliability of scheduling results; based on this, the current solution focuses on real-time determination and secure scheduling of concurrent gesture conflicts. The current approach remains passive, lacking a systematic approach to dynamically perceive business risks and adaptively adjust the execution order. This application provides an information processing method, specifically an artificial intelligence-based multi-gesture concurrent management and digital simulation real-time operation and maintenance optimization method. This method can support real-time judgment of gesture command conflicts based on business semantics and device risks in a large-screen interactive environment with multiple perspectives and multiple users, while ensuring gesture recognition accuracy and identity consistency. Based on this, it can dynamically prioritize and schedule concurrent commands, thereby maintaining system stability and business continuity even under resource constraints or sudden load changes. For example, a macroscopic comparison of the differences between this method and the current solution can be seen in Figure 2. The current solution has not yet solved the problems of resource conflict prediction, digital twin mapping and hierarchical rate limiting scheduling in multi-gesture high-concurrency scenarios. In this regard, this application provides the above-mentioned method, which can be implemented as a digital twin-risk coupled scheduling scheme for multi-gesture concurrent interaction. It involves: constructing a unified time base of "gesture-clock grid", real-time intention map-resource touchpoint matrix linkage deduction, dynamic quantification of risk by conflict vortex engine, and supplemented by deep verification of concurrent conflict feedback and hierarchical rate limiting scheduling closed loop, so as to realize instantaneous risk identification, power consumption-latency collaborative control and secure interpretable execution of the entire process of multi-user multi-device interaction, significantly reducing interaction latency, power consumption peak and abnormal interruption rate, and obtaining irreplaceable technical benefits.

[0069] As shown in Figure 3, the specific implementation of this application includes the following steps: S1, geometric calibration and clock synchronization are performed on multiple depth cameras distributed around the large screen, the frame-level point clouds collected by each camera are unified into the same three-dimensional coordinate system (i.e., global coordinates are obtained), and screen edge self-inspection, background culling and skin color filtering are performed in the point cloud (to obtain a clean point cloud set), (for the clean point cloud set) multi-view skeleton information is fused (to generate unified joint points) and hand instances are assigned according to motion consistency, and a basic gesture sequence with timestamps is generated by segmenting continuous frames based on velocity curves; this step can correspond to the above-mentioned obtaining the gesture sequence of at least one user based on point cloud data from at least two perspectives; the user corresponds to the point cloud data; wherein "assigning hand instances according to motion consistency" means aggregating the joint points of the same hand, assigning and stably updating a unique ID for each hand, and occlusion or intersection scenes can be quickly recovered through topological constraints and momentum compensation.

[0070] The continuous frames refer to the continuous frames of each user's hand obtained after fusing multi-view skeletal information.

[0071] S1 enables cross-camera spatiotemporal registration and skeleton fusion, specifically including: after completing multi-camera calibration and time synchronization, unifying point clouds from different viewpoints to the same coordinate system, automatically removing background and jitter, fusing skeletons and assigning a unique identity to each hand, resulting in a clean and non-overlapping gesture sequence. The gesture sequence can be understood as a unified coordinate frame, which is then used to enter S2.

[0072] S2. Write the basic gesture sequence into a pre-defined circular buffer in chronological order. Construct a gesture concurrency matrix using a sliding window. (The window (i.e., the gesture concurrency matrix)...) Each gesture within the sequence is mapped to a semantic instruction and then spliced ​​with skeletal features. A mutual exclusion score is then calculated, and a potential conflict index is output based on a warning threshold. This step corresponds to the above-mentioned acquisition of concurrent gesture information based on the gesture sequence. The mutual exclusion score is a normalized quantification of the conflict intensity between any two gestures under the current spatiotemporal and resource semantics, with a numerical range of [0,1]. To calculate this score, the multidimensional embedding vectors of the two gestures (such as skeletal temporal sequence, spatial pose, business semantics, etc.) are combined with the risk weights mapped to the transparent factor table. The score is then modulated using symmetric weights of the security-sensitive dimension, semantic differences, and real-time risk terms. A higher score indicates a higher probability of mutual exclusion. Elements exceeding the warning threshold are marked as potential conflict pairs and enter into subsequent priority adjudication and locking delay strategies, as detailed below.

[0073] S2 can perform concurrent window analysis and feature embedding, specifically including: writing the sequence into a circular buffer, constructing a concurrent relationship matrix within a sliding window and embedding temporal, pose, and business semantic features into a unified vector, thereby generating a dynamic conflict heat map (or as described in Figure 3: constructing a concurrent relationship matrix within a sliding window and generating a dynamic conflict heat map, where temporal, pose, and business semantic features are simultaneously embedded into a unified vector).

[0074] Based on the sliding window write sequence, it can be determined whether the buffer triggers the overwrite strategy. When chasing, the old data needs to be overwritten to maintain continuous access. If so, the overwrite can be performed; if not, that is, the window is stable, it can enter S3.

[0075] S3. Construct an instant decision pool by aggregating the potential conflict indexes at a preset period (e.g., a fixed sliding window period of 200ms). (For each instruction in the pool) query real-time factors such as device level, load, and alarms to calculate a composite priority coefficient (which can correspond to the priority factor information of each gesture instruction corresponding to the concurrent gesture information mentioned above, and obtain the priority coefficient of each gesture instruction). Generate an instruction list arranged in descending order of priority coefficients. Apply a locking delay to instructions with mutual exclusion relationships according to the difference threshold (increase the delay for instructions with low priority) and output a scheduling packet containing the list and locking parameters.

[0076] S3 can achieve real-time adjudication and priority reordering. Specifically, it can include: gathering potential conflicts every 200 milliseconds, loading a six-dimensional real-time factor and calculating a composite priority coefficient, applying a locking delay to low-scoring instructions according to the score difference threshold, forming a dynamic priority linked list; executing the linked list output and determining whether the heat coefficient has surged. If it has surged, it can immediately backtrack and reorder. If not, that is, if the priority is determined, it can proceed to S4.

[0077] Regarding the "surge in popularity can be traced back and reordered (i.e., determine if the popularity coefficient has surged, and if so, immediately trace back and reorder)": This action corresponds to the immediate adjudication and priority reordering process; for example, after the real-time update of the composite priority coefficient and window popularity coefficient is completed in S33-S34, a priority linked list is generated in S35; if a sudden increase in window popularity is detected, the reordering branch from S35 to S31 is triggered, and the paths converge and are reordered. This action can support the debouncing and security assurance of achieving a closed-loop adjudication system.

[0078] S4. Based on the scheduling package, remap the multi-gesture trajectory to a unified time base to form a two-dimensional gesture-clock grid (and obtain the intention micro-unit), disassemble the intention micro-unit and generate a real-time intention map, map the intention map to the resource touchpoint three-dimensional matrix and then perform conflict eddy current deduction (to obtain the eddy core index), (based on the eddy core index) combine the digital twin curve to calculate the hierarchical risk score and encapsulate the risk scheduling package.

[0079] S4 enables digital twin risk simulation, specifically including: mapping rearranged gestures to a resource touchpoint matrix, solving for the conflict vortex center and driving the linkage of power consumption, temperature rise, and latency curves, and outputting hierarchical risk scores and hot zone coordinates. Risk assessment is then performed: determining whether the risk score exceeds a threshold. If it does, rollback or delay of unlocking (i.e., delayed execution) can be implemented, proceeding to S5 for rollback or delay processing; if the threshold is not exceeded, proceeding to S6.

[0080] Regarding "rollback or delayed unlocking after risk assessment exceeds threshold (i.e., risk assessment: determining whether the risk score exceeds the threshold; if it does, rollback or delayed unlocking (i.e., delayed execution))": This description corresponds to node S46 and its subsequent branches. Specifically, after obtaining the vortex core index and predictions in S41-S45, the hierarchical risk score is calculated in S46. If this risk score exceeds the threshold, it enters S5 for depth verification, specifically determined by the coverage rate in S54. If the coverage rate is high, it enters S55 to generate the rollback script; if the coverage rate is low, it enters S56 to solve for the minimum delay window and unlocking conditions. If the risk score does not exceed the threshold, it can directly enter S6 for execution. Related parameters are encapsulated into a risk scheduling package in S47 and used at the execution end.

[0081] S5. Perform event shaping and multi-source comparison according to the risk scheduling package, reconstruct the concurrent contact topology and calculate the conflict impact coverage under high sampling rate, generate rollback scripts for branches that need to be rolled back, solve the minimum delay window for branches with solvable delay and write it into the delay execution script, and update the conflict index at the same time.

[0082] S5 can perform replay and script generation, specifically including: replaying the original sensor stream for high-risk or unknown conflicts, finely quantifying coverage, automatically generating rollback scripts and delay unlocking parameters, and writing them to the conflict index. Based on the obtained scripts and / or parameters, proceed to S6.

[0083] S6. Receive the security token and priority identifier, write the gesture command into the hierarchical buffer, extract the key command according to the dependency chain and grant the preemption right, combine the real-time resource snapshot to implement hierarchical rate limiting for each priority queue, align the execution thread of the executable command to trigger, and correct the threshold vector after obtaining the execution feedback and send back the abnormal command for verification.

[0084] S6 enables tiered rate limiting and parallel execution, specifically including: freezing, delaying, or immediately executing the instruction queue based on dependency chains, priorities, and device concurrency limits; dynamically adjusting thresholds; and triggering a deep validation loop upon anomaly feedback. S6 can then perform threshold correction and heat feedback, which will then affect S2.

[0085] The following provides specific examples of each of the above steps.

[0086] Step S1 may include the following sub-steps: S11, Assuming there are 3 depth cameras, including , and ,Will Fixed in the center of the front of the large screen, and Symmetrically mounted on both sides of the large screen. Intrinsic parameter vectors of each depth camera are obtained based on the calibration board. With distortion vector Then calculate the extrinsic parameter matrix. With translation vector This allows for the geometric calibration of the camera.

[0087] The origin of the world coordinate system is set at the geometric center of the screen. : Horizontal to the right, Vertically upwards, The coordinate system points to the operator along the screen normal, and all subsequent point clouds and skeletons are unified to this coordinate system. Fixed to Forward axis; Symmetrically placed along the left and right center lines of the screen, equidistant from the screen surface. Optical axis convergence The camera's final posture is automatically fine-tuned during the calibration process.

[0088] Indicates from the camera coordinate system To the world coordinate system The extrinsic parameters (based on the screen). The calibration board is used to determine the intrinsic parameters. It also provides standard checkerboard / Charuco board and other calibration carriers for the initial values ​​of external parameters.

[0089] To unify signals Triggering times of each depth camera frame Align it to synchronize it with the master clock. Obtain a unified timestamp This allows for clock synchronization of the cameras.

[0090] Local points acquired by any depth camera Perform a joint space-time transformation to obtain global coordinates. : ;in, for The optical center coordinates; This is the velocity vector at that moment, i.e., the instantaneous velocity of the observed bone / joint, used for first-order constant velocity compensation for timestamp deviation.

[0091] This ensures that the skeletal point clouds from different perspectives and at different sampling times all fall into a unified three-dimensional coordinate system.

[0092] S12. Self-inspection and background removal of the ROI (Region of Interest) at the large screen edge: Extracting edge lines based on the brightness gradient of the four edges of the large screen to construct the screen boundary polygon. Based on depth threshold Combine screen normal vector elimination The point cloud behind it; then a skin color probability model was used. Only retain those that meet the requirements. The point clouds of the upper limbs and hands were used to obtain pure point cloud sets. This reduces interference from bystanders and the background on subsequent identification. In this context, t refers to the sampling time under a unified master clock (world system time index after aligning the timestamps of multiple cameras).

[0093] Here " "Point cloud behind" refers to the polygons at the screen boundary. The fitted screen plane is used as a reference, and the point cloud located in the anti-normal half space is divided into front and back half spaces according to its normal direction.

[0094] S13, Multi-view skeleton fusion and hand instance allocation, targeting Each joint Based on camera confidence Perform weighted least squares fusion to generate unified key points. Based on the connectivity of joint groups and motion consistency, a unique identifier is assigned to each hand using the maximum matching algorithm. To avoid identity confusion in multi-user concurrent scenarios, generating unified keypoints can include: merging corresponding keypoints from multiple cameras into one based on weights; for example, mapping keypoints with the same name from each camera to the world system first, and then performing weighted least squares fusion to obtain a unified keypoint.

[0095] S14, in each Establish an event buffer under your name Calculation speed for consecutive frames With acceleration :like And the Euclidean distance between two adjacent frames is less than If the interval between two clicks is less than 10, it is considered a jittery frame and is discarded; And trajectory overlap If so, the repeated clicks are merged. The resulting smooth trajectory is then obtained. Simultaneously update The latest frame index in the database.

[0096] Step S14 processes the temporal trajectory (i.e., continuous frames) of the "uniform skeletal joints" for each hand (HID). Specifically, it calculates the velocity of continuous frames using point cloud data of representative joints of the hand (such as the palm or wrist, and if necessary, weighted key points such as fingertips as representative joints) in the world coordinate system. With acceleration Based on this, jittery frames are removed and duplicate clicks are merged to obtain a smooth trajectory and update the event cache index. The original point cloud (i.e., the data obtained in S12) is only used for skeleton extraction and multi-view fusion in the previous step S13 and is not directly used for velocity and acceleration calculation in this step.

[0097] S11~S14 above correspond to the above-mentioned fusion and data filtering of point cloud data from at least two perspectives to obtain the gesture trajectory of at least one user.

[0098] S15, Real-time Statistics Recently Average frame rate Then dynamically adjust the current window length according to the following formula. : In the formula, As the baseline window length, For speed sensitivity coefficient, For the first One hand at a time The segmentation window length (unit: frames); The velocity statistics half-window (based on the aforementioned master clock, looking back from the current time t to the nearest L frames); For frames The instantaneous velocity vector, For its model. Characterizes the intensity of local motion; stronger motion indicates stronger local motion. Shorten, or stretch; calculate Then it can be rounded up and cut to the nearest whole number. To ensure stability, among which The minimum discernible duration and sampling frequency can be set to avoid oversegmentation due to an excessively small window. The settings can be based on the high quantile of the gesture duration distribution and the device latency budget to avoid undersegmentation and response lag; then, for and EWMA can be used for small-step adaptive adjustments by combining window heat and missegmentation rate.

[0099] In this scheme, the skeletal trajectory of the most recent L frames can be obtained by looking back from the current time t based on the aforementioned master clock. This trajectory is then used to calculate the average speed and local motion intensity of the i-th hand at time t. When a small number of frames are lost, they can be aligned by time and interpolated to ensure statistical consistency.

[0100] The above information was obtained. The adaptive strategy can support extending the observation time in slow-motion scenarios and shortening the window length in fast-motion scenarios, ensuring that neither over-segmentation nor missing key actions is achieved.

[0101] The above S15 can correspond to determining the segmentation window length corresponding to each gesture trajectory based on the average speed of at least two frames of images in each gesture trajectory.

[0102] S16, in Internal calculation of the first derivative sign change of the velocity curve (corresponding to the i-th hand or the i-th user), and calibration of peak-valley pairs. If there are no less than [amount] before the peak and after the valley. A still frame is then... Divide the boundary (which can correspond to the above-mentioned segmentation boundary) into sub-segments. Record the start and end times. Candidate gesture segments are formed; corresponding to the above, based on the segmentation window length corresponding to each gesture trajectory, the corresponding gesture trajectory is segmented to obtain candidate gesture segments corresponding to each user; wherein, the step of segmenting the corresponding gesture trajectory based on the segmentation window length corresponding to each gesture trajectory to obtain candidate gesture segments corresponding to each user includes: obtaining the first derivative sign change information of the velocity curve corresponding to the first gesture trajectory based on the first segmentation window length corresponding to the first user's first gesture trajectory; determining the segmentation boundary based on the first derivative sign change information; and segmenting the first gesture trajectory based on the segmentation boundary to obtain candidate gesture segments corresponding to the first user.

[0103] In S16, the representative trajectory velocity is determined within the adaptive sliding window Wi(t) of the i-th hand. Perform first derivative sign change detection to locate local peak-valley pairs. Only when there are no less than [a certain number of] peaks and valleys respectively Candidate gesture segments are generated only when the frame is a still frame and the window length obtained in S15 is compatible with that frame. Multiple peak-valley pairs may appear within a window and form multiple segments; if the sign change originates from micro-jitter or does not meet the stillness constraint, segmentation may not be triggered.

[0104] S17, For the same Adjacent segments with no temporal overlap If its termination state and the starting state of the next sub-segment In gesture semantic map If there are connectable edges, then a splicing and merging operation is performed; this can correspond to obtaining the gesture sequence of at least one user based on the candidate gesture fragments corresponding to each user.

[0105] In particular, after S16 completes the initial segmentation based on the first derivative of velocity, a round of "segment-level calibration and connectivity determination" can be performed first, such as: identifying the state of the first and last frames of each sub-segment and giving the termination state. and the starting state of the next sub-segment The confidence levels of these two states are then determined. Next, the boundaries of each sub-segment are fine-tuned: the boundaries are snapped to local extreme frames and short-term static or noisy segments are removed. The identity continuity of the same HID is verified under a unified time base. Subsequently, based on the verified and fine-tuned sub-segments, indices such as time gap, spatial distance, directional continuity, and / or velocity continuity are calculated for adjacent sub-segments. Threshold constraints (such as gap threshold, displacement threshold (i.e., threshold for spatial distance), direction cosine threshold for directional continuity, and velocity ratio threshold for velocity continuity) and semantic consistency constraints (such as the existence of connectable edges on the semantic graph and a satisfactory confidence level) are set. Only sub-segments that simultaneously satisfy these constraints undergo semantic splicing in S17 to repair over-segmentation caused by micro-pauses, occlusion, or slight jitter. Sub-segments that do not meet the conditions remain independent and proceed to subsequent processes.

[0106] Based on the above, all confirmed fragments can form a neat, orderly, and non-overlapping basic gesture sequence. It can be accompanied by the corresponding time index. Output to the next conflict analysis step.

[0107] Among them, gesture semantic graph This can be obtained through offline annotation and rule fusion. Specifically, features f are extracted from the first and last frames of each segmented sub-segment, and a state classifier is used to obtain start / end labels (i.e., termination states). and the starting state of the next sub-segment Subsequently, state transition counts were statistically analyzed using an offline labeled corpus, and Laplace smoothing was used to estimate the transition probability based on these counts. During graph construction, edges (u,v) were added only when four conditions were simultaneously met: transition probability ≥ corresponding threshold θ, time interval between adjacent segments ≤ corresponding threshold τ, cosine value of direction change ≥ corresponding threshold γ, and operational semantic constraints were met (e.g., consistency between permissions and devices, no crossing of isolation domains, etc.). These thresholds can be adaptively updated subsequently.

[0108] The above involves: introducing a joint space-time transformation and first-order constant velocity compensation for time base correction in S11. Instead of relying solely on buffer alignment, the instantaneous velocity of each bone and / or joint is estimated in the camera frame and then transformed to the world frame. This velocity term is used to compensate for timestamp deviations, achieving consistency of landing points across cameras and time periods.

[0109] In S15-S16, the boundary is determined adaptively by the change in the sign of the first derivative of the velocity curve and the recent average velocity-driven window, rather than by a fixed window and static threshold. The boundary is synchronized with the extreme value frame and dynamically expands and contracts with the motion intensity. Combined with the jitter frame removal and duplicate click merging in S14, the basic gesture sequence is clean and non-overlapping. This is a combination of derivative monitoring, adaptive window length, and semantic splicing, which can reduce identity drift and over-segmentation or under-segmentation.

[0110] II. Step S2 may include the following sub-steps: S21, Receive the basic gesture sequence output in step S1 and corresponding timestamps , and according to Ascending write capacity is Circular buffer .

[0111] During the writing process, read and write pointers can be maintained. ,when catch up A coverage strategy can be activated to ensure continuous input and support random access at any time.

[0112] Here, "overwrite strategy" refers to the write handling when the circular buffer is full. The system maintains read pointers. With write pointer The next write will result in catch up At that time, the judgment Full. To ensure real-time performance, the default sliding strategy of "new data first" can be used: first... The oldest unconsumed candidate fragments are discarded to free up space before new fragments are written; discarding is recorded in the counter and log. Fragments that are locked or have entered the execution chain are protected and will not be overwritten. If required by business needs, blocking and / or backpressure modes can be switched to prevent overwriting of older data, but the default configuration overwrites the oldest data when the capacity limit is reached.

[0113] S22, with step size exist Slide-up window For each gesture instance within a window Constructing the concurrency relationship matrix Window Index and gesture index The correspondence satisfies the following equation:

[0114] in, , The start time of the two gestures; symbol The logical condition is that the two gestures simultaneously fall into the union of the current window and the previous window at the start time (which can correspond to the first condition mentioned above).

[0115] Constructing a matrix based on the above formula allows us to focus not only on concurrent windows but also on cross-triggered windows of adjacent windows, ensuring that edge concurrency is not overlooked.

[0116] S21~S22 above can correspond to constructing a gesture concurrency relationship matrix based on the first condition and the gesture sequence; the first condition is determined based on two adjacent sliding windows.

[0117] S23, Regarding the concurrency relationship matrix Every gesture Call the operation and maintenance command library Search its functional tags Related equipment and business level This enables the mapping from purely visual data to semantic units (i.e., mapping gestures to semantic commands); and simultaneously... Feedback written to buffer This provides semantic support for deep feature fusion.

[0118] S24. For each gesture in the matrix , skeletal trajectory sequence Duration With the tag triples obtained in step S23 After splicing (to obtain skeletal features), the data is input into the scene-based embedding. The embedded part can extract temporal features, spatial pose features, and business semantic features in parallel through a multi-branch structure, and at the end, it shares an attention pooling layer to normalize the dimension, outputting a multi-dimensional feature vector with a uniform scale. .

[0119] Among them, the skeletal trajectory sequence This refers to the gesture The temporal trajectory of skeletal points under a unified time base after multi-view fusion. The duration is... The start and end time difference of the gesture segment is derived from the boundary determined by adaptive segmentation and aligned within the sliding window. If the segment spans multiple windows, the duration is calculated uniformly based on the union of adjacent windows. The duration is based on the gesture instance and is not the cumulative session duration.

[0120] Steps S23 and S24 above correspond to embedding the corresponding skeletal trajectory sequence, duration, and semantic information into a multi-dimensional feature vector for each element in the gesture concurrency relationship matrix. The semantic information includes at least one of functional labels, associated devices, and service levels. The label triples obtained in step S23 correspond to the aforementioned semantic information.

[0121] S25. Mutual Exclusion Score Calculation and Dynamic Conflict Heatmap Generation: For any pair of mutually exclusive scores in the matrix... Retrieve its feature vector and business risk coefficient Based on this, the mutual exclusion score is calculated using the following formula. (This corresponds to obtaining the mutual exclusion score between any two elements in the gesture concurrency relationship matrix based on the multidimensional feature vector, as described above), and then filling it into the heatmap with the same number. ; In the formula, It is a symmetric weight matrix used to emphasize the security sensitivity of a specific dimension; This is a semantic difference balancing coefficient. The combined metric considers business risk weights, posture similarity, and semantic differences simultaneously, thus obtaining a conflict intensity that better reflects the operational scenario.

[0122] Among them, the heatmap with the same sequence number refers to the conflict heatmap that corresponds one-to-one with the current sliding window. Within the current sliding window, the system fills in the mutual exclusion scores of any two gestures according to the row and column positions of the concurrency relationship matrix to form a two-dimensional intensity distribution; it only reflects the strength of potential conflicts within the window and is used as input for threshold filtering, potential conflict index generation, and subsequent priority adjudication, without performing cross-window accumulation.

[0123] Among them, the risk coefficient is It can be obtained by linear mapping from the transparency factor table, and will not be elaborated here.

[0124] S26, Traversal ,Will Above the warning threshold Gesture combinations Denote them as potential conflict pairs, and based on the suprathreshold interval Subcategories Generate (potentially) conflict indexes This can correspond to the element corresponding to the mutual exclusion score above the first threshold, from which concurrent gesture information is obtained. The conflict index can correspond to the aforementioned concurrent gesture information.

[0125] Regarding "based on suprathreshold intervals" Subcategories For example, within the same sliding window, if the mutual exclusion score of a gesture pair exceeds the warning threshold, it is categorized into general conflict, severe conflict, and high-risk conflict based on the score above the threshold. Specifically: a mutual exclusion score just above the threshold can be recorded as a general conflict; a mutual exclusion score significantly higher than the threshold and falling on the same device resource can be recorded as a severe conflict; a mutual exclusion score much higher than the threshold, accompanied by high alarm density and high maintenance window occupancy, can be recorded as a high-risk conflict. The subsequent system can generate a potential conflict index based on this, recording the gesture pair, category, and timestamp for subsequent priority determination and locking latency; the boundary value (i.e., the threshold of the mutual exclusion score) can be derived from historical logs and can be fine-tuned online.

[0126] S27, put Detailed record of corresponding gestures Encapsulated as a message packet and according to timestamp The data is pushed sequentially to the subsequent priority determination process.

[0127] The message packets can use a unified data structure. This facilitates the rapid execution of locking, reordering, or delay strategies by upper-layer modules, enabling secure scheduling of concurrent multi-gesture operations.

[0128] The above involves: explicitly constructing a concurrency relationship matrix using a circular buffer and sliding window in S22, and uniformly incorporating window overlap and adjacent window cross triggering into the judgment to solve the problem of missed detection at window edges.

[0129] In S23~S24, each gesture is mapped to a semantic instruction and embedded in a contextualized manner. The temporal branch (1D-Conv / Transformer), spatial pose branch (ST-GCN), and business semantic branch are jointly encoded to output a feature vector of a uniform scale, which serves as a common representation for cross-window and cross-device comparison.

[0130] In S25, a six-dimensional factor combination of a symmetric weight matrix, semantic difference balance coefficient, and transparent factor table can be used to obtain a mutual exclusion score and generate a dynamic conflict heat map online. It is not a static rule, but a synthetic metric of posture similarity, semantic difference, and real-time risk, which can make the same posture present different mutual exclusion strengths under different device levels, alarm densities, and role domains.

[0131] The mutual exclusion score is calculated primarily using an embedding vector combined with a symmetric weight matrix and a semantic difference balance coefficient to characterize gesture similarity, semantic differences, and conflict intensity, without directly affecting priority. The six-dimensional factor in the aforementioned transparent factor table serves only as a source of risk bias, mapping it to business risk coefficient pairs to modulate the mutual exclusion score. The core purpose of this six-dimensional factor is to calculate a composite priority coefficient based on the six real-time factors in subsequent immediate adjudication and priority reordering steps. The six-dimensional factor's roles in these two areas are respectively related to popularity assessment and priority generation.

[0132] III. As shown in Figure 4 (Flowchart of Immediate Decision-Making and Priority Reordering), step S3 may include the following sub-steps: S31, Immediate Decision-Making Pool Construction, which may include: aggregating potential conflict instructions with a fixed 200ms sliding window, writing them into the decision-making pool according to their arrival time and recording their sequence numbers, and establishing a unified time base; specifically, for example: the controller uses a fixed 200ms sliding window period to process the potential conflict index generated in step S2. Converge and form a window For falling into the window Each gesture command Write to the instant adjudication pool in sequence. Simultaneously record gesture commands Arrival sequence number With arrival time The adjudication pool The subscripts are used only for indexing and alignment and have no additional mathematical meaning. The window subscript represents the current sliding window number, the adjudication pool subscript corresponds one-to-one with the window, and the instruction subscript represents the sequential number of the instruction arriving within the window; the arrival time is a timestamp under a unified time base.

[0133] The adjudication pool provides a unified time base and random access capability for subsequent factor queries and priority calculations.

[0134] Here, aggregation refers to indexing all potential conflicts generated in step S2 within the current cycle at a fixed granularity. Merged into the same time window All triggered events whose timestamps fall within this range are considered concurrent competitors and are uniformly written into the instant adjudication pool. Its purpose is to establish a unified time base, cover cross-window boundary events, and provide batched, randomly accessible input for subsequent concurrent scoring and priority reordering.

[0135] S32. Based on S31, perform real-time factor retrieval and populate the transparent factor table. Specifically, this may include: synchronously retrieving six real-time factors for each instruction in the adjudication pool, namely, device level, current load, alarm level, maintenance window occupancy rate, operator role, and geographical isolation domain, and normalizing and writing them into the factor table.

[0136] For example: targeting Each instruction within Simultaneously retrieve six real-time factors closely coupled with the scene: device level Current load Alarm Level Maintenance window occupancy rate Operator role Geographical isolation zone .

[0137] All factors are normalized to [0,1] using interval mapping and then written into the transparent factor table. ,in This indicates the number of instructions within the window. The execution weight is then recorded and the process proceeds to S33.

[0138] Each gesture command, after semantic mapping, is assigned to a specific device or business resource (such as the target device). Therefore, the six real-time factors mentioned above can be matched one-to-one with the commands and synchronously retrieved and written into the transparent factor table along with the commands. Device level can refer to the importance and risk classification of the target device; current load can refer to the real-time load ratio of the target device; alarm level can refer to the current alarm level of the system or business corresponding to the command, which can adopt the current alarm system's fatal, severe, and general classifications; operation and maintenance window occupancy rate can refer to the proportion of tasks already occupied by the target device within the current operation and maintenance window; operator role can refer to the user's permission role initiating the command; geographical isolation domain can refer to whether the command crosses a preset network domain or geographical domain. These factors can be used to calculate composite priority coefficients and, together with mutual exclusion scores, drive immediate decision-making and locking latency.

[0139] The system corresponding to the instruction refers to the operation and maintenance monitoring system or network system that carries the target service and equipment. When calculating the transparency factor table, the actual value can be "the current alarm level of the service and / or equipment corresponding to the gesture instruction in the operation and maintenance monitoring system".

[0140] The “percentage of tasks already occupied by the target device in the current maintenance window” refers to the proportion of maintenance tasks that the target device has been scheduled or is currently executing in the current maintenance window to the total available maintenance capacity of the maintenance window. This proportion can be written into the transparent factor table as the maintenance window occupancy rate to reflect the busyness of the target device in this window in the calculation of the composite priority coefficient.

[0141] S33, Basic-Real-Time Composite Priority Coefficient Calculation, may include: loading the business level tree weight vector, weighting and accumulating it with the transparent factor table, and introducing arrival delay penalty and scene heat modulation to output the real-time priority coefficient.

[0142] For example: obtaining the public weight vector from the business hierarchy tree. And load it into the stacking factor chain. Then... The system performs weighted summation, and incorporates arrival delay penalties, service level compensation, and scenario heat modulation to directly output the real-time priority coefficient. : ;in, This refers to the window heat coefficient; The value represents the business level to which the instruction belongs; This is the window start time; It is an adjustable scaling factor; The alarm category is a unique hot vector; It is a symmetric risk interconnection matrix.

[0143] The window heat coefficient is a single value at the window level. It is generated by the sliding window and shared by all gestures in the adjudication pool within that window. It can be used to uniformly modulate the composite priority coefficient (i.e., the aforementioned real-time priority coefficient) in the same batch of adjudications. ).

[0144] The aforementioned delay penalty corresponds to the difference between the window start time and the instruction arrival time in the formula (i.e., The term with as the independent variable is used to perform smooth weighting on instructions with longer waiting times.

[0145] In this formula, 'j' represents the sequence number, i.e., the j-th gesture command within the window. The master clock is the starting point of the unified time base; both the window's start and arrival times are aligned with the master clock.

[0146] In this scheme, the following can be considered: the window start time represents the start time of the sliding window under the master clock, the arrival time is the timestamp of the instruction being written to the adjudication pool (i.e., the writing time), the waiting time is the difference between these two times, the window heat coefficient only corresponds to the current window, and the adjudication pool index is used for in-window retrieval.

[0147] Based on the above formula, static weights, arrival timeliness, alarm coupling degree and window heat are integrated to obtain a scene-adaptive priority scale.

[0148] Among them, the stacking factor chain refers to the current window. The six types of factor responses, normalized in a fixed order, are denoted as follows: Loading weights into the stacked factor chain can be understood as assigning weights to each of the six factors.

[0149] The amplitude coefficient of the arrival delay term is determined. The priority is modulated by a logic function. Its initial value can be determined by a grid search of historical data, and during runtime, it can be adaptively adjusted in small steps based on the overall waiting time distribution, balancing responsiveness and stability.

[0150] Among them, item Based on waiting time The independent variable is smoothed and modulated, and the longer the wait, the greater the impact, which can balance real-time performance and fairness.

[0151] S33 can be used to obtain the priority coefficient of each gesture instruction based on the window heat coefficient and the priority factor information of each gesture instruction corresponding to the concurrent gesture information.

[0152] S34. Real-time updates of the heat index can include: online correction of scene heat index (i.e., window heat index) based on alarm density and traffic fluctuation rate within the window to ensure that the priority index adapts to changes in load.

[0153] For example: based on the alarm density within the window With flow volatility Calculate the window heat coefficient in the real-time heat channel. This corresponds to obtaining the window heat coefficient based on the alarm density and traffic fluctuation rate within the first time period. For example, the specific logic is: first... and Perform linear normalization, then use obtain in this way This enables all instructions for handling sudden high-risk windows. They rose in sync.

[0154] Wherein, α1 and β1 are channel gain coefficients, which can measure alarm density respectively. With flow volatility The linear contribution to window heat; α1 and β1 can be set empirically and are not limited here.

[0155] S35, Priority List Generation, may include: sorting all instructions in descending order according to real-time priority coefficients, generating a list containing factor contribution rates for subsequent scheduling reference. The "Priority Correction" in the diagram may be performed before "Priority List Generation"; or it may be performed after "Real-time Update of Popularity Coefficient," i.e., entering "Priority List Generation," without limitation here.

[0156] For example: based on right Sort all instructions in descending order and generate a priority linked list. The linked list simultaneously records the factor contribution rate vector for each instruction. This ensures that subsequent audits can trace the impact percentage of each factor.

[0157] S36, Differential Threshold Control and Delay Locking, may include: traversing the linked list to calculate the priority difference of mutual exclusion pairs; if the difference is insufficient, applying microsecond-level delay locking debouncing; if the difference is significant, maintaining the current order and marking the static lock duration. Subsequent encapsulation and output can be entered into S37.

[0158] For example: traversing from the head to the tail of the linked list, for instruction pairs that have a mutual exclusion relationship. Calculate the priority coefficient difference , Indication of instructions Priority coefficient, Indication of instructions Priority coefficient. If Exceeding the threshold For low-scoring instructions Apply lock delay The locking strategy employs a device level-differential joint incremental model, i.e. The following formula can be used to obtain it: ;in, This is a flag indicating the mutual exclusion strength of instructions. For window alarm density, , For the step coefficient group, It is a step function.

[0159] The priority list refers to the sequential list formed by sorting the instructions in the adjudication pool from high to low according to the real-time priority coefficient within the current sliding window; each record also retains the factor contribution in the transparent factor table for subsequent scheduling and auditing reference.

[0160] in, For instructions The corresponding device grade factor is a normalized value obtained by mapping the device grade from the transparent factor table. It is used to directly incorporate the device importance into the calculation of the lock latency amplitude. It is not the text label of the device grade itself, but can be understood as the grade weight / scale used for calculation.

[0161] in, The parameters belonging to the lock-in latency model can be preset with initial values: during the offline phase, they are calibrated based on historical concurrency logs, false trigger rate and waiting time distribution, etc. (for example, robust initial values ​​can be obtained by grid search or Bayesian optimization, or they can be directly set).

[0162] Regarding "instruction pairs that have a mutual exclusion relationship" "This can be understood as concurrent instructions that have a mutual exclusion relationship."

[0163] Regarding delay locking and timing: When the priority difference exceeds the corresponding preset threshold, delay locking can be applied to the low priority instruction; the delay is calculated from the decision time of the current window, with the master clock recording the waiting time, and is added according to the smooth increment given by the formula, and is executed again when the next scheduling or resource becomes available.

[0164] Among them, the window heat coefficient is a single value at the window level, which can be calculated online from alarm density and traffic fluctuation rate. It can reflect the overall risk and congestion level of the corresponding window and can be used to adjust the differential threshold control and delay intensity.

[0165] Step function and threshold: The step function is used to activate locking when the difference exceeds a threshold, and does not trigger it when the difference is below the threshold; the threshold adopts the above-mentioned warning threshold setting, which is derived from existing calibration and can be fine-tuned online. The "warning threshold" refers to the threshold used to determine mutually exclusive pairs.

[0166] Based on the above formula, the locking latency of high-level equipment or large difference scenarios can be guaranteed to increase in stages, avoiding the dilution of high-risk commands by low-priority instructions.

[0167] Among them, threshold With step coefficient group Robust initial values ​​can be obtained offline using grid search and / or Bayesian optimization based on historical concurrency logs; after going live, adaptive updates can be performed using exponential sliding and small-step projection, incorporating feedback from conflict handling results, waiting time, and false positive rate. Where... Derived from device level and factor chain normalized components. The step function triggers only when the difference exceeds a threshold, locking the increment. As risk and score difference increase monotonically and remain under control, the safety redundancy of high-risk scenarios is enhanced while ensuring fairness.

[0168] S37. The output of the scheduling package may include: encapsulating the linked list and locking parameters into an interpretable scheduling package and writing it to the execution end; and simultaneously synchronizing the audit log and the model backtracking pool.

[0169] For example: linking lists Lock parameters With factor contribution rate Encapsulated as an interpretable scheduling package The data is then pushed to the execution end according to the time sequence. Simultaneously, it is written to the audit log and model backtesting pool, providing a complete factual basis for weight calibration and model iteration. Afterwards, the executable threshold and log feedback are applied to S31.

[0170] "Applying to S31" refers to adaptive parameter updates: based on thresholds and log feedback, the alert threshold and its tier boundaries are fine-tuned online, and the normalized interval of the transparency factor table and the baseline values ​​of the channel gain coefficient are updated using log statistics. Subsequently, in a new sliding window, step 31 completes the retrieval and writing of potential conflicts into the adjudication pool according to the updated thresholds and normalization rules, recording the sequence number and arrival time. Specifically, "fine-tuning the alert threshold and its tier boundaries online" refers to step... The warning thresholds and their grading boundaries used to determine potential conflicts; "updating the normalized interval of the transparency factor table" refers to the steps... The interval parameters used in the normalization process for the six types of real-time factors; the "channel gain coefficient" corresponds to the step In Two channel gain coefficients; while "completing potential conflict pull and writing to the adjudication pool" refers to the step at the start of a new round of sliding window. According to the , , The updated thresholds and normalization rules, which pull data from the potential conflict index obtained in the previous round and write it to the instant adjudication pool, can logically be attributed to... The processing procedure.

[0171] S36~S37 can correspond to the above-mentioned acquisition of the difference between the priority coefficients of two mutually exclusive gesture commands; setting a lock delay for the gesture command corresponding to the target priority coefficient; the lock delay is determined based on the difference corresponding to the target priority coefficient and the device level; the target priority coefficient refers to the lower priority coefficient among the two priority coefficients corresponding to the difference greater than the second threshold; and performing a gesture command response operation based on the priority coefficient and the lock delay.

[0172] The above involves: establishing an instant adjudication pool in S31, aggregating potential conflict indexes with a fixed granularity of 200ms, unifying the time base, supporting random access, and oriented towards concurrent batch adjudication.

[0173] Six real-time factors (device level, current load, alarm level, maintenance window occupancy, operator role, and geographical isolation domain) are introduced into the transparent factor table in S32-S34, and a basic-real-time composite priority coefficient is calculated, where the window heat coefficient is modulated in real time by alarm density and traffic fluctuation rate. This supports the implementation of interpretable and online adaptive priority scaling.

[0174] In S36, a locking delay is applied using a differential threshold control and a device level-differential joint incremental model. The parameters can be obtained with robust initial values ​​through offline grid or Bayesian optimization, and then calibrated in small steps online. Unlike general insertion delays or locking commands, the locking in this application is a quantitative strategy that increases in segments with differential, mutual exclusion strength, and alarm density, and retains a factor contribution rate for each command (S35) to form a traceable decision.

[0175] IV. As shown in Figure 5 (Flowchart of Digital Twin Risk Simulation), step S4 may include the following sub-steps: S41, Trajectory remapping to a unified time base, which may include: remapping multiple gesture trajectories to a unified time base according to the attitude flow and precise timestamps, forming a two-dimensional "gesture-clock grid", and solidifying the force vector, direction vector and concurrent rhythm. Subsequent execution intent decomposition triggers, proceeding to S42.

[0176] For example: reading the interpretable scheduling packet output by S3. ,according to Attitude flow and Precision timestamps will display multiple gesture trajectories. Remapping to a unified time base This forms a two-dimensional "gesture-clock grid". In grid element Force vector of internal curing at the same moment Direction vector With concurrency rhythm This provides a complete dynamic context for the intended deconstruction.

[0177] Among them, multi-gesture trajectory refers to the posture sampling stream corresponding to multiple gesture commands to be executed in priority order within the same sliding window. Under a unified time base, the continuous changes in the strength and direction of each trajectory over time are remapped and rasterized to serve as a complete dynamic context for subsequent intent decomposition.

[0178] Among them, the gesture-clock grid is used to move the window Concurrent trajectories within Unified to global time base The resulting two-dimensional array Horizontal axis The vertical axis represents the clock index at 240 Hz. For concurrent gestures and / or device identification; each cell can record the motion intensity, direction, and corresponding business event pointer (i.e., force vector, direction vector, and concurrent rhythm) at that moment, for semantic decomposition and interpretation.

[0179] Among them, in the grid cell It represents the intensity of movement and is derived from the weighted and normalized sum of skeletal velocity, acceleration, and angular velocity. The unit direction vector for motion or palm posture; It is a business event node / business event pointer, pointing to the standardized instruction triggered at that moment and its HID, type, and risk metadata.

[0180] S42, Intent micro-unit decomposition and real-time intent graph generation (which may correspond to the gesture command intent information mentioned above), may include: decomposing grid cells into non-overlapping intent micro-units based on features such as trajectory inflection points and force changes, then generating a real-time intent graph with spatial adjacency and temporal causality as edges, and attaching six-element metadata such as permission tokens and service interfaces to each node. Subsequent execution node mapping proceeds to S43.

[0181] For example: edge semantic node traversal Based on trajectory inflection points, intensity changes, and rearrangement time windows, continuous grid cells are decomposed into non-overlapping intention micro-units. Then, using spatial adjacency and temporal causality as edges, a depth-first chaining method is used to generate a real-time intent graph. This corresponds to obtaining gesture command intent information based on the aforementioned priority coefficient. In Represents a set of nodes. Represents the set of edges.

[0182] For each node Linking six metadata: permission token Service Interface Security Domain Beat markings Initiating Role and fault tolerance margin This allows the control plane to obtain a three-dimensional behavioral profile.

[0183] Among them, "guided depth-first chaining" refers to starting from the boundary node and linking continuously along the causal chain under the constraints of the two types of edges generated: spatial adjacency and temporal causality, until it is no longer scalable and then backtracking to continue, so as to form a real-time intent graph consistent with the arrival time, which is used to ensure that the decomposed intent micro-units are sequentially integrated within the same time window.

[0184] The "six metadata elements" are the operational information attached to each node, namely, permission token, service interface, security domain, node identifier, initiator role, and fault tolerance margin. The permission token is used for access verification and authorization control; the service interface is the entry point for calling the corresponding device or service; the security domain can inherit the geographical isolation domain and network isolation domain mentioned earlier, used for boundary constraints; the node identifier is used to align sequences and audit records under a unified time base; the initiator role corresponds to the operator role, used for policy and auditing; and the fault tolerance parameter is used for retries and fallbacks in the execution plane. Subsequently, during node mapping and execution, this information is directly used: first, permission and security domain verification is performed, then the call is triggered according to the service interface, while simultaneously recording the node identifier and initiator role and handling exceptions according to the fault tolerance margin, consistent with the application of the aforementioned scheduling package and priority coefficient.

[0185] Among these, the aforementioned trajectory inflection points, intensity changes, and rearrangement time windows can be based on the aligned skeletal trajectories. The determination process involves calculating the orientation angle, curvature, and velocity, all of which are time-varying trajectory features. Inflection points are determined by local peaks and inflection points of these features. Force changes are calculated as the intensity of force inflection points after robust suppression. The rearrangement time window takes the maximum value of multiple time windows, including the adaptive segmentation window and the delay window of the scheduling layer mentioned earlier. These three elements work together to segment continuous micro-units, dividing continuous lattices into non-overlapping intentional micro-units. Among these, "aligned skeletal trajectories" are... "This refers to the time-series trajectory of the skeletal joints of the same hand under a unified time base, obtained in the stage of fusing multi-view skeletal information to generate unified joints and allocating hand instances according to motion consistency after completing multi-camera clock synchronization and coordinate unification in step S1.

[0186] The above six metadata elements can be generated by the policy library and runtime context and attached in real time. (Permission token) It can be based on RBAC (role-based access control) or ABAC (attribute-based access control) for... Acquisition of roles and target resources; service interfaces The security domain can be determined by parsing the gesture to a specific API (Application Programming Interface) or Topic (message topic); The node identifier can be determined based on the resource's network and / or permissions. Can be uniquely assigned by the event sequencer; initiating role (i.e., operator role) can be determined based on the identity session; fault tolerance margin Risk can be combined The budget is estimated and normalized online with the SLA (Service Level Agreement) and can be used for subsequent adjudication and scheduling.

[0187] The permission token is generated by role-based access control or attribute-based access control; the fault tolerance margin can be normalized online in combination with risk and service level constraints.

[0188] S43. Resource touchpoint matrix construction may include: mapping each intent node to a three-dimensional index of CPU core, network segment, and power circuit. Based on this, convolution-time window fusion is used to predict future occupancy rates, and shared locks, exclusive locks, and energy consumption thresholds are labeled to form a resource touchpoint matrix. Eddy current solution is then performed, proceeding to S44.

[0189] For example: the scheduling core will Map each node to the resource touchpoint dimension to construct a three-dimensional matrix. , where index , , These correspond to the CPU core, network segment, and power circuit, respectively; and can correspond to the resource matrix constructed based on the gesture instruction intent information to construct the resources required by the instruction.

[0190] Among them, "the scheduling core will" The mapping involved in "mapping nodes to resource touchpoint dimensions" can be done using a list and topology lookup table approach: the nodes given by the scheduling packet and intent map are mapped to the target devices according to their service interfaces, and then the device list is mapped to the three dimensions of processor core, network segment and power loop to form a three-dimensional matrix of resource touchpoints.

[0191] Optionally, during the mapping process, permission tokens and security domains are used for filtering and boundary constraints, node identifiers and initiating roles are used for audit alignment, fault tolerance margins are carried with the matrix, and priority coefficients and delay locks are reserved for subsequent solution use. (3D matrix) The relationship with the real-time intent graph is that the resource selection and constraints are directly determined by the six metadata in the real-time intent graph, such as: service interfaces pointing to devices and processor cores, security domains corresponding to network segments, and device lists indicating power circuits.

[0192] This step S43 can convert the intent information of the gesture command into a three-dimensional matrix of the required resources, and apply it in the subsequent flow solution.

[0193] right Predicting the future using convolution-temporal window fusion occupancy rate And mark the shared lock at the same position in the matrix. Exclusive lock With energy consumption threshold The matrix and occupancy rate As the initial field for the simulation of the conflict vortex.

[0194] Among them, occupancy rate This refers to the projected usage percentage of resource contacts within a future time window, affecting the corresponding processor core, network segment, and power loop location in the three-dimensional matrix. Its value is derived from online predictions of device counts and power consumption sequences under a unified time base; it is a runtime variable and not manually set.

[0195] "Corresponding position in the matrix" refers to the corresponding position of the resource touchpoint pair in the matrix.

[0196] in Device-side parameters belonging to resource touchpoint pairs can be maintained in the resource table by the scheduling core and take effect with the policy. It can be determined based on the description of the interface's idempotency and sharing capabilities. It can be generated based on the mutual exclusion matrix and security policy. The power consumption and / or temperature rise threshold of the power supply circuit can be determined based on equipment specifications and on-site calibration; while It is a runtime predictor, not a manually set parameter.

[0197] In this solution, intent graph nodes can be mapped to resource touchpoints, and then CPU, link, and / or PMIC (Power Management Integrated Circuit) counts and power consumption sequences can be collected. The prediction is obtained through TCN (Temporal Convolutional Network) + sliding window fusion. However, this is not a limitation. Here, "link" refers to the network segment dimension in the resource contact three-dimensional matrix, that is, the portion of network resource occupancy and congestion status related to the link saturation tensor, and does not refer to arbitrary connection relationships; "power consumption sequence" "This refers to the device-side power consumption time series characteristics used in the digital twin simulation and resource touchpoint matrix construction mentioned earlier, specifically..." It is the runtime power consumption time series that serves as input (which can be aligned to a unified time base with counts such as CPU utilization and bandwidth usage).

[0198] S44, Conflict Vortex Deduction and Vortex Core Indicator Calculation, may include: constructing a potential field on the resource contact matrix, iteratively solving for the vortex center by combining three pairs of coupled tensors: permission coverage, link saturation, and lock waiting, and outputting three indices: vortex core radius, lifetime, and blocking increment. Subsequent execution involves inputting the vortex core indicators, proceeding to S45.

[0199] For example: the conflict vortex engine in Upper structure potential field Then based on permission overriding Link saturation Waiting with lock (i.e., exclusive lock) These three pairs of coupled tensors are used to iteratively solve for the parameters of the vortex center: vortex center radius. ,life With blocking increment Specifically, this can be achieved based on the following formula: In the above formula, For the number of contacts, To predict the number of frames; For touch point weight, This is the concurrency coefficient; For control matrix; , , This is the basic scale term. The formula unifies the measurement of spatial gradient, lock-weight-saturation triple coupling, and concurrency density, providing a physically interpretable core index for digital twins.

[0200] Among them, permission overriding Link saturation Waiting with lock In the resource touchpoint matrix The same contact point The above is used to characterize the operational states of three different conflict sources, and the three together participate in the solution of the potential field and vortex core parameters. Specifically: (1) Permission coverage This is used to indicate the degree of coverage / overlap of the permission token and security domain constraints of the current real-time intent graph node on the resource and interface corresponding to the touchpoint. In other words, it represents the strength of the overlap of permission ranges, simultaneous access, or cross-domain access among multiple concurrent instructions on the same resource touchpoint.

[0201] (2) Link saturation The congestion level of the network segment involved in the contact point within the prediction window is typically represented by normalized telemetry data such as link bandwidth usage, queue depth, packet loss, or latency jitter, and is aligned with the network segment dimension in the resource contact point matrix.

[0202] (2) Lock wait Used to represent the mutual exclusion access waiting strength on this contact pair, specifically a normalized quantity corresponding to runtime statistics such as the waiting queue length, lock waiting time, or number of lock conflicts under shared lock / exclusive lock constraints.

[0203] In addition, permission overriding Link saturation Waiting with lock These three pairs of coupled tensors can be determined based on the joint estimation of running state counts and policy metadata.

[0204] The above S44 corresponds to the use of the conflict vortex engine to construct a potential field on the resource matrix; and to determine the vortex center parameter information based on the permission coverage tensor, link saturation tensor, lock wait tensor, and the potential field.

[0205] S45, Digital Twin-Driven Curve Co-evolution, may include: mapping the vortex center to the physical twin device set in real time (i.e., "Real-time Mapping Driven Physical Twin Device Set" in the figure), and linking the evolution of power consumption, temperature rise, and latency curves; when any curve crosses the device warning threshold, an adaptive reduction is issued and the gesture rhythm is reshaped to achieve resource-behavior co-convergence; corresponding to the above, based on the gesture instruction intent information, the digital twin network is used to execute the gesture instruction response operation corresponding to the priority coefficient to obtain the prediction result. Subsequent execution curve feedback leads to S46.

[0206] For example: put Real-time mapping to physical twin device set Drive power consumption curve Temperature rise curve With delay curve Linked evolution; when any curve crosses the equipment warning zone At that time, the scheduling core immediately issued an adaptive reduction order. And corresponding to the hand gesture rhythm The process involves reshaping to achieve resource-behavior co-convergence under soft constraints; correspondingly, based on the vortex center parameter information, a digital twin network is used to execute the gesture command response operation corresponding to the priority coefficient to obtain the prediction result. Gesture beat. This can include the trigger frequency of gesture commands on the timeline, the duration of each trigger, and the concurrency rhythm. Adaptive descent. This indicates that during the co-evolution of digital twin curves, when any of the power consumption curve, temperature rise curve, or time delay curve crosses the device's warning zone... At that time, the degradation / peak shaving amplitude control amount issued by the scheduling core is used to pull the resource usage of the corresponding branch from the high-risk area back to the safe zone.

[0207] The aforementioned "evolution" drives the synchronous updates of the three curves and the digital twin device to obtain the prediction results. When any curve crosses the device's warning zone, an adaptive reduction and beat reshaping instruction is generated (to reshape the gesture beat), enabling resources and behavior to converge collaboratively under soft constraints. The results obtained from the evolution include the prediction results, curve feedback, and beat reshaping instructions. Subsequently, in step 46, these are used as inputs to implement node reshaping and response operations, while the curve feedback is written to the audit log and model backtracking pool, forming a closed loop.

[0208] Among them, "reshaping the beat" refers to the digital twin adjusting the gesture beat on the corresponding branch when any curve of power consumption, temperature rise, or latency crosses the device's warning zone. This is done based on the vortex radius, lifespan, and blockage increment. Without changing the semantics of the gesture, the trigger frequency, single duration, and concurrent rhythm of the gesture command on the time axis are replanned. For example, a continuous drag can be split into multiple slides, the trigger frequency can be reduced, or the occupation duration can be shortened, so that the resource curve falls back into the safety zone. This series of time and rhythm level adjustments is collectively referred to as "reshaping the beat" or "reshaping the gesture beat".

[0209] Among them, "node reshaping and response operation" refers to adjusting the start and end times, concurrency and triggering conditions of the intention micro-units or branch nodes affected by the beat reshaping command on the real-time intention graph, splitting or merging the nodes when necessary, and executing the corresponding control command to the target device under the new beat after node reshaping. Therefore, "response operation" here is the specific response process executed on the device corresponding to the gesture command after the node reshaping is completed.

[0210] Among them, regarding Real-time mapping and curve driving can include: using the three vortex core parameters as control inputs, and obtaining the control vector through equipment twinning. Directly drives DVFS (Dynamic Voltage and Frequency Scaling), power gating, heat dissipation, and queue current limiting, enabling... Co-evolution. Once any curve crosses the warning zone... Issued the price reduction And for the corresponding nodes Reshaping (such as shortening segments, raising trigger thresholds, and reducing concurrency) prompts the curve to return to a safe zone. For example, for display-type graphics processing devices, when the predictive control vector approaches its upper bound, the reduction is calculated based on the weighting coefficient, and frequency reduction and refresh rate adjustment are implemented, while limiting the rendering queue; at the same time, dragging is decomposed into sliding steps and concurrency is limited, serving as node reshaping. Subsequently, the power consumption curve, temperature rise curve, and latency curve synchronously fall back to the safe zone. This process is one execution of the curve-driven and feedback closed loop.

[0211] Among them, control vector The meaning is: taking the three indicators of the surging core as control inputs, and through the equipment traction coefficient... The mapping is a comprehensive control quantity oriented towards the execution plane, used to simultaneously drive channels such as voltage and frequency regulation, power gating, heat dissipation, and queue current limiting. The amplitude and cycle time of each component corresponding to the channel are updated in real time with the curve feedback; when any feedback curve crosses the equipment warning zone, the amplitude reduction and node reshaping are triggered according to the control vector to bring the curve back to the safe zone.

[0212] The equipment pull coefficient comes from equipment specifications, on-site calibration, and safety strategies; the surge radius represents the diffusion intensity, the lifetime represents the persistence, and the blockage stack increment represents the superposition intensity; the warning zone is the equipment safety threshold; the reduction weight consists of a basic reduction term and a weight term adjusted with the blockage stack increment, consistent with the aforementioned weight matrix setting, and can be fine-tuned online based on logs.

[0213] S46, Risk Assessment and High-Risk Branch Feedback, may include: fusing intention interruption rate, vortex core index, and reduction count to calculate a tiered risk score; high-risk branches are fed back to priority reordering, while safe branches maintain their current order (this may correspond to determining the tiered risk score based on the aforementioned prediction results, intention interruption rate, vortex center parameter information, and reduction count; the vortex center parameter information includes: vortex core radius, lifetime, and blockage increment; for gesture commands corresponding to the tiered risk score below the third threshold, a gesture command response operation is executed). Model iteration and heat correction can then be performed and applied to S41.

[0214] For example: Risk assessor fusion intent interruption rate Vortex index Number of times the decline Calculate branch Layered risk classification For example, the following formula: ;in, These are the layer weight coefficients; and For the Sigmoid slope and offset; This represents the criticality vector of the equipment. This is the risk adjacency matrix. If... If a high-risk branch is identified, it is written back to the S3 scheduling queue to trigger a reordering; otherwise, the current order is maintained. (Vortex core indicator) Indicates targeting branches Vortex core index: vortex core radius ,life With blocking increment .

[0215] In this scheme, step 41 can use curve feedback (i.e. model iteration) and heat correction feedback for parameter fine-tuning. Specifically, in this step, incremental correction is made to the priority coefficient and delay lock, and written back to the interpretable scheduling packet, which takes effect with the next round of scheduling, and the algorithm flow remains unchanged.

[0216] The above risk classification The calculation formula is used to calculate the stratified risk value. The correlation coefficient can be given by the strategy and calibration. The smoothing function can be used to converge and suppress mutations. Among them, the equipment criticality and risk adjacency can be used to reflect the importance of equipment and risk transmission. The obtained stratified risk value is only used as the threshold control basis for high risk judgment (which can be understood as the stratified risk value is used to support risk stratification).

[0217] The above "branches" "" refers to the set of gesture instructions within an execution chain on the intent map. The hierarchical risk classification in this solution... The evaluation is conducted on a chain-by-chain basis, without cross-chain merging.

[0218] The above "maintain current order" means that priority reordering is not triggered within this branch, and the process continues according to the sequence already determined in the immediate decision pool. The decision pool mechanism maintains its existing logic.

[0219] Step S46 proceeds as follows: branches deemed high-risk revert to S3 for priority reordering (i.e., reordering the priority of all gesture commands within that branch); non-high-risk branches directly enter the curve-driven and feedback phase, i.e., step 45 continues execution; specifically, Characterizing the starting point of the curve-driven and feedback loop, This is the encapsulation and output sub-step within this process. It's merely a branching node that assesses the risk of branching. When determined to be high-risk, it... Back to Re-prioritize; if determined to be non-high-risk, then follow the order of priority. The curve-driven and feedback loop, starting from this point, continues to execute, naturally entering the next step in sequence. The risk scheduling package is encapsulated. Therefore, non-high-risk branches directly enter the curve-driven and feedback stage, i.e., step [step / step]. Continuing execution means returning to the main line of this stage and continuing to move forward.

[0220] Wherein: the intent interruption rate, vortex core index, and number of drops can be determined by online statistics from the window intent map (i.e., the real-time intent map mentioned above) and the device twin. Intent interruption rate Define as a branch The normalized ratio of the number of time-series breakpoints caused by concurrency and / or rearrangement within a window to the total number of segments can be smoothed using EWMA (Exponentially Weighted Moving Average). Branch risk density is derived from the initial potential field. : Number of decreases The count of degradation and / or throttling (i.e., load reduction) triggers of the branch triggering device after a digital twin alarm is performed (i.e., "adaptive throttling" is executed). (Number of times). Keyness vector It can aggregate the security level, rollback cost, idempotency, and capacity percentage of the device (i.e., the target device obtained from the intent map); adjacency matrix It can be calibrated by cross-device risk coupling and accident propagation records. Weight and the slope and offset of the Sigmoid The threshold can be determined through offline training using log regression and / or Bayesian optimization with labeled logs, and can be adaptively adjusted in small steps based on false positives and / or false negatives; It can be optimally determined by ROC (Responder Operating Characteristic Curve). Among them, (1) Rollback cost refers to the cost required to restore the device and business state to a state that can continue to be executed safely when a branch or instruction is determined to need to be rolled back. It is usually reflected in the time consumption of the rollback script execution, the interruption of ongoing tasks, and the possible secondary rearrangement and recovery overhead; (2) Idempotency means that when the corresponding service interface or instruction is repeatedly triggered under the same input, the final effect is the same as the trigger once, and no additional side effects are produced; (3) Capacity ratio refers to the proportion or share of the target device or target resource in the total available capacity of the current system, which is used to reflect the scarcity and impact of the device on the overall resource pool.

[0221] Among them, branches This refers to real-time intent graphs The executable path units extracted by spatial adjacency and temporal causality rules using DFS (Depth-First Search) and / or BFS (Breadth-First Search) are candidate execution flows composed of several consecutive intent micro-units, all oriented towards the same target device or resource interface. Branches The window contains a semantically consistent sequence of actions and is associated with the specific device. Establish a mapping to aggregate load and risk statistics, so that it can be used in formulas Security is assessed at the branch granularity level; when If the branch is not in a priority order, it is written back to the scheduling queue to trigger a reordering or demotion; otherwise, it continues to proceed according to the current priority order.

[0222] Where, formula : The initial potential field used to construct the branch risk density. This represents the initial potential field strength of branch u within the current window; For branches The radius of the surge center, Indicates branch The number of nodes contained, i.e., the set The cardinality is used to normalize the summation result; the summation index v traverses this branch. Each node in; This is the index of the contact point pair corresponding to node v in matrix Mq; χv represents the predicted occupancy rate of the touch point in the current window; χv is the node weight, derived from existing weighted items such as device criticality. The local potential function is determined by the shared lock, exclusive lock, and energy consumption threshold of the matrix in the same position, affecting the occupancy rate. It is obtained by gating conversion.

[0223] The adaptive throttling count is the cumulative number of frequency reduction, load reduction, and other actions actually triggered by the branch within a given time window, provided by curve feedback and audit logs.

[0224] S47. Divide the risks Coordinates of the vortex core hot zone The beat-corrected gesture commands are encapsulated into a multi-dimensional risk scheduling package, which is pushed to the execution end and simultaneously written into the learning repository. .

[0225] Among them, "gesture command corrected by beat" refers to the version of gesture command sequence formed after the digital twin issues adaptive amplitude reduction and beat reshaping commands based on vortex radius, lifetime and blockage increment in step S45 above, and the time axis and concurrent rhythm are adjusted in "node reshaping and response operation". That is, the original gesture command is updated after the beat reshaping and node reshaping processing in S45.

[0226] Among them, the learning warehouse It can maintain metadata and result labels through time-series indexing, providing continuous samples for the self-evolution of digital twins and the iteration of multi-gesture conflict strategies.

[0227] The core hot zone coordinates refer to the horizontal and vertical index positions on the aforementioned resource touchpoint matrix, determined by core deduction under the same time base, corresponding to the specific landing points of the target device and network segment. These coordinates are used directly in two subsequent instances: first, they are encapsulated into a multi-dimensional risk scheduling package along with risk stratification and rhythm-corrected gesture commands to ensure accurate positioning and overshoot determination at the execution end; second, they are synchronously written into the learning repository as a time-series index and tag for playback, alignment, and model backtracking. The metadata in the learning repository is the same as the six metadata items mentioned earlier, including permission tokens, service interfaces, security domains, node identifiers, initiating roles, and fault tolerance margins; the result tags include risk stratification, core radius, lifetime, blockage stack increment, curve feedback, rhythm reshaping results, and whether descent and reordering were triggered, etc.

[0228] The above involves extending concurrent comparison to the physical resource layer and forming an interpretable deduction: in S41-S42, the rearranged trajectory is remapped into a gesture-clock grid, the intent micro-units are decomposed, a real-time intent graph is generated, and semantic nodes and six-element metadata are provided for subsequent resource mapping.

[0229] In S43, a three-dimensional matrix of resource contacts (CPU core / network segment / power circuit) is constructed. TCN+ sliding window fusion is used to predict contact occupancy, and shared locks / exclusive locks and energy consumption thresholds are marked at the same position in the matrix.

[0230] The S44 introduces a conflict vortex engine, which constructs a potential field by coupling the permission coverage tensor, the link saturation tensor, and the lock wait tensor. Iteratively solves the three vortex core indices: vortex core radius, lifetime, and blocking increment. These three indices can project concurrent interference onto a physically interpretable resource layer for quantification.

[0231] In S45-S46, the power consumption, temperature rise, and / or delay curves of the digital twin are driven by the three core indicators. When the warning zone is crossed, the reduction and cycle time reshaping are issued, and the reordering is triggered by the hierarchical risk write-back in S3. This can realize the risk closed loop of deduction-execution write-back.

[0232] V. Step S5 may include the following sub-steps: S51, Event shaping: The scheduling core merges the high-risk frames marked in S4 with the synchronously captured simulated anomalies according to the millisecond time sequence, and extracts the gesture identifier. Contact surface pixel block Resource usage trajectory And write a uniform structure conflict feedback frame. Then put Push to a dedicated agent model instance.

[0233] Event shaping refers to rearranging and normalizing multi-source asynchronous samples on a unified timeline at the millisecond level. It involves merging high-risk frames marked by S4 with corresponding blur anomalies from the sensing side (such as motion blur, sudden drops in keypoint confidence, contact mask jitter, and occupancy jumps in the original data stream acquired by the camera) according to their arrival order with spatial adjacencies. The shaped events carry gesture tags. Contact image blocks With resource usage trajectory Each entry is accompanied by a unified timestamp and serial number for subsequent concurrent assessment and seepage simulation.

[0234] In this solution: it can be done in the window. The camera frames and skeleton stream are denoised and resampled to millisecond beats; the ROI can be obtained by cropping the control contact area based on the hit test. Gesture networks can be used to infer categories and confidence levels from short clips. Events can be mapped to resource touchpoints and based on current monitoring usage. With unit load Resource touchpoints are synthesized, and then predicted using exponential smoothing or one-dimensional convolution. Finally, the three types of data (gesture labels) can be combined. Contact image blocks With resource usage trajectory Packed with timestamps Write to the circular buffer and submit to the scheduler. The difference between camera frames and skeleton streams lies in their data format and semantic level: a camera frame (also called a camera module frame) is the raw image / depth data stream captured by the camera, containing complete scene information; the skeleton stream is the temporal data of joint points obtained through pose estimation based on camera frames (usually depth camera frames), belonging to a structured abstraction of hand / human motion; "Hit test cropping" refers to cropping the corresponding region of the camera frame after determining the location and range of the contact based on the hit test, obtaining the ROI image patch related to the contact. "Short clips" refers to clips taken at the window. The term "resource touchpoint" refers to the resource landing point in the resource touchpoint matrix after the event is mapped to the resource layer. It is the specific location where a certain instruction occupies / mutually excludes / transmits on a certain resource.

[0235] Among them, "current monitoring usage" "" refers to the real-time occupancy status of the corresponding resource touchpoints collected by the operation and maintenance monitoring system at a unified master clock time. For example, the CPU utilization rate, link bandwidth occupancy ratio, or power circuit current ratio of the touchpoint have been aligned to the same time axis by window.

[0236] Among them, "unit load" "" refers to the standardized load contribution of a certain type of gesture command on a resource touchpoint, that is, the average amount of resources consumed in a single event or per unit of time (such as unit power consumption or unit bandwidth usage), which is pre-defined by historical logs and digital twin templates. By combining the current monitored usage with the unit load, the resource usage intensity related to the gesture at that moment can be obtained, and the resource usage trajectory mentioned above can be constructed accordingly for conflict assessment and deduction.

[0237] Among them, a high-risk frame refers to a set of touch points that are triggered by the gesture command of this branch within the same time window and have been marked as high-risk in S4. These touch points fall on the resource touch point matrix obtained by the intent map and are evidence of the running state of this branch, rather than a single command itself.

[0238] Regarding "merging high-risk frames marked in S4 with synchronously captured simulated anomalies," the specific steps may include: the scheduling core, according to the S4 markings, synchronously extracts corresponding anomaly segments from data streams such as camera, micro-sensor, contact points, and occupancy waveforms, aligns and fuses them under a unified time base. The data streams such as "camera, micro-sensor, contact points, and occupancy waveforms" are actually unified representations of the acquisition channels already included in the system, based on the S4 markings: camera data corresponds to the point cloud and skeleton stream obtained from the multi-view depth camera in step S1; micro-sensor data is the output of optional motion sensors (such as inertial or pressure sensors) synchronized with the camera, used to supplement attitude information; contact points originate from front-end hit tests or touch panels, corresponding to the contact area markings on the screen or controls; occupancy waveforms are time series data such as power consumption, bandwidth, and CPU utilization collected from the resource contact matrix and the operation and maintenance monitoring system. After the above data streams are aligned under a unified time base, corresponding segments are extracted from the time window of the high-risk frames marked in S4 for event shaping and anomaly merging.

[0239] Among them, a dedicated agent model instance refers to a lightweight decision agent for a single resource domain or execution branch, dynamically instantiated at the window level, and capable of receiving... Combined with context parameters, quickly calculate the risk score and priority correction for this frame. With possible lock delay The results are then written back to the scheduling core. Agents can employ a uniformly structured gray-box model, whose parameters can be calibrated offline using historical logs and fine-tuned online with sliding error to provide interpretable prior decisions.

[0240] S52, Multi-source comparison, i.e., the proxy model compares each... Simultaneously search the local conflict index. Historical similar gesture samples With real-time hardware telemetry (such as temperature rise, current spikes, link jitter), and expressed as a three-dimensional similarity vector. Execute the judgment. Each represents a line Conflicting indexes with local Similarity to historical similar gesture samples Similarity and real-time hardware telemetry Consistency; "Retrieving local conflicting indexes" Historical similar gesture samples With real-time hardware telemetry This corresponds to the category parameter information for obtaining the target gesture command mentioned above; the category parameter information includes at least one of: trajectory, context, and telemetry signature. The "trajectory, context, and telemetry signature" can be obtained from the "local conflict index". Historical similar gesture samples With real-time hardware telemetry Features extracted from ".

[0241] like The probability of the corresponding category is greater than the threshold. Then (will) The part marked "Known" is located within the interval. Marked as "suspected", below Marking it "Unknown" provides a branch for subsequent adaptive playback.

[0242] The three-dimensional similarity vector is obtained as follows: In multi-source comparison, the gesture command of this branch is matched with the local conflict index, historical similar gesture samples, and real-time hardware telemetry under the same time base. The similarity is then calculated and normalized according to the existing similarity metric (i.e., the preset trajectory similarity), resulting in three components representing the similarity with the local conflict index, the similarity with historical similar gesture samples, and the consistency with real-time hardware telemetry, respectively. Here, "this branch" refers to an execution branch to be evaluated in the real-time intent map. This refers to a sequence of gesture commands composed of several consecutive intentional micro-units. Under a unified time base, the proxy model associates the gesture commands in this branch with the "local conflict index." Historical similar gesture samples With real-time hardware telemetry "Matching one by one, the three-dimensional similarity vector is obtained by calculating the preset trajectory similarity, context similarity, and telemetry consistency." ,in Indicates trajectory similarity. Indicates contextual similarity, The values ​​represent telemetry consistency, and are all normalized similarity indices.

[0243] For each event Construct a three-way similarity vector Gesture trajectories are encoded as vectors. In historical examples of similar gestures Perform nearest neighbor search and retrieve ; Index local conflicts Mapping to features Combined with the model center calculate From real-time hardware telemetry Capture window sequence , and signature Perform DTW (Dynamic Time Warping) and / or cross-correlation to obtain Three-way similarity can be aggregated into class probabilities. : threshold It can be calibrated using ROC and fine-tuned online from labeled data. This represents the distance metric from the context to the pattern center, used to exponentially smooth the context similarity to obtain the components. Specifically: (1) Signature It is with the center of the pattern The corresponding reference telemetry sequence is used to set the real-time hardware telemetry window sequence. Align and compare with typical resource consumption patterns of similar historical events / risk models. (Signature) The method of acquisition is: during the construction of a local conflict index. At that time, extract CPU, link, and / or PMIC counts and power consumption sequences corresponding to the type of event from the tagged historical event samples. After denoising, resampling, and normalizing within the same time window, representative sequences of this type of sample are obtained and solidified. Online runtime also allows for the use of new samples without changing the definitions. Make small updates to keep it consistent with the statistical characteristics of the device.

[0244] (2) This is used to normalize the dimensions and magnitude of the DTW distance to a comparable exponential similarity scale, i.e. Controlling the decay rate and sensitivity: The larger the difference in DTW, the better. The weaker the punishment, the greater the tolerance for mistakes; The smaller the value, the more sensitive it is to timing deviations. In implementation... The DTW distance statistics can be preset on the labeled data and smoothly calibrated according to the equipment load and noise level during the online phase.

[0245] Regarding the above criteria, it may include: for each event The agent retrieves from the cache And query the local And replay the near Real-time hardware telemetry ; Calculate according to the above formula With category probability .when Mark known information and execute according to the matching template; Falling Similar markings are used between them, employing conservative measures and implementing human review and / or self-learning; Below If the data is unknown, it will be assigned a lower weight in the scheduling process, and the samples and telemetry data will be written back to the database for online updates.

[0246] The process involves performing a nearest neighbor search in the sample library to select the closest reference sample for the gesture trajectory of the current branch. This reference sample is used to calculate the three-way similarity and output the class probability. The meaning and acquisition method of the three-way similarity are as follows: Trajectory Similarity Take the maximum cosine similarity between the current trajectory's bending vector and the candidate sample's bending vector; context similarity. Telemetry consistency is obtained by exponentially smoothing the distance from the context set to the pattern center; The distance between the telemetry curve and the sample signature within this time window, after dynamic time warping alignment, is obtained through exponential mapping. Here, "pattern center" refers to the representative center point obtained in the learning pod after aggregating historical similar gesture samples by gesture label and context, under the joint description of trajectory and context. It can be understood as the centroid of the representative trajectory and context for this type of task, used to calculate the distance from the current context to this center and map it as a context similarity component. "This type of task" corresponds to the task type corresponding to "historical similar gesture samples," that is, the task corresponding to the set of historical similar gesture samples that have the same gesture label and are in the same business scenario as the current branch to be evaluated.

[0247] Historical similar gesture samples This refers to the collection of samples archived in the learning repository according to similar labels, used for nearest neighbor retrieval and similarity evaluation; local conflict index. It is a runtime local conflict dictionary and index used to retrieve known conflict patterns in the current scenario.

[0248] Real-time hardware telemetry, namely the device-side sampling of temperature rise, current spikes, link jitter, etc. mentioned in the text, has been aligned with the gesture trajectory in the same time window and participated in the consistency calculation.

[0249] In the formula The meaning is: It is the trajectory similarity component, and maxcos represents the maximum value of the cosine similarity in the candidate sample set; Let be the bending stiffness vector of the current branch. The bending vector of the candidate sample is used to select the closest reference sample for stable discrimination.

[0250] S53. When a returned frame is marked as "suspected" or "unknown" (i.e., marked as high-risk or unknown conflict), replay the entire original sensor stream from the buffer and reduce the sampling time granularity to [a specific value]. (This corresponds to increasing the sampling rate of the original sensor stream for the target gesture command when the category parameter information indicates that the target gesture command belongs to the target category, as described above), and reconstructing the concurrent contact topology based on the entire original sensor stream at an ultra-high sampling rate. Competing Chain with Lock During the reconstruction process, implicit interaction events not displayed in S4 are recorded. This ensures that the quantization process has a complete source of interference. The replayed "all raw sensor streams" and the feedback frames correspond to the same time window. The feedback frames can be understood as segments extracted or compressed for feedback verification. The "all raw sensor streams" emphasize retrieving the unpruned, undownsampled raw multi-source sensor data from the buffer within that time window for use in finer-grained time processing (downsampling to... in the text). The contact trajectory is realigned and reconstructed to compete with the lock chain.

[0251] The "return frames" refer to those frames marked as high-risk or judged as "suspected" or "unknown" in S4, along with their regions of interest. The purpose of the return frames is to perform threshold correction, sample labeling, and impact quantization after the sampling granularity is reduced, and to simultaneously write back to the learning module. The "region of interest" refers to the localized region of interest (ROI) automatically delineated by the system based on abnormal characteristics on the camera view, point cloud, and / or touch plane corresponding to the frames marked as high-risk, suspected, or unknown in S4.

[0252] Concurrent contact topology "Based on the resource contact matrix, it reconstructs the contacts within the time window at an ultra-high sampling rate while simultaneously establishing adjacency relationships; "lock contention chain" "Implicit interaction events" are a sequence of locks and contention events derived from resource activation times and mutual exclusion relationships, used to supplement interference sources. "" refers to subtle interactions that are not apparent in S4 but can only be confirmed after replay at an ultra-high sampling rate. These interactions are automatically detected according to the original sensor stream and written to the log and learning repository. The occupancy trajectories and monitoring waveforms of each resource touchpoint obtained earlier can be extracted to obtain the time series "from idle to occupied, load increase and / or decrease," which serves as the "resource activation timeline." The "resource activation timeline" can be used in conjunction with mutual exclusion relationships to deduce the "lock and contention timeline chain."

[0253] S54, will With gesture intention diagram Equipment critical curve and business priorities Perform multi-layer stacking to generate a two-dimensional conflict-affected ellipse. And calculate the (conflict impact) coverage. (This corresponds to determining the conflict coverage rate based on the sampling rate mentioned above.) When Above the threshold If you enter a branch that requires a rollback (which corresponds to the above-mentioned reacquisition of priority coefficients), or if you enter a branch that allows for a delayed solution (which corresponds to the above-mentioned increase in delay duration and unlocking conditions).

[0254] The coverage rate can be obtained using the following formula: ; ;in, For the corresponding equipment risk domain; It is a coordinate vector; As the center of the conflict; It is a positive definite shape matrix (such as a covariance matrix).

[0255] Among them, gesture intention diagram The current projection from the real-time intent map has been written back after curve-driven and thermal correction and maintains the same time base; device critical curve It can generate online statistics based on equipment specifications, on-site calibration, and audit logs, reflecting safety zone boundaries such as power consumption, temperature rise, and latency; business priority. Priority coefficients from the decision pool can be mapped to this branch according to the weight matrix of the strategy library. The weight.

[0256] Among them, "will" With gesture intention diagram Equipment critical curve and business priorities "Multi-layer overlay" may include combining gesture intent maps in the resource touchpoint coordinate domain. Equipment critical curve and business priorities Multi-layer overlay is performed, such as using the coordinates of the hot zone of the surge core or the weighted center of the branch contact as the center of the ellipse, and using the shape matrix to characterize the expansion direction and scale to obtain the conflict impact ellipse; the coverage rate is calculated according to the proportion of the overlapping area between the ellipse and the equipment risk domain. If it exceeds the threshold, it is judged as a branch that needs to be rolled back. If it does not exceed the threshold, it enters the delayed solvable branch, and then the closed loop can be executed to continue feedback and write back.

[0257] Coverage can be used to measure the geometric proportion of conflict impacts within the device risk domain; its physical meaning is... exist The coverage ratio in Taking 0 indicates that the value does not fall within the risk domain. Setting it to 1 indicates complete coverage; It is a dimensionless metric that can be thresholded and used to determine whether to enter a backtracking branch.

[0258] coordinate vector It is a branch of equipment (i.e., the local two-dimensional coordinates of path u above, such as coordinates on the working surface and / or control surface). Specifically, visual or touch data is first estimated in world coordinates to determine the contact position, and then mapped to the local coordinate system using device extrinsic parameters and surface parameters; non-planar devices can use locally unfolded or parametric surface coordinates, while virtual resources are replaced by logical surface coordinates. The resulting... and The density centroid and principal axis scales within this domain can be characterized separately.

[0259] The conflict here refers to the conflict between concurrent commands and resource usage within a window on the device. The resulting spatiotemporal interference integrates constraints such as locking semantics, access control, and link saturation, and forms a density field through layering. And was In summary, this affects the outline. Coverage. That is, the interference pair The actual degree of penetration is used to drive scheduling degradation and rollback.

[0260] All metrics in this section are expressed in terms of equipment or resource branches. For objects. It is equipment The risk domain , , All are targeted in the current window Event generation; threshold Subsequent rollback branches also only apply to this. Its related instructions take effect. Different devices evaluate independently and output their own results. And trigger the corresponding scheduling action.

[0261] S51 to S54 above can correspond to the category parameter information of the target gesture command obtained above; the category parameter information includes at least one of trajectory, context and telemetry signature; when the category parameter information indicates that the target gesture command belongs to the target category, the sampling rate of the original sensor stream for the target gesture command is increased; based on the sampling rate, the collision coverage rate is determined.

[0262] S55. For branches marked as needing to be retracted, locate the affected fiber assembly. Automatically generate fine-grained rollback scripts The script only rolls back the beats of conflicting fibers and preserves the rest of the gesture rhythms, providing a separate minimum rollback window. With re-queue number This will minimize overall execution latency.

[0263] Among them, the affected fiber set refers to the branch Occupied trajectory In the middle, with equipment risk domain Or overlock The set of smallest continuous segments that intersect The impact of conflict on the ellipse and coverage As a criterion: if a certain segment satisfy Or exist Then include Based on intent maps Path binding and These fragments can be located using time stamps, and scripts that only roll back conflicting fragments can be generated accordingly. Without affecting the rhythm of other gestures. Among them, the condition " Or exist Used to filter this branch Whether a contact within the time window enters rollback or delay can be determined. This indicates the j-th contact of the branch in the resource contact matrix; The conflict affects the elliptic set; This is the device risk domain. Indicator function. A value of 1 indicates that the contact point belongs to the corresponding set; If the product of these two is 1, it means that the contact point is located in the overlapping area of ​​the conflict impact ellipse and the device risk domain. It is the weight in the lock contention chain, derived from the statistics of resource activation time traces and mutual exclusion relationships. A value greater than zero indicates that there is actual contention between contact i and contact j in that time window.

[0264] Regarding "shortest rollback window" With re-queue number The acquisition of "" may include: Map to interface primitives and schedule them according to the dependency graph, such as: for each The time taken to perform the cancel→unlock→flush→resync operation is recorded as follows: Shortest rollback window Ensure the entire system returns to its normal operating position; re-enter the queue. Given by a stable sorting algorithm, the sort key is... With risk The synthesized score is kept fair and monotonous. The system ultimately returns... , and This allows for rollback and reordering with minimal latency. The undo time refers to the actual time taken from initiation to completion for each molecule in branch u, executing the cancel, unlock, refresh, and synchronize actions sequentially according to interface dependencies. The undo time can be obtained from audit logs and device-side sampling under a unified time base and fine-tuned based on curve feedback. The undo time can be used to determine the shortest rollback window: the undo time of each molecule is taken, and the maximum value is used as the rollback window for that branch, covering the slowest path and ensuring a global return to a safe state after rollback. Here, "molecule" refers to each gesture command contained in the branch.

[0265] S56, Instructions for entering delayed solvable branches It can be based on the real-time power consumption curve With dynamic power consumption limit Solving for the minimum delay window (This corresponds to the delay duration determined above based on real-time power consumption information and dynamic power consumption limit) and unlocking conditions. If the following formula is used to solve it: ;in For the current moment, Duration of the instruction; Power consumption curves for other concurrent gestures; Power consumption penalty factor (e.g., excess energy penalty weight); dynamic power consumption limit It can be formed by combining twin prediction with power and / or thermal limits; Specifically, it can be an instruction. The instantaneous power consumption curve from startup. The solution obtained... and The script will be written to a delayed execution script, which the scheduler will execute autonomously within the allowed interval (i.e., execute only when the conditions are met). Here, δ represents the minimum delay window for instruction 1 under the delayed solvable branch, i.e., the amount of time to wait from the current moment, ensuring that the total power consumption during the waiting and execution process does not exceed the dynamic power consumption limit; δ is a non-negative time value, with units consistent with the time unit of the power consumption curve. h represents the index of the concurrent branch, used to traverse other gestures that coexist with instruction 1 during summation, integrating and accumulating their power consumption impact within a specified time period. Indicates the first The optimal delay obtained by solving for the instruction (or branch) that enters the delay-solvable branch is, in other words, the delay variable. After substituting the power consumption constraints and penalty terms, through The solution obtained; unlocking conditions It can be composed of two parts: a power consumption criterion and a lock contention criterion. The above formula gives the power consumption criterion: after obtaining the minimum delay window of the instruction, if the excess power consumption term in parentheses... A value of zero indicates that, throughout the entire duration following the delay, the combined power consumption of the target instruction and concurrent instructions no longer exceeds the dynamic power consumption limit, at which point power-side unlocking is successful. Subsequently, based on the lock contention chain mentioned earlier, it is checked whether the contention weight associated with this instruction has reached zero. If it has been cleared, lock-side unlocking is successful. Both conditions must be met simultaneously for the unlocking condition of instruction 1 to be met; if either condition is not met, the delay is increased and recalculated until the condition is met or the fallback strategy is triggered.

[0266] S51~S56 above can correspond to the above-mentioned determination of conflict coverage rate for target gesture instructions; the target gesture instruction is the gesture instruction corresponding to the hierarchical risk score that is greater than or equal to the third threshold; according to the conflict coverage rate, a first operation is performed for the target gesture instruction; the first operation includes: increasing the delay duration and unlocking conditions; or, re-acquiring the priority coefficient.

[0267] S57. Finally, the final judgment result. , rollback script Unlock parameters Write the conflict index and push the learning repository synchronously. .

[0268] The learning pod adaptively updates conflict weights and confidence levels based on new data, continuously improving the prediction accuracy of subsequent multi-gesture concurrent conflict resolution, while providing high-quality labels for expanding the behavior space of the digital twin model.

[0269] in, This can be obtained by maximizing the utility of multi-source evidence, such as by first determining the probability of discovery. (i.e., class probability obtained from three-way similarity), and coverage is calculated from spatial overlap. The minimum delay and unlocking condition are determined by the energy consumption constraint (i.e., the cancellation time). .definition Perform a delayed rollback within the feasible region. make The maximum is .when or If the time is not specified, it will be directly judged as a rollback, and a corresponding rollback will be generated. . , , , These are preset weights used to adjust the proportions of class probability, coverage, time window, and business priority in utility composition. , , , The value can be obtained from online updates of the strategy library and learning repository, maintaining consistency with real-world performance. Utility function The meaning is to combine the above-mentioned class probability, coverage, time window and business priority into a single score according to weight, use it to select the optimal rollback window and give the queue number, and the higher the score, the higher the priority of execution. This usually indicates the unlocking condition. The indicator function, which converts whether the unlock condition is met, into... Signal: When the first Instructions that enter a delayed solvable branch in the delay Once the necessary conditions for unlocking and continuing the process are met, If the required lock cannot be obtained within the prediction window, or if there is an unacceptable lock wait / conflict causing the unlocking condition to be invalid, then Among them, "when" or If the time is not specified, it will be directly judged as a rollback, and a corresponding rollback will be generated. "Includes branching" The rule for determining a rollback is: when the coverage... (Risk coverage exceeds the threshold) or If the unlock condition is not met, it will be automatically reverted and a corresponding rollback will be generated. Otherwise, a delayed rollback will be executed in the action domain. Utility function Take the maximum value .

[0270] Regarding "adaptive update of conflict weights and confidence," it may include: Along with the trajectory, telemetry, and context, write them to the local conflict index. With Learning Container (e.g., writing to a local conflict index) With Learning Container (Corresponding tables / indexes / logs). Perform prototype incremental updates on known samples. This involves incrementally updating the weights w; creating new clusters for similar / unknown datasets and attaching digital twin playback results as strong labels. Training examples use input... Organize and perform offline batch revaluation Using a power consumption template, weights and thresholds are updated online with only hot updates, enabling continuous learning and rapid convergence. Here, w represents the original value of the conflict weights, adjusted according to... Perform incremental updates, where α2 is the incremental coefficient. This represents the class probability of that class. Class probability The calibration coefficient.

[0271] The above involves introducing feedback depth verification and refining it into two types of executable branches: In S51-S52, through event shaping and multi-source comparison, high-risk frames and sensor anomalies are fused on the millisecond time axis, and combined with a dedicated agent model instance, the trajectory / context / telemetry signature three-way similarity (i.e., three-way similarity vector) is used. ) Determine the returned frame The returned frames are categorized as known, suspected, or unknown, and those marked as suspected or unknown are further categorized. Reconstruct concurrent contact topology and lock contention chains using S53's ultra-high sampling rate to capture implicit interactions.

[0272] In S54, a conflict-affected ellipse is constructed, coverage is calculated, and branches requiring rollback and delayed solvable branches are divided according to a threshold. In S55, a fine-grained rollback script is generated only for the affected fiber set, not a global rollback. In S56, the minimum delay window and guard conditions (i.e., unlocking conditions) are solved, and an executable delayed execution script can be obtained using a piecewise convex one-dimensional search.

[0273] VI. Step S6 may include the following sub-steps: S61. The scheduling core receives the security token generated in S5. Priority identifier The gesture command buffer is written according to three priority levels. Simultaneously record the dependency chain. Target device identification and minimum delay window This lays the data structure foundation for parallel scheduling.

[0274] Among them, security token Twin-band and lock strategy combination from S45; priority identification Risks stemming from S33 / S46—preferred aggregation and discretization by threshold; three queues This refers to a hierarchical queue for scheduling buffers. Dependency chains. From the intent map The target device identifier is derived from the resource topology. From the contact matrix The mapping is determined.

[0275] S62. Periodically scan the buffer based on dependency chains. Extract the set of key instructions Generate key linked list It also grants absolute preemption rights. The linked list covers instructions for high-risk scenarios such as emergency stop, screen switching, and security domain switching, ensuring that it can enter the execution channel in the shortest possible time even when competitive resources are scarce.

[0276] S63, the scheduling core can every Summary of front-end touch count GPU frame rendering rate and bus utilization And integrate the vortex load index from S4. Quickly generate the next time window Resource consumption snapshot Snapshot and threshold vector Real-time comparison is performed to provide trigger credentials for tiered rate limiting. Among these, It can be understood as a predicted value, which is a short-term predicted snapshot for the next time window.

[0277] S64, when Approximating the corresponding threshold in any dimension Low-priority queues can be frozen immediately. ; for medium priority queues A sliding window delay strategy is applied, with the window width determined by each instruction. For high-priority queues Maintain real-time delivery. The rate limiting strategy avoids overall congestion caused by resource spikes by dynamically adjusting window boundaries.

[0278] The "sliding window delay strategy" includes, for example, shifting the earliest start time of each instruction backward by the smallest delay window obtained from the solution. The window is then periodically adjusted based on the latest load forecast. This dynamic adjustment of the window boundaries is based on... and The gap, adaptive adjustment The check cycle allows low- and mid-level tasks to start within a manageable range, thereby smoothing out peaks and valleys and avoiding global blockages caused by spikes.

[0279] S65, the scheduling core is based on the target device set. Device concurrency limit Map the current executable queue to the physical channel matrix. Execution is triggered after thread alignment of parallelizable instructions. The scheduling core listens for a completion signal after each batch is triggered. And immediately release the corresponding thread slot to keep the channel saturation close to the optimal level.

[0280] S66, Execution end can return delay Power consumption Status indicators The scheduling core uses the latest feedback (latency). Power consumption Status indicators Dynamically corrected threshold vector And based on the time delay jitter coefficient The sliding window automatically tightens; when jitter increases and crosses the warning boundary, the freeze level adaptively escalates, prioritizing protection. The real-time performance and reliability of instructions. The sliding window here is used by the scheduling kernel to update the threshold vector in online feedback. The time window / sample window is used to aggregate the latency of the most recent execution return. Power consumption and status indicators And accordingly, the control measures will be dynamically tightened.

[0281] Among them, "time delay" "Refers to the end-to-end time consumption feedback of the execution end for branch u within the current window; "Power consumption" "Instantaneous power consumption feedback" refers to the power consumption feedback on the device side within the same window, which is consistent with the power consumption curve mentioned above; "Desired power consumption" refers to the target value or target upper limit given by the device critical curve and power consumption template, which is used to compare with the feedback power consumption and correct the threshold vector. The scheduling core updates the threshold according to the "Desired power consumption" and tightens the sliding window according to the delay jitter coefficient.

[0282] The "adaptive upgrade of freeze level" is a graded rate limiting method based on the intensity of jitter.

[0283] S67, if or Exceeding the deviation threshold Immediately interrupt the current queue and remove the unexecuted instruction set. Along with exception codes The deep verification process is sent back to S5; the results of successfully completed instructions and their execution context are synchronously written to the audit log. This provides complete samples for model backtracking. Indicates the expected power consumption. The corresponding number is Instruction (or the first) The expected duration / time budget given by each execution unit on the planning side is used to generate the expected completion time of the instruction.

[0284] in, Based on the starting point of the plan Duration get: Expected power consumption Can be taken from model curves The peak or integral, and in S56 according to With return value Perform online calibration. Tolerance. The threshold is set at the device level (which can be determined by calibration + SLA) and adaptively scaled according to the residual's EMA; exception codes. The deviation magnitude, direction, and duration can be mapped from the code table to determine the abnormal code. This can be used to select rollback and review branches. The rollback value... This represents the threshold vector that was corrected and written back by S66 in the previous scheduling cycle. S6 only uses this return value and the dynamic power consumption limit cap for online calibration, without changing its source. If there is no return value in the first round, it can be initialized according to the default threshold of the device critical curve and power consumption template, and subsequently updated by step 66.

[0285] The "return depth verification process to S5" refers to S55–S57: generating the rollback script. u The feasibility and power consumption are verified and solved, and indexing and / or learning are written back; the current queue is a three-tier queue. Activity batches currently being consumed (i.e., executed); audit logs This refers to the system's non-repudiable audit log library, used for traceability and twin calibration.

[0286] It should be noted that the above content does not restrict the value of the same symbol in different positions; the values ​​can be the same or different.

[0287] Based on the above, this solution addresses the following: 1. Problem Solved: In high-concurrency, multi-view real-time gesture scenarios, the lack of a closed-loop process from identity continuity and conflict resolution to resource scheduling and digital twin linkage leads to difficulties in coordinating judgment results with hardware load. The technical means provided by this solution is a spatiotemporal integrated skeleton fusion and adaptive segmentation mechanism. It uniformly registers multi-camera point clouds, dynamically monitors the velocity curve and its derivative, and adjusts the gesture segment boundaries in real time according to motion trends. It also generates mutually exclusive scores, dynamic priorities, and locking delays using a cross-window concurrency relationship matrix and an instantaneous adjudication pool, providing complete and consistent trajectory and adjudication input. This solution eliminates the disconnect between fixed frame windows and single-threshold segmentation, transforming scene evidence into an executable priority chain. Specifically, the mechanism for dynamically determining gesture segment boundaries based on velocity derivatives is as follows: real-time monitoring of velocity and its first derivative; when the derivative crosses zero or abruptly changes at a local extremum, window scaling is triggered, synchronizing the segment boundary with the extremum frame; subsequently, the boundary drifts in real time as the velocity curve rises or falls, completing adaptive segmentation.

[0288] 2. This solution enables fine-grained calculation and adjudication in conflict scenarios. The technical methods involved include: constructing a cross-window concurrency relationship matrix to uniformly quantify overlapping windows and intersecting adjacent windows; integrating posture similarity, business tags, and dynamic risk factors to calculate mutual exclusion scores; and dynamically sorting and locking latency in the adjudication pool based on real-time load and alarm density. This solution achieves adaptive matrix-based adjudication and supports stable instruction reordering and latency locking under high concurrency.

[0289] 3. To achieve a closed loop in risk simulation and resource collaborative scheduling, the technical means involved are as follows: mapping the rearranged gesture trajectory to a three-dimensional matrix of resource touchpoints; using a conflict eddy current engine and digital twin curves to synchronously simulate power consumption, temperature rise, and latency; adjusting priority and delay windows in real time based on eddy core indicators; and executing this in hierarchical current limiting scheduling. This solution introduces an eddy core radius-lifetime coupling model and a power consumption-temperature rise co-evolution mechanism, which enables scheduling to be linked with physical load and has self-correcting capabilities.

[0290] In addition, in this scheme, the queuing rules for gestures without conflicts can be as follows: without entering the backoff and delay process, the gestures are directly sorted and executed stably based on the priority matrix of the decision pool, arrival time and interface dependency order; only when the power consumption on the device side reaches the dynamic limit, a micro-delay is made according to the minimum delay window obtained in the steps, without changing the queue number; otherwise, the gestures can be executed immediately.

[0291] In summary, the spatiotemporal integrated skeleton fusion and adaptive segmentation mechanism proposed in this application dynamically monitors the velocity curve and its derivative at the acquisition layer, enabling the gesture segment to adjust its boundary in real time according to the movement trend, thereby maintaining identity continuity even when the posture changes rapidly or the camera is briefly occluded. This can significantly reduce the probability of identity drift during multi-view fusion and provide a complete and consistent trajectory input for subsequent conflict determination.

[0292] At the concurrency analysis and adjudication layer, this solution incorporates overlapping and intersecting events between windows into the calculation through a cross-window gesture concurrency relationship matrix. This is then combined with gesture similarity, business tags, and dynamic risk factors to form a mutually exclusive score, enabling fine-grained quantification of conflict scenarios. The matrix and adjudication pool in this solution together form an adaptive priority generation chain, allowing instructions to be sorted and latency locked based on multi-source information such as real-time load and alarm density, fundamentally improving scheduling accuracy and system stability in high-concurrency environments.

[0293] During the execution phase, this solution maps the rearranged gesture trajectory to a three-dimensional matrix of resource touchpoints and uses a conflict eddy current engine and digital twin curves to synchronously extrapolate indicators such as power consumption, temperature rise, and latency. The risk assessor adjusts priorities and delay windows in real time based on the eddy core index, forming a closed-loop control path of perception-analysis-examination-execution; it can self-correct under sudden load changes or drastic fluctuations in equipment status, enabling the system to maintain predictability and safety margins in resource usage while ensuring real-time response.

[0294] Specifically, the methods employed in this solution to address the problem include: S11 first-order constant velocity compensation time base correction; S15~S17 velocity derivative-driven adaptive segmentation and semantic splicing; S22 cross-window concurrent relationship matrix; S24 scene-based embedding to generate unified representation; S25 mutual exclusion score synthesis and dynamic conflict heatmap; S31~S36 instant adjudication pool + composite priority coefficient + differential threshold control and time delay locking model; S41~S43 gesture-clock grid, real-time intent map and resource touchpoint three-dimensional matrix; S44~S46 conflict vortex engine and vortex core three-index driven digital twin co-evolution and hierarchical risk scoring; S51~S56 feedback depth verification, conflict impact ellipse coverage determination and minimum delay window solution.

[0295] In summary, this application addresses the issues of identity drift, mutual exclusion conflicts, and device overload that easily arise from multi-view, high-concurrency gesture commands in public large screens or similar interactive environments. This solution primarily manifests in three aspects: First, it utilizes a spatiotemporal integrated skeleton fusion and adaptive segmentation mechanism to uniformly register point clouds from multiple depth cameras and dynamically determine gesture segment boundaries based on velocity derivatives, ensuring identity consistency from the source. Second, it introduces a cross-window gesture concurrency relationship matrix and an instant adjudication pool, integrating posture similarity, business semantics, and real-time risk factors into a mutual exclusion score and dynamically generating priorities and locking delays to achieve fine-grained conflict judgment and command rearrangement. Third, it constructs a closed-loop risk scheduling link through conflict eddy current deduction and digital twin curves, enabling the system to promptly correct thresholds and maintain resource stability during sudden load changes. The overall solution relies on the above continuous processes to form a unique technical combination.

[0296] In addition, this solution also has the following advantages: 1. Improved display quality and user experience; Since the core algorithm remains relatively neutral to sensor type, as long as it can output a time-synchronized 3D skeletal flow, the conflict judgment and risk scheduling link can be reused, thus the solution has the potential for cross-industry migration. In medical training or remote consultation scenarios, multiple doctors can perform contactless operations simultaneously without interfering with each other's view control; in industrial collaborative robots or smart factories, the system can parse the gesture commands of multiple workstations, evaluate the mutual exclusion relationship with the robotic arm in real time, and reduce the probability of line stoppage; for metaverse, multi-person AR / VR (augmented reality and / or virtual reality) and e-sports events, the algorithm can support high frame rate concurrent capture and unified adjudication on the server side to ensure interaction latency and content consistency; in smart cockpits, home central control or public security terminals, the solution can also implement dynamic prioritization of cross-device commands and coordinate audio, video, navigation and alarm resources.

[0297] 2. Wide applicability to various industries; This solution is not limited to a specific industry but is adaptable across industries. Whether it's commercial advertising, public displays, smart city infrastructure, command centers, or medical imaging, all can benefit from the application of this solution. By covering multiple industry sectors, the technology can meet broader market demands, increase commercialization opportunities, expand market reach, and improve return on investment.

[0298] Overall, this solution allows for rapid product line iteration and multi-industry reuse; driven by the increasing demand for multi-user interaction and the widespread adoption of edge computing capabilities, the related technology has considerable market expansion prospects.

[0299] The following are examples illustrating some of the relevant definitions mentioned above: 1. The concurrency relationship matrix in S22 refers to a two-dimensional data structure that pairs all active gesture segments within each sliding time window, recording their degree of overlap in the spatiotemporal dimension, business semantic relevance, and shared resource conflict factors in a row and column format. Matrix elements typically contain composite values ​​such as overlap duration, intersection angle, and target device mapping, thus providing a sufficient quantitative basis for subsequent conflict determination after a single traversal.

[0300] For example, in the following scenario: Within sliding window W1, user A's "zoom in" gesture G1 and user B's "exit" gesture G2 are active simultaneously, forming a pair (G1, G2). The matrix records the duration of overlap and spatial intersection within the same window. In the adjacent sliding window W2, G1 continues while user C's "screenshot" gesture G3 newly enters, forming a pair (G1, G3). The matrix records the intersection between adjacent windows. For each pair, S32 dimensions can be superimposed. For example, if the device level is high, the current load is 80%, and the operator role is administrator, the mutual exclusion score is increased, and a lockout delay is triggered.

[0301] 2. The mutual exclusion score calculation and dynamic conflict heatmap in S25 introduce a weighting function on top of the concurrency relationship matrix, integrating gesture similarity, resource mutual exclusion, and business risk factors into a normalized score. The system refreshes a pseudo-color or grayscale conflict heatmap in real time according to the score, visually displaying the potential interference level of each gesture pair within the current window. The heatmap is not only for direct observation by operations and maintenance personnel, but can also serve as input for the algorithm layer to adaptively adjust thresholds and scheduling strategies.

[0302] 3. The device level, current load, alarm level, maintenance window occupancy rate, operator role, and geographical isolation domain mentioned in S32 are the six core dimensions of the risk factor model. Device level can be pre-weighted according to hardware importance and failure cost; current load can be mapped to a ballast coefficient using real-time indicators such as CPU (Central Processing Unit), GPU (Graphics Processing Unit), and power consumption; alarm level reflects the emergency state of the system or service; maintenance window occupancy rate refers to the ratio of scheduled or executing background tasks in the current time period; operator role can distinguish between ordinary users, administrators, or privileged scripts, involving permission priorities; geographical isolation domain can be used to mark whether instructions cross physical or network partitions to avoid cross-regional operations. These dimensions collectively determine the risk-side weight allocation for mutually exclusive scores. "Alarm level" can be the alarm level on the service side, used to reflect the current emergency state, derived from monitoring and service agreements, and written back in real-time during runtime.

[0303] 4. Real-time updates of the heat coefficient in S34 may include: when a significant change is detected in a matrix element or an external risk dimension, the conflict heat is corrected online by exponential moving average or Kalman filtering; the system can thus converge to the new risk level in a timely manner during sudden load increases, network jitter, or concentrated user operations, thereby maintaining a balance between judgment sensitivity and false alarm rate.

[0304] 5. The priority list in S35 is a dynamic data sequence generated by sorting according to mutual exclusion score, popularity coefficient, and role weight. List nodes may contain information such as gesture segment identifier, target device, and expected execution time stamp. The entire list supports fast insertion, deletion, and reordering within each update cycle, ensuring that the scheduler always schedules instructions to be executed with the latest global priority. The scheduler may be included in the aforementioned device or set up independently; this is not limited here. The scheduler can also be described as a scheduling center, etc.

[0305] 6. The difference threshold control and delay locking in S36 is a kind of anti-jitter mechanism. The system can first calculate the mutual exclusion score difference of adjacent linked list nodes. Only when the score difference exceeds the adaptive threshold will the execution order switch be triggered. If the score difference is insufficient, the current order can be maintained by applying a delay lock at the microsecond to millisecond level to avoid frequent scheduling or priority jitter caused by short-term heat fluctuations.

[0306] This application embodiment also provides an information processing device, as shown in FIG6, including: a first acquisition module 61, used to acquire a gesture sequence of at least one user based on point cloud data from at least two perspectives; the user corresponds to the point cloud data; a second acquisition module 62, used to acquire concurrent gesture information based on the gesture sequence; a third acquisition module 63, used to acquire the priority coefficient of each gesture instruction based on the priority factor information of each gesture instruction corresponding to the concurrent gesture information; the priority factor information includes at least one of the following: the device level of the device targeted by the gesture instruction, the current load of the device targeted by the gesture instruction, the alarm level of the service and / or device targeted by the gesture instruction, the occupancy rate of the operation and maintenance window of the device targeted by the gesture instruction, the operator role of the user corresponding to the gesture instruction, and the isolation domain information corresponding to the gesture instruction; and a first execution module 64, used to execute the gesture instruction response operation according to the priority coefficient.

[0307] The information processing device provided in this application embodiment acquires a gesture sequence of at least one user based on point cloud data from at least two perspectives; the user corresponds to the point cloud data; concurrent gesture information is acquired based on the gesture sequence; priority coefficients of each gesture command are acquired based on priority factor information of each gesture command corresponding to the concurrent gesture information; the priority factor information includes at least one of the following: device level of the device targeted by the gesture command, current load of the device targeted by the gesture command, alarm level of the service and / or device targeted by the gesture command, occupancy rate of the operation and maintenance window of the device targeted by the gesture command, operator role of the user corresponding to the gesture command, and isolation domain information corresponding to the gesture command; and performs gesture command response operation based on the priority coefficient; it can support dynamic generation of gesture command priorities in high-concurrency scenarios to achieve fine-grained conflict determination and command sorting, and also supports timely and fine-grained queuing and rate limiting strategies when the number of gestures increases sharply or the status of backend resources fluctuates rapidly, improving the accuracy of command response, and effectively solving the problem of poor response accuracy in existing information processing schemes for gesture commands.

[0308] The step of obtaining a gesture sequence of at least one user based on point cloud data from at least two perspectives includes: fusing and filtering point cloud data from at least two perspectives to obtain a gesture trajectory of at least one user; determining the segmentation window length corresponding to each gesture trajectory based on the average velocity of at least two frames in each gesture trajectory; segmenting the corresponding gesture trajectory based on the segmentation window length to obtain candidate gesture segments for each user; and obtaining the gesture sequence of the at least one user based on the candidate gesture segments for each user.

[0309] In this embodiment of the application, the step of segmenting the corresponding gesture trajectory according to the segmentation window length of each gesture trajectory to obtain candidate gesture segments corresponding to each user includes: obtaining the first derivative sign change information of the velocity curve corresponding to the first gesture trajectory according to the first segmentation window length of the first user's first gesture trajectory; determining the segmentation boundary according to the first derivative sign change information; and segmenting the first gesture trajectory according to the segmentation boundary to obtain candidate gesture segments corresponding to the first user.

[0310] The step of obtaining concurrent gesture information based on the gesture sequence includes: constructing a gesture concurrency relationship matrix based on a first condition and the gesture sequence; the first condition is determined based on two adjacent sliding windows; for each element in the gesture concurrency relationship matrix, the corresponding skeletal trajectory sequence, duration, and semantic information are concatenated and embedded to obtain a multi-dimensional feature vector; the semantic information includes at least one of function tags, associated devices, and service levels; obtaining the mutual exclusion score between any two elements in the gesture concurrency relationship matrix based on the multi-dimensional feature vector; and obtaining concurrent gesture information based on the elements corresponding to the mutual exclusion scores that are higher than a first threshold.

[0311] In this embodiment of the application, obtaining the priority coefficient of each gesture instruction based on the priority factor information of each gesture instruction corresponding to the concurrent gesture information includes: obtaining the priority coefficient of each gesture instruction based on the window heat coefficient and the priority factor information of each gesture instruction corresponding to the concurrent gesture information.

[0312] Furthermore, the information processing device further includes: a fourth acquisition module, used to acquire the window heat coefficient based on the alarm density and traffic fluctuation rate within the first time period.

[0313] The step of performing a gesture command response operation based on the priority coefficient includes: obtaining the difference between the priority coefficients of two mutually exclusive gesture commands; setting a lock delay for the gesture command corresponding to a target priority coefficient; the lock delay is determined based on the difference corresponding to the target priority coefficient and the device level; the target priority coefficient refers to the lower priority coefficient among the two priority coefficients corresponding to the difference greater than a second threshold; and performing a gesture command response operation based on the priority coefficient and the lock delay.

[0314] In this embodiment of the application, the step of executing a gesture command response operation based on the priority coefficient includes: obtaining gesture command intent information based on the priority coefficient; executing the gesture command response operation corresponding to the priority coefficient using a digital twin network based on the gesture command intent information to obtain a prediction result; determining a hierarchical risk score based on the prediction result, intent interruption rate, vortex center parameter information, and reduction number; the vortex center parameter information includes: vortex center radius, lifetime, and blockage increment; and executing a gesture command response operation for gesture commands corresponding to the hierarchical risk score that is below a third threshold.

[0315] The step of executing the gesture command response operation corresponding to the priority coefficient using a digital twin network based on the gesture command intent information to obtain the prediction result includes: constructing a resource matrix corresponding to the resources required by the command based on the gesture command intent information; constructing a potential field on the resource matrix using a conflict vortex engine; determining vortex center parameter information based on the permission coverage tensor, link saturation tensor, lock wait tensor, and the potential field; and executing the gesture command response operation corresponding to the priority coefficient using a digital twin network based on the vortex center parameter information to obtain the prediction result.

[0316] Furthermore, the information processing device further includes: a first determining module, configured to determine a conflict coverage rate for a target gesture instruction; the target gesture instruction is a gesture instruction corresponding to the hierarchical risk score that is greater than or equal to the third threshold; and a second execution module, configured to perform a first operation for the target gesture instruction based on the conflict coverage rate; the first operation includes: increasing the delay duration and unlocking conditions; or, re-acquiring the priority coefficient.

[0317] The step of determining the conflict coverage rate for a target gesture instruction includes: acquiring category parameter information of the target gesture instruction; the category parameter information includes at least one of trajectory, context, and telemetry signature; when the category parameter information indicates that the target gesture instruction belongs to a target category, increasing the sampling rate of the original sensor stream for the target gesture instruction; and determining the conflict coverage rate based on the sampling rate.

[0318] Furthermore, the information processing device further includes a second determining module, used to determine the delay duration based on real-time power consumption information and dynamic power consumption limit.

[0319] The implementation embodiments of the above information processing method are all applicable to the embodiments of the information processing device and can achieve the same technical effect.

[0320] This application embodiment also provides an information processing device, as shown in FIG7, including: a processor 71; the processor 71 is configured to acquire a gesture sequence of at least one user based on point cloud data from at least two perspectives; the user corresponds to the point cloud data; acquire concurrent gesture information based on the gesture sequence; acquire a priority coefficient of each gesture instruction based on priority factor information of each gesture instruction corresponding to the concurrent gesture information; the priority factor information includes at least one of the following: device level of the device targeted by the gesture instruction, current load of the device targeted by the gesture instruction, alarm level of the service and / or device targeted by the gesture instruction, occupancy rate of the operation and maintenance window of the device targeted by the gesture instruction, operator role of the user corresponding to the gesture instruction, and isolation domain information corresponding to the gesture instruction; and execute a gesture instruction response operation based on the priority coefficient.

[0321] The information processing device provided in this application embodiment acquires a gesture sequence of at least one user based on point cloud data from at least two perspectives; the user corresponds to the point cloud data; concurrent gesture information is acquired based on the gesture sequence; priority coefficients of each gesture command are acquired based on priority factor information of each gesture command corresponding to the concurrent gesture information; the priority factor information includes at least one of the following: device level of the device targeted by the gesture command, current load of the device targeted by the gesture command, alarm level of the service and / or device targeted by the gesture command, occupancy rate of the operation and maintenance window of the device targeted by the gesture command, operator role of the user corresponding to the gesture command, and isolation domain information corresponding to the gesture command; and performs gesture command response operation based on the priority coefficient; it can support dynamic generation of gesture command priorities in high-concurrency scenarios to achieve fine-grained conflict determination and command sorting, and also supports timely and fine-grained queuing and rate limiting strategies when the number of gestures increases sharply or the status of backend resources fluctuates rapidly, improving the accuracy of command response, and effectively solving the problem of poor response accuracy in existing information processing schemes for gesture commands.

[0322] The step of obtaining a gesture sequence of at least one user based on point cloud data from at least two perspectives includes: fusing and filtering point cloud data from at least two perspectives to obtain a gesture trajectory of at least one user; determining the segmentation window length corresponding to each gesture trajectory based on the average velocity of at least two frames in each gesture trajectory; segmenting the corresponding gesture trajectory based on the segmentation window length to obtain candidate gesture segments for each user; and obtaining the gesture sequence of the at least one user based on the candidate gesture segments for each user.

[0323] In this embodiment of the application, the step of segmenting the corresponding gesture trajectory according to the segmentation window length of each gesture trajectory to obtain candidate gesture segments corresponding to each user includes: obtaining the first derivative sign change information of the velocity curve corresponding to the first gesture trajectory according to the first segmentation window length of the first user's first gesture trajectory; determining the segmentation boundary according to the first derivative sign change information; and segmenting the first gesture trajectory according to the segmentation boundary to obtain candidate gesture segments corresponding to the first user.

[0324] The step of obtaining concurrent gesture information based on the gesture sequence includes: constructing a gesture concurrency relationship matrix based on a first condition and the gesture sequence; the first condition is determined based on two adjacent sliding windows; for each element in the gesture concurrency relationship matrix, the corresponding skeletal trajectory sequence, duration, and semantic information are concatenated and embedded to obtain a multi-dimensional feature vector; the semantic information includes at least one of function tags, associated devices, and service levels; obtaining the mutual exclusion score between any two elements in the gesture concurrency relationship matrix based on the multi-dimensional feature vector; and obtaining concurrent gesture information based on the elements corresponding to the mutual exclusion scores that are higher than a first threshold.

[0325] In this embodiment of the application, obtaining the priority coefficient of each gesture instruction based on the priority factor information of each gesture instruction corresponding to the concurrent gesture information includes: obtaining the priority coefficient of each gesture instruction based on the window heat coefficient and the priority factor information of each gesture instruction corresponding to the concurrent gesture information.

[0326] Furthermore, the processor is also used to: obtain the window heat coefficient based on the alarm density and traffic fluctuation rate within a first time period.

[0327] The step of performing a gesture command response operation based on the priority coefficient includes: obtaining the difference between the priority coefficients of two mutually exclusive gesture commands; setting a lock delay for the gesture command corresponding to a target priority coefficient; the lock delay is determined based on the difference corresponding to the target priority coefficient and the device level; the target priority coefficient refers to the lower priority coefficient among the two priority coefficients corresponding to the difference greater than a second threshold; and performing a gesture command response operation based on the priority coefficient and the lock delay.

[0328] In this embodiment of the application, the step of executing a gesture command response operation based on the priority coefficient includes: obtaining gesture command intent information based on the priority coefficient; executing the gesture command response operation corresponding to the priority coefficient using a digital twin network based on the gesture command intent information to obtain a prediction result; determining a hierarchical risk score based on the prediction result, intent interruption rate, vortex center parameter information, and reduction number; the vortex center parameter information includes: vortex center radius, lifetime, and blockage increment; and executing a gesture command response operation for gesture commands corresponding to the hierarchical risk score that is below a third threshold.

[0329] The step of executing the gesture command response operation corresponding to the priority coefficient using a digital twin network based on the gesture command intent information to obtain the prediction result includes: constructing a resource matrix corresponding to the resources required by the command based on the gesture command intent information; constructing a potential field on the resource matrix using a conflict vortex engine; determining vortex center parameter information based on the permission coverage tensor, link saturation tensor, lock wait tensor, and the potential field; and executing the gesture command response operation corresponding to the priority coefficient using a digital twin network based on the vortex center parameter information to obtain the prediction result.

[0330] Furthermore, the processor is also configured to: determine a conflict coverage rate for a target gesture instruction; the target gesture instruction is a gesture instruction corresponding to the hierarchical risk score that is greater than or equal to the third threshold; perform a first operation for the target gesture instruction based on the conflict coverage rate; the first operation includes: increasing the delay duration and unlocking conditions; or, re-acquiring the priority coefficient.

[0331] The step of determining the conflict coverage rate for a target gesture instruction includes: acquiring category parameter information of the target gesture instruction; the category parameter information includes at least one of trajectory, context, and telemetry signature; when the category parameter information indicates that the target gesture instruction belongs to a target category, increasing the sampling rate of the original sensor stream for the target gesture instruction; and determining the conflict coverage rate based on the sampling rate.

[0332] Furthermore, the processor is also used to: determine the delay duration based on real-time power consumption information and dynamic power consumption limit.

[0333] The implementation embodiments of the above information processing method are all applicable to the embodiments of the information processing device and can achieve the same technical effect.

[0334] This application also provides an information processing device, including a memory, a processor, and a program stored in the memory and executable on the processor; when the processor executes the program, it implements the above-described information processing method.

[0335] The implementation embodiments of the above information processing method are all applicable to the embodiments of the information processing device and can achieve the same technical effect.

[0336] This application also provides a readable storage medium storing a program thereon, which, when executed by a processor, implements the steps in the information processing method described above.

[0337] The implementation embodiments of the above information processing method are all applicable to the embodiments of the readable storage medium and can achieve the same technical effect.

[0338] This application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, they implement the various processes of the above-described information processing method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0339] It should be noted that many of the functional components described in this specification are referred to as modules in order to more specifically emphasize the independence of their implementation.

[0340] In this embodiment, the module can be implemented in software so that it can be executed by various types of processors. For example, an identified executable code module may include one or more physical or logical blocks of computer instructions, which may be constructed as objects, procedures, or functions. Nevertheless, the executable code of the identified module does not need to be physically located together, but may include different instructions stored in different bits, which, when logically combined, constitute the module and achieve the module's intended purpose.

[0341] In practice, an executable code module can be a single instruction or many instructions, and can even be distributed across multiple different code segments, different programs, and across multiple memory devices. Similarly, operational data can be identified within the module and can be implemented in any suitable form and organized within any suitable data structure. This operational data can be collected as a single dataset or distributed across different locations (including different storage devices), and can exist, at least in part, solely as electronic signals within the system or network.

[0342] When a module can be implemented using software, considering the current level of hardware technology, modules that can be implemented in software can be implemented using hardware circuits by those skilled in the art to achieve the corresponding functions, without considering cost. These hardware circuits include conventional very-large-scale integrated circuits (VLSI) or gate arrays, as well as existing semiconductors such as logic chips and transistors, or other discrete components. Modules can also be implemented using programmable hardware devices, such as field-programmable gate arrays, programmable array logic, and programmable logic devices.

[0343] The above describes the preferred embodiments of this application. It should be noted that those skilled in the art can make several improvements and modifications without departing from the principles described in this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An information processing method, characterized in that, include: Based on point cloud data from at least two perspectives, obtain the gesture sequence of at least one user; Based on the gesture sequence, obtain concurrent gesture information; Based on the priority factor information of each gesture command corresponding to the concurrent gesture information, the priority coefficient of each gesture command is obtained. The priority factor information includes at least one of the following: the device level of the device targeted by the gesture command, the current load of the device targeted by the gesture command, the alarm level of the service and / or device targeted by the gesture command, the occupancy rate of the operation and maintenance window of the device targeted by the gesture command, the operator role of the user corresponding to the gesture command, and the isolation domain information corresponding to the gesture command; the gesture command response operation is executed according to the priority factor.

2. The information processing method according to claim 1, characterized in that, The step of obtaining a gesture sequence of at least one user based on point cloud data from at least two perspectives includes: fusing and filtering point cloud data from at least two perspectives to obtain a gesture trajectory of at least one user; determining the segmentation window length corresponding to each gesture trajectory based on the average velocity of at least two frames in each gesture trajectory; segmenting the corresponding gesture trajectory based on the segmentation window length to obtain candidate gesture segments for each user; and obtaining the gesture sequence of the at least one user based on the candidate gesture segments for each user.

3. The information processing method according to claim 2, characterized in that, The step of segmenting the corresponding gesture trajectory according to the segmentation window length of each gesture trajectory to obtain candidate gesture segments for each user includes: obtaining the first derivative sign change information of the velocity curve corresponding to the first gesture trajectory according to the first segmentation window length of the first user's first gesture trajectory; determining the segmentation boundary according to the first derivative sign change information; and segmenting the first gesture trajectory according to the segmentation boundary to obtain candidate gesture segments for the first user.

4. The information processing method according to claim 1, characterized in that, The step of obtaining concurrent gesture information based on the gesture sequence includes: constructing a gesture concurrency relationship matrix based on a first condition and the gesture sequence; the first condition is determined based on two adjacent sliding windows; for each element in the gesture concurrency relationship matrix, the corresponding skeletal trajectory sequence, duration, and semantic information are concatenated and embedded to obtain a multi-dimensional feature vector; the semantic information includes at least one of function label, associated device, and service level; obtaining the mutual exclusion score between any two elements in the gesture concurrency relationship matrix based on the multi-dimensional feature vector; and obtaining concurrent gesture information based on the element corresponding to the mutual exclusion score that is higher than a first threshold.

5. The information processing method according to claim 1, characterized in that, The step of obtaining the priority coefficient of each gesture instruction based on the priority factor information of each gesture instruction corresponding to the concurrent gesture information includes: obtaining the priority coefficient of each gesture instruction based on the window heat coefficient and the priority factor information of each gesture instruction corresponding to the concurrent gesture information.

6. The information processing method according to claim 5, characterized in that, Also includes: The window heat coefficient is obtained based on the alarm density and traffic fluctuation rate within the first time period.

7. The information processing method according to claim 1, characterized in that, The step of executing a gesture command response operation based on the priority coefficient includes: obtaining the difference between the priority coefficients of two mutually exclusive gesture commands; setting a lock delay for the gesture command corresponding to a target priority coefficient; the lock delay is determined based on the difference corresponding to the target priority coefficient and the device level; the target priority coefficient refers to the lower priority coefficient among the two priority coefficients corresponding to the difference that is greater than a second threshold; and executing a gesture command response operation based on the priority coefficient and the lock delay.

8. The information processing method according to claim 1 or 7, characterized in that, The step of executing a gesture command response operation based on the priority coefficient includes: obtaining gesture command intent information based on the priority coefficient; executing the gesture command response operation corresponding to the priority coefficient using a digital twin network based on the gesture command intent information to obtain a prediction result; determining a hierarchical risk score based on the prediction result, intent interruption rate, vortex center parameter information, and reduction number; the vortex center parameter information includes: vortex center radius, lifetime, and blockage increment; and executing a gesture command response operation for gesture commands corresponding to the hierarchical risk score that is below a third threshold.

9. The information processing method according to claim 8, characterized in that, The step of executing the gesture command response operation corresponding to the priority coefficient using a digital twin network based on the gesture command intent information to obtain the prediction result includes: constructing a resource matrix corresponding to the resources required by the command based on the gesture command intent information; constructing a potential field on the resource matrix using a conflict vortex engine; determining vortex center parameter information based on the permission coverage tensor, link saturation tensor, lock wait tensor, and the potential field; and executing the gesture command response operation corresponding to the priority coefficient using a digital twin network based on the vortex center parameter information to obtain the prediction result.

10. The information processing method according to claim 8, characterized in that, Also includes: Determine the conflict coverage rate for the target gesture command; The target gesture instruction is the gesture instruction corresponding to the hierarchical risk score that is greater than or equal to the third threshold. Based on the conflict coverage rate, perform a first operation on the target gesture command; The first operation includes: increasing the delay duration and unlocking conditions; Alternatively, re-acquire the priority coefficient.

11. The information processing method according to claim 10, characterized in that, The step of determining the conflict coverage rate for a target gesture command includes: acquiring category parameter information of the target gesture command; the category parameter information includes at least one of trajectory, context, and telemetry signature; when the category parameter information indicates that the target gesture command belongs to a target category, increasing the sampling rate of the original sensor stream for the target gesture command; and determining the conflict coverage rate based on the sampling rate.

12. The information processing method according to claim 10, characterized in that, Also includes: The delay duration is determined based on real-time power consumption information and dynamic power consumption limit.

13. An information processing device, characterized in that, include: The first acquisition module is used to acquire at least one user's gesture sequence based on point cloud data from at least two perspectives; The second acquisition module is used to acquire concurrent gesture information based on the gesture sequence; The third acquisition module is used to acquire the priority coefficient of each gesture instruction based on the priority factor information of each gesture instruction corresponding to the concurrent gesture information. The priority factor information includes at least one of the following: the device level of the device targeted by the gesture command, the current load of the device targeted by the gesture command, the alarm level of the service and / or device targeted by the gesture command, the maintenance window occupancy rate of the device targeted by the gesture command, the operator role of the user corresponding to the gesture command, and the isolation domain information corresponding to the gesture command. The first execution module is used to execute gesture command response operations according to the priority coefficient.

14. An information processing device, characterized in that, include: processor; The processor is used to acquire a gesture sequence of at least one user based on point cloud data from at least two perspectives. Based on the gesture sequence, obtain concurrent gesture information; Based on the priority factor information of each gesture command corresponding to the concurrent gesture information, the priority coefficient of each gesture command is obtained. The priority factor information includes at least one of the following: the device level of the device targeted by the gesture command, the current load of the device targeted by the gesture command, the alarm level of the service and / or device targeted by the gesture command, the occupancy rate of the operation and maintenance window of the device targeted by the gesture command, the operator role of the user corresponding to the gesture command, and the isolation domain information corresponding to the gesture command; the gesture command response operation is executed according to the priority factor.

15. An information processing device, comprising a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that, When the processor executes the program, it implements the information processing method as described in any one of claims 1 to 12.

16. A readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the information processing method as described in any one of claims 1 to 12.

17. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the steps of the information processing method as described in any one of claims 1 to 12.

Images