A human motion trajectory prediction method based on time sequence transformer
By using a temporal Transformer-based approach, combining human pose and planned path corridor features, a dual-stream temporal token sequence is constructed and streaming inference is performed. This solves the security and real-time issues of human motion trajectory prediction in robot systems, and achieves stable and reliable prediction under occlusion and abrupt action.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 广东宽恒云数字科技有限公司
- Filing Date
- 2026-03-05
- Publication Date
- 2026-07-21
AI Technical Summary
Existing technologies struggle to achieve safe and reliable prediction of human motion trajectories in robot systems under conditions of occlusion, sudden movements, and diverse scenarios. Furthermore, deep learning models suffer from high latency and memory pressure during edge inference, making it impossible to directly support robot safety decisions.
A human motion trajectory prediction method based on temporal Transformer is adopted. Temporal image data and robot state data are acquired through visual sensors to generate human posture and planned path corridor features. A dual-stream temporal token sequence is constructed, and streaming inference and local segment gating caching are performed on the robot. Combined with hazard and warning corridor constraints, cross-stream interaction modeling and risk adaptive pruning are realized.
It achieves stable and reliable prediction during path planning and risk switching, reduces the risk of missed stops, improves response speed, reduces edge latency and memory usage, and enhances stability under occlusion and false detection conditions.
Smart Images
Figure CN122435665A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of robot vision technology, and more specifically, to a method for predicting human motion trajectories based on temporal Transformer. Background Technology
[0002] In scenarios such as collaborative robots, mobile robots, and service robots, robots need to operate safely in environments where they coexist with humans.
[0003] In existing technologies, one type of method extrapolates human trajectories based on traditional filtering or kinematic models, which is difficult to adapt to occlusion, sudden movements, and diverse scenarios. Another type of method uses deep learning models (such as RNN, temporal convolution, Transformer, etc.) to predict human keypoint sequences or pedestrian trajectories. Although this can improve prediction accuracy, it generally suffers from the following problems: Most trajectory prediction models focus on predicting the error index of the body trajectory, without explicitly encoding the safety corridor constraints of the robot's planned path into the model input or forming a "prediction-control" closed-loop triggering logic. This makes it difficult for the prediction results to directly support robot safety decisions, or requires additional post-processing modules, resulting in untimely responses. At the same time, on the robot's edge, model inference needs to meet the requirements of low latency and stable frame rate. If a full attention calculation is performed directly on the full window token, it is easy to cause latency and memory pressure. However, if a simple pruning or sparse strategy is used, key context may be lost at high-risk moments, resulting in missed stops or false stops.
[0004] Therefore, there is a need for a method and system for predicting human motion trajectories that can run in real time on the edge, is strongly coupled with the planned corridor, and is stable and reliable during planning fine-tuning and risk switching. Summary of the Invention
[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a human motion trajectory prediction method based on temporal Transformer to solve the problems mentioned in the background art.
[0006] To achieve the above objectives, the present invention provides the following technical solution: a human motion trajectory prediction method based on temporal Transformer, applied to a robot system, comprising: Step S1, Input Acquisition: Time-series image data is acquired by a vision sensor installed on the robot body. The time-series image data includes RGB image sequences and depth image sequences. Simultaneously, robot state data synchronized with the time-series image data is acquired, and the robot state data includes at least robot pose information and robot current planned path information; Output synchronized time-series image data and robot state data; Step S2, Human pose and confidence information generation: Perform human detection and pose estimation frame by frame on the time-series image data to obtain the human skeleton key point sequence; Output the keypoint coordinates and keypoint confidence at each time step, and generate a missing mask based on the keypoint confidence threshold; The key point coordinates are transformed to the robot base coordinate system to obtain the human body relative pose features, and the geometric center of the key point coordinates that are not marked as missing by the missing mask is taken as the human body center point. When the number of available keypoints is less than 5 at a certain moment, the human body center point of the previous moment is used as the current human body center point; output the human body center point, keypoint confidence and missing mask; Step S3, Generation of dual-level corridors and path segmentation versions: Based on the planned path information, construct the planned path corridor features, including dangerous corridors and warning corridors, wherein the corridor radius of the warning corridor is greater than that of the dangerous corridor; The planned path is resampled into 200 path points at equal intervals according to the travel arc length, and the 200 path points are divided into 10 continuous path segments of 20 each. For each path segment, a segment version identifier is generated, which is composed of a hash value or checksum generated from the coordinate sequence of 20 path points within the path segment. Calculate the minimum distance between the human body and the planned path based on the human body center point and the 200 path points, and determine the distance level information; Outputs information on dangerous corridors, warning corridors, distance levels, and segment version identifiers for 10 path segments; Step S4: Dual-stream temporal token construction: Construct a dual-stream temporal token sequence in chronological order, wherein the dual-stream temporal token sequence includes at least a human posture token and a path corridor token; The human pose token includes at least key point coordinates, key point confidence, missing mask, and motion change features obtained by difference between adjacent time points; The path corridor token includes at least a parameterized representation of the danger corridor and the warning corridor, distance level information, and a segment version identifier of at least one path segment corresponding to the current prediction time range; Output a dual-stream timing token sequence; Step S5, End-side streaming inference and local segment gated key / value caching: Establish a sliding time window on the robot to cache human pose tokens and path corridor tokens from recent moments; The attention layer of the temporal Transformer maintains a first Key / Value cache related to the human pose token and a second Key / Value cache related to the path corridor token, and uses the segment version identifier as the cache key of the second Key / Value cache; Based on the gating of planned path changes and risk changes, the second Key / Value cache is gated and updated, where: Planning path change gating: When it is detected that the segment version identifier of any path segment is inconsistent with the segment version identifier of the corresponding path segment in the previous period, cache invalidation and reconstruction are performed only on the second key / value cache entry corresponding to that path segment; Risk change gating: When the distance level information changes from non-dangerous to dangerous or from dangerous to non-dangerous, cache invalidation and reconstruction are performed only on the second Key / Value cache entry of the path segment corresponding to the current prediction time range; The second Key / Value cache is reused if none of the above-mentioned gating updates are triggered. In any case, only the newly entered human pose tokens within the sliding time window are incrementally generated with a first key / value and written to the first key / value cache, so as to reuse the first key / value corresponding to the historical human pose tokens within the window; Output the first Key / Value cache and the second Key / Value cache after partial reconstruction; Step S6, Risk Adaptive Pruning and Forced Retention of Sensitive Event Anchor Points: Determine the risk level based on distance level information, key point confidence, and missing mask, and switch between high-risk mode and low-risk mode; When the distance level is dangerous or the average confidence of key points is lower than the preset control threshold or the missing proportion is higher than the preset missing proportion threshold, the high-risk mode is entered. In the high-risk mode, pruning is stopped and all tokens within the sliding time window are retained to participate in the prediction. In the low-risk mode, the importance of tokens within the sliding time window is filtered to form a set of clipped tokens. The importance filtering is determined based on key point confidence, motion change characteristics and distance level information. In the low-risk mode, sensitive event anchor tokens are forcibly retained. The sensitive event anchor tokens are determined in the following way: human posture tokens within the same sliding time window remain unchanged, and corridor perturbation is performed only on path corridor tokens and prediction is performed once for each. The corridor perturbation includes expanding the radius of dangerous corridors by 10% to 20% and shrinking the radius of warning corridors by 10% to 20%. The predicted trajectories before and after the disturbance are respectively judged as "whether they enter the dangerous corridor, whether they enter the warning corridor, and whether a corridor crossing event occurs" and judgment labels are generated. When there is an inconsistency between the judgment labels before and after the disturbance, the token at the corresponding moment is marked as the sensitive event anchor token and forcibly retained; In low-risk mode, in addition to forcibly retaining sensitive event anchor tokens, the six most recent tokens within the sliding time window are also forcibly retained; and after the forced retention is completed, the total number of tokens to be pruned is limited to 60% of the length of the sliding time window. Output the set of tokens used for prediction; Step S7, Trajectory Prediction Output: Input the set of tokens used for prediction into a temporal Transformer to perform cross-flow interaction modeling of human pose flow and path corridor flow. The cross-flow interaction modeling includes performing interaction attention calculation in at least the first interaction attention sub-layer with human pose token as query and path corridor token as key and value. Output the sequence of trajectory points for the human body in the next few steps, along with the progressive confidence information corresponding to each trajectory point; Step S8, Control closed-loop output and backtracking: Generate robot control instructions based on trajectory point sequence, progressive confidence information and two-level corridor features, wherein: when the predicted trajectory enters a dangerous corridor or a corridor crossing event occurs, an emergency stop instruction is output; When the predicted trajectory enters the warning corridor but not the danger corridor, a speed limit command is output; otherwise, a path planning recalculation command or a motion maintenance command is output. The low confidence percentage of the progressive confidence information is calculated. When the low confidence percentage exceeds a preset percentage threshold, a rollback operation is performed. The rollback operation includes at least stopping the pruning and / or increasing the sliding time window length to improve the reliability of subsequent predictions. The control commands are output to the robot actuator.
[0007] In a preferred embodiment, the segment version identifier is obtained by encoding the coordinate sequence of 20 path points within the corresponding path segment and generating a fixed-length verification digest; when any verification digest changes, only the second Key / Value cache entry corresponding to that path segment is reconstructed while the second Key / Value cache entries corresponding to the other path segments are reused, so as to reduce the end-side reconstruction overhead.
[0008] In a preferred embodiment, the "path segment corresponding to the current prediction time range" is estimated by the robot's current speed and predicted step length to determine the robot's reachable distance within the prediction time range, and one or two path segments that intersect with the coverage of the distance are identified as the corresponding path segments, so that invalidation and reconstruction are performed only on the second Key / Value cache entries of the corresponding path segments.
[0009] In a preferred embodiment, the corridor crossing event includes: at least two consecutive steps in the predicted trajectory point sequence where adjacent trajectory points are located on both sides of the dangerous corridor boundary, and the stepwise confidence levels corresponding to the at least two consecutive steps are not lower than a preset confidence threshold; when the corridor crossing event is detected, an emergency stop command is directly output and the stop clipping in the rollback operation is triggered.
[0010] In a preferred embodiment, the importance screening is performed in a low-risk mode according to the following priority order: tokens with a distance level of alert are given priority over tokens with a distance level of safe. When distance levels are the same, tokens with greater motion change characteristics are given priority; when motion change characteristics are the same, tokens with higher average confidence scores at key points are given priority. Furthermore, tokens with a missing proportion exceeding a preset missing proportion threshold will not be included in the mandatory retention process as anchor tokens for sensitive events.
[0011] In a preferred embodiment, the method further includes an online temporal consistency verification step: after each output of trajectory prediction results for several future steps, the consistency evaluation is performed using the human body center point actually observed at the next moment and the one-step prediction point in the trajectory prediction results. When the consistency is lower than a preset consistency threshold, at least one of the corridor disturbance ratio and the preset ratio threshold of the low confidence ratio is adaptively adjusted to reduce false triggering caused by long-term operation drift.
[0012] In a preferred embodiment, a human motion trajectory prediction system based on temporal Transformer, applied to a robotic system, includes: The acquisition module is used to acquire RGB image sequences and depth image sequences, and obtain robot pose information and current planned path information synchronized with them in time; The preprocessing module is used to perform human detection and pose estimation, output key point coordinates, key point confidence and missing mask, and take the geometric center of the key point coordinates that are not marked as missing by the missing mask as the human center point; When the number of available keypoints is less than 5, the human body center point from the previous moment is used. The corridor and segmentation module is used to generate dangerous corridors and warning corridors, and resample the planned path into 200 path points at equal intervals according to the travel arc length, and divide it into 10 path segments with 20 path points each. It also generates a segment version identifier for each path segment and calculates distance level information. The token building module is used to construct a dual-stream time-series token sequence of human posture tokens and path corridor tokens; The cache management module is used to establish a sliding time window and maintain the first key / value cache and the second key / value cache respectively. The segment version identifier is used as the cache key of the second key / value cache. When the segment version identifier changes or the distance level containing the danger level is switched, the second key / value cache entries of the corresponding path segment are invalidated and rebuilt only. At the same time, the first key / value is incrementally written to the human posture token. The Risk and Pruning module is used to stop pruning and retain all tokens within the window in high-risk mode, and to perform importance screening in low-risk mode. It identifies and forcibly retains sensitive event anchor tokens by checking the consistency of corridor disturbances through expanding the radius of dangerous corridors by 10% to 20% and shrinking the radius of warning corridors by 10% to 20%. At the same time, it forcibly retains the 6 most recent tokens in time and limits the total number of tokens after pruning to 60% of the sliding time window length. The trajectory prediction module is used to perform cross-stream interactive inference in at least the first interactive attention sublayer, using human pose token as query and path corridor token as key and value, and outputs a sequence of trajectory points for several future steps and corresponding progressive confidence information. The control decision module is used to output an emergency stop command when the predicted trajectory enters a dangerous corridor or a corridor crossing event occurs, output a speed limit command when the predicted trajectory enters a warning corridor but does not enter a dangerous corridor, and perform a rollback operation when the low confidence percentage exceeds a threshold.
[0013] In a preferred embodiment, the cache management module is configured to: when the segment version identifier of a certain path segment changes, only perform cache invalidation and reconstruction on the second Key / Value cache entry corresponding to the path segment, while keeping the second Key / Value cache entry corresponding to the unchanged path segment reused, and keeping the first Key / Value cache not reconstructed, so as to improve the real-time performance of inference in the scenario of local fine-tuning of the planned path.
[0014] In a preferred embodiment, the risk and trimming module is configured to: perform a corridor disturbance consistency check in a low-risk mode and use the change of the judgment label of "entering a dangerous corridor, entering a warning corridor, or experiencing a corridor crossing event" as the identification condition for the sensitive event anchor token; The control decision module is configured to output an emergency stop command and stop clipping when it detects two consecutive corridor crossing events and the confidence level of each step is not lower than a preset confidence threshold.
[0015] The technical effects and advantages of this invention are as follows: 1. By using dual-level constraints and distance level calculations for dangerous / warning corridors, and incorporating corridor tokens into cross-flow interaction modeling, the prediction results can directly trigger emergency stops / speed limits / recalculations, achieving a "prediction-control" closed loop, reducing the risk of missed stops and improving response speed.
[0016] 2. The planned path is resampled to 200 points and segmented to generate segment version identifiers. The second KV cache uses the segment version identifier as the key and only rebuilds the path segment when it fails locally. This achieves efficient reuse in the scenario of fine-tuning the plan and reduces the latency and peak memory usage on the edge.
[0017] 3. When the distance level switches between dangerous and non-dangerous, the cached entries for the path segment corresponding to the current prediction time range are invalidated and rebuilt to reduce misjudgments caused by inconsistencies between the corridor context and the current risk.
[0018] 4. By checking the consistency of corridor disturbances, identify tokens that are sensitive to the "entry / crossing" decision and force their retention. Combined with retaining the latest 6 tokens and a 60% retention rate, this ensures that critical events can still be reliably identified even in low-risk mode, reducing missed stops caused by mis-pruning.
[0019] 5. Key point confidence and missing mask are used in input representation and risk assessment. When there are insufficient available key points, the center point of the previous time step is used for compensation to enhance stability under occlusion, false detection and deep noise conditions.
[0020] 6. Trigger backoff by using a low confidence percentage threshold, and stop pruning or expand the window if necessary to improve the stability of subsequent predictions and avoid continuous unreliable outputs causing jitter in the control. Attached Figure Description
[0021] Figure 1 This is a diagram of a human motion trajectory prediction system based on temporal Transformer. Detailed Implementation
[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0023] For ease of understanding, the following terminology definitions and parameter constraints are provided (the examples may use some specific values; those not explicitly defined can be set according to industry conventions): Robot base coordinate system: A three-dimensional coordinate system with reference to the robot's base, denoted as... .
[0024] Temporal image data: Synchronous sampling results of RGB image sequences and depth image sequences; depth can come from an RGB-D camera or be generated by a depth estimation module.
[0025] Human skeleton key points: The set of key points output by human pose estimation, it is recommended to take 17 points / 25 points, etc.; each key point includes two-dimensional or three-dimensional coordinates and confidence level.
[0026] Missing mask: When the confidence level of a key point is below a threshold A key is marked as missing; the missing percentage is the proportion of missing keypoints to the total number of keypoints. Recommendation ∈[0.2,0.6].
[0027] Human body center point: formed by the absence of key points The geometric center is obtained from the previous moment; when there are insufficient available key points, the center point of the previous moment is used for compensation.
[0028] Path planning: The path output by the robot planning module can be represented as a sequence of path points or a spline; this invention performs arc-length equidistant resampling to 200 path points.
[0029] Path segmentation: Divide the 200 path points into 10 consecutive path segments of 20 points each.
[0030] Segment version identifier: A hash value / checksum (such as CRC, SHA digest, etc.) generated after encoding the coordinate sequence of 20 path points in each segment; used as the cache key for the second KV cache on the corridor side.
[0031] Dangerous corridor / warning corridor: A strip-shaped area constructed with the planned path as its centerline; the radius of the warning corridor is larger than that of the dangerous corridor. The radius can be determined according to the robot size and safety margin; a dangerous radius is recommended. Warning radius , .
[0032] Distance level: determined based on the relationship between the minimum distance from the center point of the human body to the planned path and the corridor radius; it should include at least "dangerous / non-dangerous (or warning / safe)".
[0033] The minimum distance between the human body and the planned path can be approximated by the minimum Euclidean distance from the center point of the human body to the resampled path point set. When higher accuracy is required, the point-to-segment distances of the line segments formed by adjacent path points can be further calculated, and the minimum value is taken. The distance level is determined based on the relationship between the minimum distance and the radius of the danger / warning corridor. The radius of the danger and warning corridor can be set by the robot's equivalent external dimensions and safety margin.
[0034] Sliding time window length W: Cache the most recent W time-to-time tokens, recommended. Examples of implementations may include 20, 24, etc.
[0035] Low-risk pruning retention ratio: The claim is hardcoded to retain a total of 60% of the window length.
[0036] Force the retention of the latest token count: The claim must be hardcoded to the latest 6 tokens.
[0037] Corridor disturbance consistency check: While keeping the human token unchanged, apply disturbances to the corridor tokens and perform a prediction for each token; the disturbances include: expanding the danger radius by 10% to 20% and shrinking the warning radius by 10% to 20%.
[0038] Prediction steps T: Outputs the trajectory points for the next T steps and the progressive confidence level; suggestions .
[0039] Stepwise confidence and low confidence percentage: Stepwise confidence is the output of the confidence level of each trajectory point at each step; low confidence percentage is the percentage of points that are below a certain threshold. The percentage of steps, it is recommended Percentage threshold .
[0040] Corridor crossing event: The preferred definition is that the predicted trajectory appears at adjacent points on both sides of the boundary of the dangerous corridor in two consecutive steps, and the confidence level of the two steps is not lower than the confidence threshold. (suggestion ).
[0041] Rollback operation: includes at least stopping the clipping (preserving the window token in its entirety) and / or increasing the window length W (e.g., expanding it to 24, 32, etc.).
[0042] Dangerous corridor radius radius of the warning corridor The robot's equivalent external radius The safety margin δ is determined, where , and > ;For example A depth of 0.3m to 0.8m is acceptable. Desirable +0.2m~0.6m; Distance levels are based on the minimum distance from the center point of the human body to the planned path. and , Comparatively confirmed: ≤ It is dangerous. < ≤ For alert purposes, the rest are for safety. The sliding window length W can be 16 to 32, the prediction steps T can be 8 to 30, and the stepwise confidence threshold and the low confidence percentage threshold can be 0.4 to 0.7 and 0.3 to 0.6 respectively, so that they can be configured according to safety requirements on different robot platforms.
[0043] Reference Figure 1 A human motion trajectory prediction method based on temporal Transformer, applied to robot systems, including: Step S1, Input Acquisition: Time-series image data is acquired by a vision sensor installed on the robot body. The time-series image data includes RGB image sequences and depth image sequences. Simultaneously, robot state data synchronized with the time-series image data is acquired, and the robot state data includes at least robot pose information and robot current planned path information; Output synchronized time-series image data and robot state data; Both the RGB image sequences and depth image sequences output by the vision sensor carry acquisition timestamps. The robot's state data carries a state timestamp. The system performs nearest neighbor matching in the buffer queue with a preset synchronization window Δt (e.g., 10ms to 50ms). - When |≤Δt, synchronization is considered successful and a joint sample frame is formed; when Δt is exceeded or a frame is lost, the valid state data of the previous moment is used first to maintain it, and the SyncFail flag is set when synchronization fails for N consecutive frames (e.g., 3 to 5 frames), and the control decision module enters a conservative strategy of rate limiting / recalculation to avoid misjudgment caused by asynchrony.
[0044] Step S2, Human pose and confidence information generation: Perform human detection and pose estimation frame by frame on the time-series image data to obtain the human skeleton key point sequence; Output the keypoint coordinates and keypoint confidence at each time step, and generate a missing mask based on the keypoint confidence threshold; The key point coordinates are transformed to the robot base coordinate system to obtain the human body relative pose features, and the geometric center of the key point coordinates that are not marked as missing by the missing mask is taken as the human body center point. When the key points are first output in pixel coordinates, the pixel point (u, v) and the depth value d are restored to the three-dimensional point Pc in the camera coordinate system by using the camera intrinsic parameters; then, the Pc is transformed into the base coordinate system point Pb = T_base_cam·Pc by using the extrinsic transformation matrix T_base_cam from the camera to the robot base; the extrinsic parameters can be obtained by factory calibration or online hand-eye calibration and stored in the parameter area; when the depth data is missing, the approximate depth can be given by the depth estimation module or the three-dimensional position can be approximately restored by using the fixed-height plane constraint, and at the same time, the missing ratio is increased to trigger a more conservative high-risk mode to ensure control safety.
[0045] When the number of available key points is less than 5 at a certain moment, the human body center point of the previous moment is used as the current human body center point; the human body center point, the key point confidence, and the missing mask are output. Human body detection and pose estimation can be implemented by using any model that can output the key points of the human body skeleton. The key point set can be 17 points or 25 points; the coordinates (which can be pixel coordinates or three-dimensional coordinates) and the confidence conf∈[0,1] are output for each key point. When conf < ConfThr (such as 0.2 - 0.5), the key point is set as missing in the missing mask Mask; the missing ratio is calculated as the number of missing key points / the total number of key points and is used for subsequent risk mode switching; the human body center point is calculated as the geometric center of the coordinates of the non-missing key points. If the number of available key points is less than 5, the center point of the previous moment is used to avoid trajectory jumps caused by occlusion / misdetection.
[0046] Step S3, generation of the two-level corridor and path segment version: Based on the planned path information, the planned path corridor features are constructed, including the dangerous corridor and the warning corridor, where the corridor radius of the warning corridor is greater than the corridor radius of the dangerous corridor. The planned path is resampled at equal intervals of the travel arc length into 200 path points, and the 200 path points are divided into 10 consecutive path segments according to every 20 path points. A segment version identifier is generated for each path segment, and the segment version identifier is composed of the hash value or checksum generated from the coordinate sequence of 20 path points within the path segment. The segment version identifier is obtained by encoding the coordinate sequence of 20 path points within the corresponding path segment and generating a fixed-length checksum; when any checksum changes, only the second Key / Value cache entry corresponding to the path segment is reconstructed, while the second Key / Value cache entries corresponding to the remaining path segments are reused to reduce the reconstruction overhead on the edge side. To ensure that the segment version identifier does not jitter due to minor noise, fixed-point quantization encoding can be performed on the path point coordinates first. For example, the coordinates can be rounded / truncated at a preset resolution. Then, the quantized coordinate sequence is concatenated into a byte string according to a fixed byte order. Finally, a CRC or hash digest is generated from the byte string as the segment version identifier. This method can generate stable and distinct identifiers when the path segment undergoes geometric changes, while suppressing invalid reconstructions caused by floating-point perturbations.
[0047] Calculate the minimum distance between the human body and the planned path based on the human body center point and the 200 path points, and determine the distance level information; Outputs information on dangerous corridors, warning corridors, distance levels, and segment version identifiers for 10 path segments; "Equidistant arc length resampling can be achieved by using the cumulative arc length parameterization method: calculate the distance between adjacent points on the planned path according to the original path point sequence and accumulate them to obtain an arc length array. Then, generate the target arc length position according to the target arc length interval, and obtain the corresponding coordinate point through linear interpolation or spline interpolation to obtain a fixed number of resampled path points."
[0048] When the effective length of the planned path is insufficient to cover the target arc length range, the end points can be repeatedly filled or the end segments can be interpolated and extrapolated to ensure that a fixed number of path points are obtained; if the planned path is unavailable or the interpolation fails, the PathInvalid flag is set and the control module is triggered to enter the conservative strategy of speed limiting / recalculation.
[0049] Step S4: Dual-stream temporal token construction: Construct a dual-stream temporal token sequence in chronological order, wherein the dual-stream temporal token sequence includes at least a human posture token and a path corridor token; The human pose token includes at least key point coordinates, key point confidence, missing mask, and motion change features obtained by difference between adjacent time points; The path corridor token includes at least a parameterized representation of the danger corridor and the warning corridor, distance level information, and a segment version identifier of at least one path segment corresponding to the current prediction time range; Output a dual-stream timing token sequence; Step S5, End-side streaming inference and local segment gated key / value caching: Establish a sliding time window on the robot to cache human pose tokens and path corridor tokens from recent moments; The attention layer of the temporal Transformer maintains a first Key / Value cache related to the human pose token and a second Key / Value cache related to the path corridor token, and uses the segment version identifier as the cache key of the second Key / Value cache; Based on the gating of planned path changes and risk changes, the second Key / Value cache is gated and updated, where: Planning path change gating: When it is detected that the segment version identifier of any path segment is inconsistent with the segment version identifier of the corresponding path segment in the previous period, cache invalidation and reconstruction are performed only on the second key / value cache entry corresponding to that path segment; Risk change gating: When the distance level information changes from non-dangerous to dangerous or from dangerous to non-dangerous, cache invalidation and reconstruction are performed only on the second Key / Value cache entry of the path segment corresponding to the current prediction time range; The “path segment corresponding to the current prediction time range” estimates the robot’s reachable distance within the prediction time range using the robot’s current speed and predicted step size, and identifies one or two path segments that intersect with the coverage of this route as the corresponding path segment, so that invalidation and reconstruction are performed only on the second Key / Value cache entry of the corresponding path segment. The second Key / Value cache is reused if none of the above-mentioned gating updates are triggered. In any case, only the newly entered human pose tokens within the sliding time window are incrementally generated with a first key / value and written to the first key / value cache, so as to reuse the first key / value corresponding to the historical human pose tokens within the window; Output the first Key / Value cache and the second Key / Value cache after partial reconstruction; The second Key / Value cache establishes entries with (SegVersion, prediction time range index) as the key. Each entry contains at least: the Key matrix and Value matrix encoded by the corridor token sequence of the corresponding path segment, and the entry status bit Valid. When a change in SegVersion is detected, the corresponding entry Valid is set to invalid and corridor token encoding and attention pre-computation are re-executed only for that path segment to reconstruct the Key / Value. When gating is not triggered, the Valid entry is directly reused to avoid full reconstruction. The first Key / Value cache only generates and writes incremental human pose tokens that newly enter the sliding window. The window slides out part according to the FIFO strategy, thereby ensuring that the memory on the edge is bounded.
[0050] The path segment corresponding to the predicted time range can be estimated based on the robot's current speed and control cycle, and the path segment covered by the projection of this route onto the planned path is determined as the corresponding segment. If the coverage area crosses the segment boundary, the two intersecting adjacent segments are selected as the corresponding segment; if the robot is stationary or its speed is unavailable, the current segment is selected by default as the corresponding segment.
[0051] Step S6, Risk Adaptive Pruning and Forced Retention of Sensitive Event Anchor Points: Determine the risk level based on distance level information, key point confidence, and missing mask, and switch between high-risk mode and low-risk mode; When the distance level is dangerous or the average confidence of key points is lower than the preset control threshold or the missing proportion is higher than the preset missing proportion threshold, the high-risk mode is entered. In the high-risk mode, pruning is stopped and all tokens within the sliding time window are retained to participate in the prediction. In the low-risk mode, the importance of tokens within the sliding time window is filtered to form a set of clipped tokens. The importance filtering is determined based on key point confidence, motion change characteristics and distance level information. In the low-risk mode, sensitive event anchor tokens are forcibly retained. The sensitive event anchor tokens are determined in the following way: human posture tokens within the same sliding time window remain unchanged, and only the path corridor tokens are subjected to corridor perturbation and prediction is performed once for each. The importance screening is performed in low-risk mode according to the following priority order: tokens with a distance level of alert are given priority over tokens with a distance level of safe. When distance levels are the same, tokens with greater motion change characteristics are given priority; when motion change characteristics are the same, tokens with higher average confidence scores at key points are given priority. Furthermore, tokens with a missing proportion exceeding a preset missing proportion threshold will not be included in the mandatory retention process as anchor tokens for sensitive events; The corridor disturbances include an expansion disturbance of the radius of the dangerous corridor by 10% to 20% and a contraction disturbance of the radius of the warning corridor by 10% to 20%. The predicted trajectories before and after the disturbance are respectively judged as "whether they enter the dangerous corridor, whether they enter the warning corridor, and whether a corridor crossing event occurs" and judgment labels are generated. For the trajectory point sequences obtained from the two predictions before and after the disturbance, calculate the minimum distance from each trajectory point to the planned path step by step. and according to and , The system generates entry danger / entry warning labels; it implements the determination of corridor crossing events by crossing danger boundaries in two consecutive steps as described in the claims; when two predictions differ on either label (e.g., one prediction of entry danger and the other of no entry danger, or one prediction of crossing and the other of no crossing), it considers the corresponding moment within the window to be sensitive to corridor parameters, marks the moment token as an anchor point and forcibly retains it, thereby improving the robustness of critical safety events.
[0052] The corridor crossing event includes: at least two consecutive steps in the predicted trajectory point sequence where adjacent trajectory points are located on both sides of the dangerous corridor boundary, and the stepwise confidence level corresponding to the at least two consecutive steps is not lower than a preset confidence threshold. To suppress misjudgments caused by single-step prediction noise, corridor crossing events can be verified using two consecutive steps: when two consecutive steps in the predicted trajectory point sequence show adjacent points located on both sides of the dangerous corridor boundary, and the confidence levels of the corresponding two steps are not lower than the confidence threshold, it is determined to be a corridor crossing event and an emergency stop and stop clipping back are triggered.
[0053] When the corridor crossing event is detected, an emergency stop command is directly output and the stop clipping in the rollback operation is triggered. When there is an inconsistency between the judgment labels before and after the disturbance, the token at the corresponding moment is marked as the sensitive event anchor token and forcibly retained; In low-risk mode, in addition to forcibly retaining sensitive event anchor tokens, the six most recent tokens within the sliding time window are also forcibly retained; and after the forced retention is completed, the total number of tokens to be pruned is limited to 60% of the length of the sliding time window. Output the set of tokens used for prediction; In low-risk mode, an importance score (Score) is calculated for each token within the window. The Score is composed of three weighted components: the mean confidence level of key points, the magnitude of movement change between adjacent time points, and the distance level (alertness is higher than safety). The TopK tokens are selected from highest to lowest Score to form a pruning set. Before performing TopK selection, the sensitive event anchor token and the 6 most recent tokens are added to the reserved set and locked. Then, the remaining tokens are used to fill the set to K. If the number of anchors and the latest tokens exceeds K, the anchors are truncated according to the rule that anchors take priority over the latest tokens and the latest tokens take priority over ordinary tokens, so as to ensure that the critical event identification is not lost due to pruning. In the corridor disturbance consistency check, the expansion of the dangerous corridor and the contraction of the warning corridor can be achieved using a relative proportional disturbance: the disturbance radius is obtained by multiplying the original radius by (1+α) or (1-β), where α and β are values within the range of 10% to 20%. Judgment labels include: whether any step of the predicted trajectory falls into the dangerous corridor, whether any step falls into the warning corridor, and whether a corridor crossing event is detected. If any label changes before or after the disturbance, the corresponding time-time token is considered sensitive to the safety event judgment and is marked as a sensitive event anchor token.
[0054] Step S7, Trajectory Prediction Output: Input the set of tokens used for prediction into a temporal Transformer to perform cross-flow interaction modeling of human pose flow and path corridor flow. The cross-flow interaction modeling includes performing interaction attention calculation in at least the first interaction attention sub-layer with human pose token as query and path corridor token as key and value. Output the sequence of trajectory points for the human body in the next few steps, along with the progressive confidence information corresponding to each trajectory point; Step S8, Control closed-loop output and backtracking: Generate robot control instructions based on trajectory point sequence, progressive confidence information and two-level corridor features, wherein: when the predicted trajectory enters a dangerous corridor or a corridor crossing event occurs, an emergency stop instruction is output; When the predicted trajectory enters the warning corridor but not the danger corridor, a speed limit command is output; otherwise, a path planning recalculation command or a motion maintenance command is output. The low confidence percentage of the progressive confidence information is calculated. When the low confidence percentage exceeds a preset percentage threshold, a rollback operation is performed. The rollback operation includes at least stopping the pruning and / or increasing the sliding time window length to improve the reliability of subsequent predictions. The control commands are output to the robot actuator.
[0055] It also includes an online time consistency verification step: after each output of trajectory prediction results for several future steps, the consistency assessment is performed using the human body center point obtained from the actual observation at the next moment and the one-step prediction point in the trajectory prediction results. When the consistency is lower than a preset consistency threshold, at least one of the corridor disturbance ratio and the preset ratio threshold of the low confidence ratio is adaptively adjusted to reduce false triggering caused by long-term operation drift.
[0056] The low-confidence percentage can be calculated as the proportion of steps with confidence levels below a threshold in the stepwise confidence sequence out of the total number of steps. When this proportion exceeds the threshold, a backoff is triggered: at least the low-risk mode is switched to the high-risk mode (clipping is stopped), and the sliding window length is increased to the preset safe window length to obtain more sufficient context in subsequent frames; at the same time, the control output frequency can be reduced or a planning recalculation can be triggered to enhance safety.
[0057] The step confidence StepConf is the confidence level output by the model for each step of the trajectory point. The low-confidence ratio is calculated as the number of steps where {StepConf < LowConfThr} divided by the prediction step number T. When the low-confidence ratio > LowConfRatioThr (e.g., 0.3 - 0.6), the FallbackFlag is set in the same control cycle, and the risk mode in the next cycle is forced to switch to the high-risk mode (stop cropping, retain all window tokens), and at the same time, the window length W is increased to the preset safety window length (e.g., increased from 20 to 32). The control instructions are sent to the actuator through the robot control interface. The emergency stop instruction has the highest priority, followed by speed limit, and then recalculation / holding, to ensure the certainty and safety of the control closed-loop.
[0058] A human motion trajectory prediction system based on a temporal Transformer, applied to a robot system, includes: An acquisition module, used to acquire RGB image sequences and depth image sequences, and obtain the robot pose information and the current planned path information synchronized with their time. A preprocessing module, used to perform human detection and pose estimation, output the key point coordinates, key point confidence levels, and missing masks, and use the geometric center of the key point coordinates not marked as missing by the missing mask as the human center point. When the number of available key points is less than 5, the human center point at the previous moment is adopted. A corridor and segmentation module, used to generate dangerous corridors and warning corridors, resample the planned path at equal intervals according to the travel arc length into 200 path points, divide them into 10 path segments every 20 path points, and generate a segment version identifier for each path segment and calculate the distance level information. A token construction module, used to construct a two-stream temporal token sequence of human pose tokens and path corridor tokens. A cache management module, used to establish a sliding time window and maintain the first Key / Value cache and the second Key / Value cache respectively, and use the segment version identifier as the cache key of the second Key / Value cache, and only invalidate and reconstruct the cache entries of the second Key / Value cache corresponding to the path segment when the segment version identifier changes or the distance level containing the danger level switches, and at the same time perform incremental writing of the first Key / Value for the human pose tokens. The cache management module is configured such that when the segment version identifier of a certain path segment changes, only the second key / value cache entry corresponding to that path segment is invalidated and rebuilt, while the second key / value cache entry corresponding to the unchanged path segment is reused, and the first key / value cache is not rebuilt, so as to improve the real-time performance of inference in the scenario of local fine-tuning of the planned path. The Risk and Pruning module is used to stop pruning and retain all tokens within the window in high-risk mode, and to perform importance screening in low-risk mode. It identifies and forcibly retains sensitive event anchor tokens by checking the consistency of corridor disturbances through expanding the radius of dangerous corridors by 10% to 20% and shrinking the radius of warning corridors by 10% to 20%. At the same time, it forcibly retains the 6 most recent tokens in time and limits the total number of tokens after pruning to 60% of the sliding time window length. The risk and trimming module is configured to perform a corridor disturbance consistency check in low-risk mode and use the change of the judgment label of "entering a dangerous corridor, entering a warning corridor, or experiencing a corridor crossing event" as the identification condition for the sensitive event anchor token. The control decision module is configured to output an emergency stop command and stop clipping when it detects two consecutive corridor crossing events and the corresponding confidence levels are not lower than a preset confidence threshold. The trajectory prediction module is used to perform cross-stream interactive inference in at least the first interactive attention sublayer, using human pose token as query and path corridor token as key and value, and outputs a sequence of trajectory points for several future steps and corresponding progressive confidence information. The control decision module is used to output an emergency stop command when the predicted trajectory enters a dangerous corridor or a corridor crossing event occurs, output a speed limit command when the predicted trajectory enters a warning corridor but does not enter a dangerous corridor, and perform a rollback operation when the low confidence percentage exceeds a threshold.
[0059] More specifically, the prediction system includes: Data Acquisition Module: Input: Camera trigger / timestamp; Output: RGB frames Depth frames Robot pose (Pose(t)) and planned path (Path(t)).
[0060] Preprocessing module (detection + attitude estimation + coordinate transformation): enter: , Pose(t); Processing: Human detection → Keypoint estimation → Confidence calculation → Missing mask generation → Keypoint projection / coordinate transformation ; Output: , Conf(t), Mask(t), ; Corridor and segmentation modules (resampling, segmentation, segment version identifier, distance level): Input: Path(t), ; Processing: Resample 200 points at equal intervals along the arc length → Divide into 10 segments of 20 points each → Generate Hash / CRC for each segment → Construct danger / warning corridors → Calculate minimum distance → Distance level; Output: , , Level(t), SegVersion1..10.
[0061] Token building block: Input: , Conf(t), Mask(t), , , Level(t), SegVersion(t); Output: Human posture token H(t), corridor token G(t).
[0062] Cache management module (KV cache, local segment gating): Input: H(t), G(t), SegVersion(t), Level(t); Handling: Maintenance and If only the corresponding segment is rebuilt after a segment version change. Entries; if the switch between dangerous and non-dangerous states occurs, only the segment corresponding to the current prediction time range will be reconstructed. entry; Incremental writes only.
[0063] Output: , (Including entry status); Risk and Pruning Module (High / Low Risk, Sensitivity Anchors, Latest 6, 60% Retention): Input: Level(t), Conf(t), Mask(t), WindowTokens; Output: SelectedTokens.
[0064] Trajectory prediction module (two-stream interactive Transformer): Input: SelectedTokens, , ; Processing: At least the first interaction attention sub-layer: Query = human token, Key / Value = corridor token; Output: TrajPred[1..T], StepConf[1..T].
[0065] Control decision module (emergency stop / speed limit / recalculation + rollback): Input: TrajPred, StepConf, Corridor_d / w, Level(t); Output: And FallbackFlag.
[0066] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A human motion trajectory prediction method based on temporal Transformer, applied to a robot system, characterized in that, include: Step S1, Input Acquisition: Time-series image data is acquired by a vision sensor installed on the robot body. The time-series image data includes RGB image sequences and depth image sequences. Simultaneously, robot state data synchronized with the time-series image data is acquired, and the robot state data includes at least robot pose information and robot current planned path information; Output synchronized time-series image data and robot state data; Step S2, Human pose and confidence information generation: Perform human detection and pose estimation frame by frame on the time-series image data to obtain the human skeleton key point sequence; Output the keypoint coordinates and keypoint confidence at each time step, and generate a missing mask based on the keypoint confidence threshold; The key point coordinates are transformed to the robot base coordinate system to obtain the human body relative pose features, and the geometric center of the key point coordinates that are not marked as missing by the missing mask is taken as the human body center point. When the number of available keypoints is less than 5 at a certain moment, the human body center point of the previous moment is used as the current human body center point. Output the confidence scores and missing mask for the human body center point and key points; Step S3, Generation of dual-level corridors and path segmentation versions: Based on the planned path information, construct the planned path corridor features, including dangerous corridors and warning corridors, wherein the corridor radius of the warning corridor is greater than that of the dangerous corridor; The planned path is resampled into 200 path points at equal intervals according to the travel arc length, and the 200 path points are divided into 10 continuous path segments of 20 each. For each path segment, a segment version identifier is generated, which is composed of a hash value or checksum generated from the coordinate sequence of 20 path points within the path segment. Calculate the minimum distance between the human body and the planned path based on the human body center point and the 200 path points, and determine the distance level information; Outputs information on dangerous corridors, warning corridors, distance levels, and segment version identifiers for 10 path segments; Step S4: Dual-stream temporal token construction: Construct a dual-stream temporal token sequence in chronological order, wherein the dual-stream temporal token sequence includes at least a human posture token and a path corridor token; The human pose token includes at least key point coordinates, key point confidence, missing mask, and motion change features obtained by difference between adjacent time points; The path corridor token includes at least a parameterized representation of the danger corridor and the warning corridor, distance level information, and a segment version identifier of at least one path segment corresponding to the current prediction time range; Output a dual-stream timing token sequence; Step S5, End-side streaming inference and local segment gated key / value caching: Establish a sliding time window on the robot to cache human pose tokens and path corridor tokens from recent moments; The attention layer of the temporal Transformer maintains a first Key / Value cache related to the human pose token and a second Key / Value cache related to the path corridor token, and uses the segment version identifier as the cache key of the second Key / Value cache; Based on the gating of planned path changes and risk changes, the second Key / Value cache is gated and updated, where: Planning path change gating: When it is detected that the segment version identifier of any path segment is inconsistent with the segment version identifier of the corresponding path segment in the previous period, cache invalidation and reconstruction are performed only on the second key / value cache entry corresponding to that path segment; Risk change gating: When the distance level information changes from non-dangerous to dangerous or from dangerous to non-dangerous, cache invalidation and reconstruction are performed only on the second Key / Value cache entry of the path segment corresponding to the current prediction time range; The second Key / Value cache is reused if none of the above-mentioned gating updates are triggered. In any case, only the newly entered human pose tokens within the sliding time window are incrementally generated with a first key / value and written to the first key / value cache, so as to reuse the first key / value corresponding to the historical human pose tokens within the window; Output the first Key / Value cache and the second Key / Value cache after partial reconstruction; Step S6, Risk Adaptive Pruning and Forced Retention of Sensitive Event Anchor Points: Determine the risk level based on distance level information, key point confidence, and missing mask, and switch between high-risk mode and low-risk mode; When the distance level is dangerous or the average confidence of key points is lower than the preset control threshold or the missing proportion is higher than the preset missing proportion threshold, the high-risk mode is entered. In the high-risk mode, pruning is stopped and all tokens within the sliding time window are retained to participate in the prediction. In the low-risk mode, the importance of tokens within the sliding time window is filtered to form a set of clipped tokens. The importance filtering is determined based on key point confidence, motion change characteristics and distance level information. In the low-risk mode, sensitive event anchor tokens are forcibly retained. The sensitive event anchor tokens are determined in the following way: human posture tokens within the same sliding time window remain unchanged, and corridor perturbation is performed only on path corridor tokens and prediction is performed once for each. The corridor perturbation includes expanding the radius of dangerous corridors by 10% to 20% and shrinking the radius of warning corridors by 10% to 20%. The predicted trajectories before and after the disturbance are respectively judged as "whether they enter the dangerous corridor, whether they enter the warning corridor, and whether a corridor crossing event occurs" and judgment labels are generated. When there is an inconsistency between the judgment labels before and after the disturbance, the token at the corresponding moment is marked as the sensitive event anchor token and forcibly retained; In low-risk mode, in addition to forcibly retaining sensitive event anchor tokens, the six most recent tokens within the sliding time window are also forcibly retained; and after the forced retention is completed, the total number of tokens to be pruned is limited to 60% of the length of the sliding time window. Output the set of tokens used for prediction; Step S7, Trajectory Prediction Output: Input the set of tokens used for prediction into a temporal Transformer to perform cross-flow interaction modeling of human pose flow and path corridor flow. The cross-flow interaction modeling includes performing interaction attention calculation in at least the first interaction attention sub-layer with human pose token as query and path corridor token as key and value. Output the sequence of trajectory points for the human body in the next few steps, along with the progressive confidence information corresponding to each trajectory point; Step S8, Control closed-loop output and backtracking: Generate robot control instructions based on trajectory point sequence, progressive confidence information and two-level corridor features, wherein: when the predicted trajectory enters a dangerous corridor or a corridor crossing event occurs, an emergency stop instruction is output; When the predicted trajectory enters the warning corridor but not the danger corridor, a speed limit command is output; otherwise, a path planning recalculation command or a motion maintenance command is output. The low confidence percentage of the progressive confidence information is calculated. When the low confidence percentage exceeds a preset percentage threshold, a rollback operation is performed. The rollback operation includes at least stopping the pruning and / or increasing the sliding time window length to improve the reliability of subsequent predictions. The control commands are output to the robot actuator.
2. The human motion trajectory prediction method based on temporal Transformer according to claim 1, characterized in that: The segment version identifier is obtained by encoding the coordinate sequence of 20 path points within the corresponding path segment and generating a fixed-length verification digest. When any verification digest changes, only the second Key / Value cache entry corresponding to that path segment is reconstructed, while the second Key / Value cache entries corresponding to the other path segments are reused to reduce the reconstruction overhead on the end side.
3. The human motion trajectory prediction method based on temporal Transformer according to claim 1, characterized in that: The "path segment corresponding to the current predicted time range" estimates the robot's reachable distance within the predicted time range using the robot's current speed and predicted step size, and identifies one or two path segments that intersect with the coverage of this distance as the corresponding path segment, so that invalidation and reconstruction are performed only on the second Key / Value cache entry of the corresponding path segment.
4. The human motion trajectory prediction method based on temporal Transformer according to claim 1, characterized in that: The corridor crossing event includes: at least two consecutive steps in the predicted trajectory point sequence where adjacent trajectory points are located on both sides of the dangerous corridor boundary, and the stepwise confidence level corresponding to the at least two consecutive steps is not lower than a preset confidence threshold. When the corridor crossing event is detected, an emergency stop command is directly output and the stop clipping in the rollback operation is triggered.
5. The human motion trajectory prediction method based on temporal Transformer according to claim 1, characterized in that: The importance screening is performed in low-risk mode according to the following priority order: tokens with a distance level of alert are given priority over tokens with a distance level of safe. When distance levels are the same, tokens with greater motion change characteristics are given priority; when motion change characteristics are the same, tokens with higher average confidence scores at key points are given priority. Furthermore, tokens with a missing proportion exceeding a preset missing proportion threshold will not be included in the mandatory retention process as anchor tokens for sensitive events.
6. The human motion trajectory prediction method based on temporal Transformer according to claim 1, characterized in that: It also includes an online time consistency verification step: after each output of trajectory prediction results for several future steps, the consistency assessment is performed using the human body center point obtained from the actual observation at the next moment and the one-step prediction point in the trajectory prediction results. When the consistency is lower than a preset consistency threshold, at least one of the corridor disturbance ratio and the preset ratio threshold of the low confidence ratio is adaptively adjusted to reduce false triggering caused by long-term operation drift.
7. A human motion trajectory prediction system based on temporal Transformer, applied to a robot system, characterized in that, include: The acquisition module is used to acquire RGB image sequences and depth image sequences, and obtain robot pose information and current planned path information synchronized with them in time; The preprocessing module is used to perform human detection and pose estimation, output key point coordinates, key point confidence and missing mask, and take the geometric center of the key point coordinates that are not marked as missing by the missing mask as the human center point; When the number of available keypoints is less than 5, the human body center point from the previous moment is used. The corridor and segmentation module is used to generate dangerous corridors and warning corridors, and resample the planned path into 200 path points at equal intervals according to the travel arc length, and divide it into 10 path segments with 20 path points each. It also generates a segment version identifier for each path segment and calculates distance level information. The token building module is used to construct a dual-stream time-series token sequence of human posture tokens and path corridor tokens; The cache management module is used to establish a sliding time window and maintain the first key / value cache and the second key / value cache respectively. The segment version identifier is used as the cache key of the second key / value cache. When the segment version identifier changes or the distance level containing the danger level is switched, the second key / value cache entries of the corresponding path segment are invalidated and rebuilt only. At the same time, the first key / value is incrementally written to the human posture token. The Risk and Pruning module is used to stop pruning and retain all tokens within the window in high-risk mode, and to perform importance screening in low-risk mode. It identifies and forcibly retains sensitive event anchor tokens by checking the consistency of corridor disturbances through expanding the radius of dangerous corridors by 10% to 20% and shrinking the radius of warning corridors by 10% to 20%. At the same time, it forcibly retains the 6 most recent tokens in time and limits the total number of tokens after pruning to 60% of the sliding time window length. The trajectory prediction module is used to perform cross-stream interactive inference in at least the first interactive attention sublayer, using human pose token as query and path corridor token as key and value, and outputs a sequence of trajectory points for several future steps and corresponding progressive confidence information. The control decision module is used to output an emergency stop command when the predicted trajectory enters a dangerous corridor or a corridor crossing event occurs, output a speed limit command when the predicted trajectory enters a warning corridor but does not enter a dangerous corridor, and perform a rollback operation when the low confidence percentage exceeds a threshold.
8. A human motion trajectory prediction system based on temporal Transformer according to claim 7, characterized in that: The cache management module is configured such that when the segment version identifier of a certain path segment changes, only the second key / value cache entry corresponding to that path segment is invalidated and rebuilt, while the second key / value cache entry corresponding to the unchanged path segment is reused, and the first key / value cache is not rebuilt, so as to improve the real-time performance of inference in the scenario of local fine-tuning of the planned path.
9. A human motion trajectory prediction system based on temporal Transformer according to claim 8, characterized in that: The risk and trimming module is configured to perform a corridor disturbance consistency check in low-risk mode and use the change of the judgment label of "entering a dangerous corridor, entering a warning corridor, or experiencing a corridor crossing event" as the identification condition for the sensitive event anchor token. The control decision module is configured to output an emergency stop command and stop clipping when it detects two consecutive corridor crossing events and the confidence level of each step is not lower than a preset confidence threshold.