Method and system for generating anthropomorphic sliding track

Abstract

The invention relates to the technical field of computers, and discloses an anthropomorphic sliding track generation method and system, and the method comprises the steps: obtaining video data containing human sliding operation, and carrying out the preprocessing, thereby obtaining a static image and scalar time; inputting the static image into an image encoder, inputting scalar time into a time encoder, respectively extracting space task and time constraint features, and fusing the space task and time constraint features into fusion condition features; the method comprises the following steps of: inputting a track point into a layered generator, outputting a current track point coordinate by a track generation head at each time step of the generator, and generating a track point sequence and a video frame sequence by a video generation head in combination with the coordinate, a fusion feature and a frame generated in a previous time step; and inputting the track and the video frame into a discriminator, and returning an authenticity result to adjust parameters of the generator after combined judgment by the discriminator until the output reaches the standard, thereby obtaining the anthropomorphic sliding track. According to the method, anthropomorphic similarity and sample diversity of track generation are improved, and a richer data set close to real human behaviors can be provided for downstream applications.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically, to a method and system for generating anthropomorphic sliding trajectories. Background Technology

[0002] With the development of internet security technology, sliding CAPTCHAs have become the mainstream verification method to distinguish human users from automated programs. Meanwhile, the demand for automated testing, penetration testing, and CAPTCHA mechanism research in the field of cybersecurity attack and defense is growing, making the generation of highly realistic sliding behavior data that simulates human operation a critical requirement. However, current technologies have significant shortcomings: traditional automated scripts generate sliding trajectories that are mostly uniform straight lines or simple Bézier curves, lacking the non-linear characteristics inherent in human sliding, such as speed changes, jitter, and slight hesitations, making them easily detectable by human-machine recognition models; the generation process often ignores the actual visual effect of the trajectory on the screen, and even if the trajectory is mathematically complex, the generated video may still be recognized due to a lack of realism; furthermore, it is difficult to dynamically adjust the trajectory speed curve according to given time constraints, making it impossible to generate videos that meet timing requirements. Therefore, there is an urgent need for an end-to-end method to generate highly human-like, visually realistic sliding videos from static scenes and time constraints.

[0003] In existing technologies, on the one hand, there are human-machine recognition and discrimination techniques based on manual feature extraction. These techniques employ a discriminative model approach: by collecting real trajectories, extracting multi-dimensional manual features such as speed, acceleration, angle, and jitter frequency, and then using models such as random forests and neural networks for classification and discrimination. This type of method highly relies on expert knowledge to design features. Once an attacker understands its feature system, they can specifically optimize the generation algorithm to bypass detection, leading to an arms race of "feature iteration" between attackers and defenders. On the other hand, there are offensive techniques represented by simple trajectory simulation techniques based on image processing and positioning. These techniques focus on locating sliders through image processing and simulating sliding with preset simple trajectories. Their generation capabilities are weak, and they cannot produce diverse and complex trajectories that conform to the statistical distribution of human behavior. Ultimately, the sliding trajectories generated by existing technologies have low anthropomorphic similarity and diversity, making it difficult to meet the needs of related fields for high-quality simulated data. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies in generating sliding trajectories with low anthropomorphic similarity and diversity, the present invention proposes the following technical solution: Firstly, this invention proposes a method for generating anthropomorphic sliding trajectories, comprising: Acquire video data containing human swiping actions, preprocess the video data to obtain a static image with start and end point markers, and a scalar time representing the total swiping duration; The static image is input to the image encoder to extract spatial task features, the scalar time is input to the time encoder to extract temporal constraint features, and the spatial task features and the temporal constraint features are fused to obtain fused condition features. The fusion condition features are input into a pre-constructed hierarchical generator. At each time step, the hierarchical generator first outputs the coordinates of the current trajectory point from the trajectory generation head, and then inputs the coordinates of the current trajectory point, the fusion condition features, and the generated frame from the previous time step into the video generation head to generate a sequence of trajectory points and a corresponding sequence of video frames. The trajectory point sequence and the video frame sequence are input into a pre-constructed discriminator. The discriminator performs joint judgment processing on the trajectory point sequence and the video frame sequence, outputs the authenticity judgment result, and returns the authenticity judgment result to the hierarchical generator to adjust the parameters until the trajectory sequence and video frame sequence output by the hierarchical generator reach the preset authenticity threshold under the discriminator, thus obtaining the anthropomorphic sliding trajectory.

[0005] As a preferred technical solution, video data containing human sliding operations is acquired, and the video data is preprocessed to obtain a static image with start and end point markers, and a scalar time representing the total sliding duration, including: The video data is processed by extracting video frames and decoding the original video into a series of independent image files at a fixed frame rate to obtain a video frame image sequence. The video frame image sequence is subjected to resolution normalization processing, and all video frame images are scaled to a uniform size to obtain a frame image sequence with uniform size; Finger position annotation is performed on a sequence of frame images of uniform size. First, a pre-trained hand keypoint detection model or lightweight object detection model is used to automatically predict the coordinates of the fingertip in each frame. Then, the coordinates of the fingertip in each frame are verified and adjusted to obtain the accurate coordinates of the fingertip at each time step. The starting and ending points of the sliding are determined based on the precise coordinates and marked on the corresponding frames to form a static image with the starting and ending point marks. Time information is extracted from the video data, the total duration of the original video is calculated, and the total duration is used as a scalar time representing the total sliding duration.

[0006] As a preferred technical solution, the fusion condition features are input into a pre-constructed hierarchical generator. At each time step, the hierarchical generator first outputs the coordinates of the current trajectory point from the trajectory generation head, and then inputs the current trajectory point coordinates, the fusion condition features, and the previously generated frame into the video generation head to generate a trajectory point sequence and a corresponding video frame sequence, including: The fusion conditional features are input into the Transformer decoder of the hierarchical generator. The Transformer decoder receives the output of the previous time step as input at each time step, focuses on the generated sequence through a self-attention mechanism, and extracts conditional information from the fusion conditional features through a cross-attention mechanism, and outputs the hidden state of the current time step. The trajectory generation head is derived from the hidden state of the current time step. The trajectory generation head processes the hidden state of the current time step through a linear layer, and then outputs the coordinates of the current trajectory point through an activation function. The current trajectory point coordinates, the fusion condition features, and the video frame generated in the previous time step are input into the video generation head. First, the hidden state, current trajectory point coordinates, and fusion condition features of the current time step are concatenated to form an initial vector. Then, a linear layer is used to map and reshape the initial vector into a spatial feature map. After upsampling the spatial feature map through several upsampling blocks, the video frame of the current time step is output through a convolutional layer. Based on the obtained trajectory point coordinates and video frames, generate a trajectory point sequence and a video frame sequence.

[0007] As a preferred technical solution, the trajectory point sequence and the video frame sequence are input into a pre-constructed discriminator. The discriminator performs joint judgment processing on the trajectory point sequence and the video frame sequence, outputs a authenticity judgment result, and returns the authenticity judgment result to the hierarchical generator for parameter adjustment, including: The video frame sequence is input into the discriminator's video processing path, then divided into three-dimensional blocks and input into the Transformer encoder. After processing by the Transformer encoder, the final output using a specific identifier token is used as a compact representation of the video. The trajectory point sequence is input into the trajectory processing path of the discriminator. The coordinates of each trajectory point are mapped to a high-dimensional space through a linear layer and a position encoding is added before being input into the Transformer encoder. The final output using a specific identifier token is used as a compact representation of the trajectory. The compact representation of the video and the compact representation of the trajectory are concatenated to obtain the concatenated feature vector; The concatenated feature vector is input into the fusion and classification head, and then processed through several linear layers, activation function layers and dropout layers. Finally, the output of the activation function represents the probability value that the input is true, which is used as the result of the true or false judgment. The true / false judgment result is returned to the hierarchical generator for parameter adjustment.

[0008] As a preferred technical solution, the loss function for adjusting the parameters of the hierarchical generator includes supervised pre-training loss. Total losses during the adversarial fine-tuning phase; in, Supervised pre-training loss The expression is as follows:

[0009] In the formula, For trajectory reconstruction loss, For the loss in video reconstruction, For perceptual loss based on deep neural network features, and These are the weighting coefficients; The total loss during the adversarial fine-tuning phase includes the discriminator loss. and generator loss Discriminator loss The expression is as follows:

[0010] In the formula, To determine the distribution of real data Find the mathematical expectation of the sample x in the sample. The output of the discriminator for the real sample x, To analyze the distribution of potential noise Find the mathematical expectation of the noise z in the given information. The generator produces fake samples by taking noise z as input. Generator loss The expression is as follows:

[0011] In the formula, For counter-loss, These are the weight coefficients of the pre-training loss. For pre-training loss, The weighting coefficients are those for the composite anthropomorphic regularization loss. The loss is a composite anthropomorphic regularization loss.

[0012] As a preferred technical solution, trajectory reconstruction loss The expression is as follows:

[0013] In the formula, The total length of the trajectory sequence. Let i be the coordinates of the i-th trajectory point generated. The coordinates of the i-th trajectory point are the actual coordinates. Video reconstruction loss The expression is as follows:

[0014] In the formula, This is a structural similarity function used to calculate the structural similarity between two images. For the generated i-th frame of video image, For the actual i-th frame of the video image; Perceptual loss based on deep neural network features The expression is as follows:

[0015] In the formula, This is the feature extraction function for a deep neural network.

[0016] As a preferred technical solution, composite anthropomorphic regularization loss The expression is as follows:

[0017] In the formula, These are the weighting coefficients for the speed pattern similarity loss. The weighting coefficients for the micro-jitter feature loss are... The weighting coefficient for the task completion loss. For speed pattern similarity loss, For micro-jitter feature loss, This represents a loss in task completion rate.

[0018] As a preferred technical solution, speed pattern similarity loss The expression is as follows:

[0019] In the formula, For dynamic time-normalized distance, To generate the velocity sequence of the trajectory, As a reference for the velocity sequence of human trajectories, The length of the generated velocity sequence; Micro-jitter feature loss The expression is as follows:

[0020] In the formula, The trajectory sequence generated for the model, For high-frequency feature extraction functions, The mean of the high-frequency features; Task completion loss The expression is as follows:

[0021] In the formula, These are hyperparameters used to control the intensity of the penalty. The last coordinate point of the generated trajectory , These are the actual, desired coordinates of the target point.

[0022] Secondly, the present invention also proposes an anthropomorphic sliding trajectory generation system, applied in the anthropomorphic sliding trajectory generation method as described in any embodiment of the first aspect, comprising: The preprocessing module is used to acquire video data containing human sliding operations, preprocess the video data to obtain a static image with start and end point markers, and a scalar time representing the total sliding duration. The feature extraction and fusion module is used to extract spatial task features from the static image input image encoder, extract temporal constraint features from the scalar time input time encoder, and fuse the spatial task features and the temporal constraint features to obtain fusion condition features; A hierarchical generator is used to generate a trajectory point sequence and a corresponding video frame sequence by first outputting the current trajectory point coordinates from the trajectory generation head at each time step based on the fusion condition features. The current trajectory point coordinates, the fusion condition features, and the generated frames from the previous time step are then input into the video generation head. The discriminator is used to perform joint judgment processing on the trajectory point sequence and the video frame sequence, output the authenticity judgment result, and return the authenticity judgment result to the hierarchical generator to adjust the parameters until the trajectory sequence and video frame sequence output by the hierarchical generator reach the preset authenticity threshold under the discriminator, thus obtaining the anthropomorphic sliding trajectory.

[0023] Thirdly, the present invention also proposes an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the operations performed by the anthropomorphic sliding trajectory generation method as described in any of the embodiments of the first aspect.

[0024] The beneficial effects of the present invention include at least the following: This invention preprocesses video data containing human sliding operations to obtain static images with start and end point markers and a scalar time representing the total sliding duration, providing a clear spatial task and temporal constraint basis for trajectory generation. Then, an image encoder and a temporal encoder extract spatial task features and temporal constraint features respectively, fusing them to obtain fusion condition features. At each time step, the hierarchical generator first outputs the current trajectory point coordinates from the trajectory generation head based on these fusion condition features. Then, combining the current trajectory point coordinates, the fusion condition features, and the frames generated in the previous time step, the video generation head generates the corresponding video frames. Simultaneously, a discriminator performs a joint authenticity judgment on the trajectory point sequence and video frame sequence, returning the result to the hierarchical generator to adjust parameters. This process iterates repeatedly until a preset threshold is met. This approach ensures that the generated sliding trajectories closely match real human sliding behavior in terms of path direction and speed changes. Furthermore, based on different spatial start and end points and time duration constraints, it can derive various trajectories and videos that conform to differences in human operation, effectively improving the anthropomorphic similarity and sample diversity of the generated sliding trajectories. This provides richer datasets that closely resemble real human behavior for downstream applications. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the anthropomorphic sliding trajectory generation method provided in an embodiment of the present invention.

[0026] Figure 2 This is an architecture diagram for model training provided in an embodiment of the present invention.

[0027] Figure 3 This is an architecture diagram of the anthropomorphic sliding trajectory generation system provided in an embodiment of the present invention.

[0028] Figure 4 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0029] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred technical solutions. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred technical solutions are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0030] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0031] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0032] Example 1 This embodiment proposes a method for generating anthropomorphic sliding trajectories, such as... Figure 1 As shown, Figure 1 This is a flowchart illustrating a method for generating anthropomorphic sliding trajectories provided in this embodiment. The method includes the following steps: S1: Acquire video data containing human sliding operations, preprocess the video data to obtain a static image with start and end point markers, and a scalar time representing the total sliding duration; S2: The static image is input to the image encoder to extract spatial task features, the scalar time is input to the time encoder to extract time constraint features, and the spatial task features and the time constraint features are fused to obtain fused condition features; S3: Input the fusion condition features into a pre-constructed hierarchical generator. At each time step, the hierarchical generator first outputs the current trajectory point coordinates from the trajectory generation head, and then inputs the current trajectory point coordinates, the fusion condition features, and the generated frames from the previous time step into the video generation head to generate a trajectory point sequence and a corresponding video frame sequence. S4: Input the trajectory point sequence and the video frame sequence into a pre-constructed discriminator. The discriminator performs joint judgment processing on the trajectory point sequence and the video frame sequence, outputs the authenticity judgment result, and returns the authenticity judgment result to the hierarchical generator to adjust the parameters until the trajectory sequence and video frame sequence output by the hierarchical generator reach the preset authenticity threshold under the discriminator, thus obtaining the anthropomorphic sliding trajectory.

[0033] Understandably, by preprocessing video data containing human sliding operations, static images with start and end point markers and a scalar time representing the total sliding duration are obtained, providing a clear spatial task and temporal constraint basis for trajectory generation. Then, the image encoder and temporal encoder extract spatial task features and temporal constraint features respectively, and fuse them to obtain fusion condition features. This allows the hierarchical generator to output the current trajectory point coordinates from the trajectory generation head at each time step based on these fusion condition features. Then, combining the current trajectory point coordinates, fusion condition features, and the frames generated in the previous time step, the video generation head generates the corresponding video frames. Simultaneously, a discriminator performs a joint authenticity judgment on the trajectory point sequence and video frame sequence, and returns the result to the hierarchical generator to adjust parameters. This process iterates repeatedly until a preset threshold is met. This approach ensures that the generated sliding trajectories closely match real human sliding behavior in terms of path direction and speed changes. Furthermore, based on different spatial start and end points and time duration constraints, it can derive various trajectories and videos that conform to differences in human operation, effectively improving the anthropomorphic similarity and sample diversity of sliding trajectory generation. This provides richer datasets that closely resemble real human behavior for downstream applications.

[0034] Example 2 This embodiment is an improvement on the anthropomorphic sliding trajectory generation method proposed in Embodiment 1.

[0035] In this embodiment, video data containing human swiping operations is acquired, and the video data is preprocessed to obtain a static image with start and end point markers, and a scalar time representing the total swiping duration, including: The video data is processed by extracting video frames and decoding the original video into a series of independent image files at a fixed frame rate to obtain a video frame image sequence. The video frame image sequence is subjected to resolution normalization processing, and all video frame images are scaled to a uniform size to obtain a frame image sequence with uniform size; Finger position annotation is performed on a sequence of frame images of uniform size. First, a pre-trained hand keypoint detection model or lightweight object detection model is used to automatically predict the coordinates of the fingertip in each frame. Then, the coordinates of the fingertip in each frame are verified and adjusted to obtain the accurate coordinates of the fingertip at each time step. The starting and ending points of the sliding are determined based on the precise coordinates and marked on the corresponding frames to form a static image with the starting and ending point marks. Time information is extracted from the video data, the total duration of the original video is calculated, and the total duration is used as a scalar time representing the total sliding duration.

[0036] As an example, in the specific implementation process, the collected video data containing human swiping operations is first decoded frame by frame at a fixed frame rate to obtain a set of image files arranged in chronological order. Then, using an image processing library, all frame images are scaled to a uniform size to form an image sequence of consistent size. Based on this, a pre-trained hand keypoint detection model or a lightweight object detection model finely tuned to the fingertip is used to automatically predict the fingertip pixel coordinates for each frame to obtain initial annotation results. Then, the inaccurate positions are manually verified and corrected by dragging through a graphical user interface to output the precise two-dimensional coordinates of the fingertip corresponding to each frame. Next, the obtained pixel coordinates are linearly mapped according to the image width and height so that the coordinate values ​​fall within the range of negative one to positive one, obtaining the coordinates corresponding to the image width and height. The process involves: 1) Decoupling the resolution of the normalized trajectory coordinate sequence; 2) Simultaneously calculating the total duration or total number of frames of the original video to obtain scalar temporal information; 3) Performing spatial enhancement on the image sequence and normalized trajectory coordinate sequence of consistent size, while maintaining consistency between the background image and frame image transformations, performing random cropping and scaling, random horizontal flipping, random rotation, and color dithering, and applying the same geometric affine transformation to the trajectory coordinates; 4) Performing temporal enhancement on the trajectory and frame sequences, simulating speed changes by randomly discarding or copying frames to adjust the total duration accordingly; 5) Obtaining the normalized, synchronized, and enhanced image sequence and trajectory coordinate sequence, which serve as input for the subsequent image encoder to extract spatial task features and the temporal encoder to extract temporal constraint features.

[0037] In this embodiment, the fusion condition features are input into a pre-constructed hierarchical generator. At each time step, the hierarchical generator first outputs the current trajectory point coordinates from the trajectory generation head, then inputs the current trajectory point coordinates, the fusion condition features, and the previously generated frames into the video generation head to generate a trajectory point sequence and a corresponding video frame sequence, including: The fusion conditional features are input into the Transformer decoder of the hierarchical generator. The Transformer decoder receives the output of the previous time step as input at each time step, focuses on the generated sequence through a self-attention mechanism, and extracts conditional information from the fusion conditional features through a cross-attention mechanism, and outputs the hidden state of the current time step. The trajectory generation head is derived from the hidden state of the current time step. The trajectory generation head processes the hidden state of the current time step through a linear layer, and then outputs the coordinates of the current trajectory point through an activation function. The current trajectory point coordinates, the fusion condition features, and the video frame generated in the previous time step are input into the video generation head. First, the hidden state, current trajectory point coordinates, and fusion condition features of the current time step are concatenated to form an initial vector. Then, a linear layer is used to map and reshape the initial vector into a spatial feature map. After upsampling the spatial feature map through several upsampling blocks, the video frame of the current time step is output through a convolutional layer. Based on the obtained trajectory point coordinates and video frames, generate a trajectory point sequence and a video frame sequence.

[0038] In this embodiment, the trajectory point sequence and the video frame sequence are input into a pre-constructed discriminator. The discriminator performs joint judgment processing on the trajectory point sequence and the video frame sequence, outputs a true / false judgment result, and returns the true / false judgment result to the hierarchical generator for parameter adjustment, including: The video frame sequence is input into the discriminator's video processing path, then divided into three-dimensional blocks and input into the Transformer encoder. After processing by the Transformer encoder, the final output using a specific identifier token is used as a compact representation of the video. The trajectory point sequence is input into the trajectory processing path of the discriminator. The coordinates of each trajectory point are mapped to a high-dimensional space through a linear layer and a position encoding is added before being input into the Transformer encoder. The final output using a specific identifier token is used as a compact representation of the trajectory. The compact representation of the video and the compact representation of the trajectory are concatenated to obtain the concatenated feature vector; The concatenated feature vector is input into the fusion and classification head, and then processed through several linear layers, activation function layers and dropout layers. Finally, the output of the activation function represents the probability value that the input is true, which is used as the result of the true or false judgment. The true / false judgment result is returned to the hierarchical generator for parameter adjustment.

[0039] In this embodiment, the loss function for adjusting the parameters of the hierarchical generator includes supervised pre-training loss. Total losses during the adversarial fine-tuning phase; in, Supervised pre-training loss The expression is as follows:

[0040] In the formula, For trajectory reconstruction loss, For the loss in video reconstruction, For perceptual loss based on deep neural network features; and These are weighting coefficients, and their proportional relationships constitute part of this invention, used to balance the learning priorities of the model among the three objectives of geometric accuracy, pixel fidelity, and advanced visual perception. The value range is [0.5, 2.0], with a recommended value of 1.0, used to balance trajectory accuracy and video pixel quality; The value range is [0.01, 0.1], and the recommended value is 0.05.

[0041] In this embodiment, trajectory reconstruction loss The expression is as follows:

[0042] In the formula, The total length of the trajectory sequence. Let i be the coordinates of the i-th trajectory point generated. Let be the coordinates of the actual i-th trajectory point.

[0043] It should be noted that the trajectory reconstruction loss This is used to ensure that the generated trajectory points are as close as possible to the real trajectory points in terms of geometric location. This represents the sequence of trajectory points generated by the model over a time series, where It is the two-dimensional coordinate point generated at the i-th time step. This represents the corresponding ground truth trajectory point sequence, where It is the actual two-dimensional coordinate point at the i-th time step. : Calculate the square of the Euclidean distance between two coordinate points, i.e.

[0044] Video reconstruction loss The expression is as follows:

[0045] In the formula, This is a structural similarity function used to calculate the structural similarity between two images. For the generated i-th frame of video image, This is the actual i-th frame of the video image.

[0046] It should be noted that video reconstruction loss This is used to ensure the basic visual quality of the generated video, making it close to real video at the pixel level. This represents the sequence of video frames generated by the model, where It is the image frame generated at the i-th time step. This represents the corresponding sequence of actual video frames. The function used to calculate the structural similarity between two images has a range of [-1, 1]. The closer the value is to 1, the more similar the two images are.

[0047] Perceptual loss based on deep neural network features The expression is as follows:

[0048] In the formula, This is the feature extraction function for a deep neural network.

[0049] It should be noted that video reconstruction loss Measuring the difference between generated and real images at higher-level semantic features is more effective than pixel-level loss in improving visual realism. This represents the sequence of video frames generated by the model, where It is the image frame generated at the i-th time step. This represents the corresponding sequence of actual video frames. Typically, it is a deep convolutional neural network pre-trained on a large image dataset. In this embodiment, we choose the intermediate layer of VGG-16, which maps the input image to a high-dimensional feature space.

[0050] The total loss during the adversarial fine-tuning phase includes the discriminator loss. and generator loss Discriminator loss The expression is as follows:

[0051] In the formula, To determine the distribution of real data Find the mathematical expectation of the sample x in the sample. The output of the discriminator for the real sample x, To analyze the distribution of potential noise Find the mathematical expectation of the noise z in the given information. The generator produces fake samples by taking noise z as input. It should be noted that during the training of the discriminator, a batch of samples x are drawn from the real dataset, and the generator G generates a batch of fake samples G(z) from the conditional input z. The goal of the discriminator D is to maximize... That is, let D(x) approach 1 (to determine that the real sample is true), let Approaching 0 (determining the sample to be fake).

[0052] Generator loss The expression is as follows:

[0053] In the formula, For counter-loss, These are the weight coefficients of the pre-training loss. For pre-training loss, The weighting coefficients are those for the composite anthropomorphic regularization loss. The loss is a composite anthropomorphic regularization loss.

[0054] Adversarial loss This loss is what drives the generator to produce outputs that can confuse the discriminator. It is the core driving force behind the generator's learning of complex, implicit features in the real data distribution. It measures how successfully the generator "fools" the discriminator.

[0055] In this embodiment, the composite anthropomorphic regularization loss is used. The expression is as follows:

[0056] In the formula, These are the weighting coefficients for the speed pattern similarity loss. The weighting coefficients for the micro-jitter feature loss are... The weighting coefficient for the task completion loss. For speed pattern similarity loss, For micro-jitter feature loss, This represents a loss in task completion rate.

[0057] In this embodiment, velocity pattern similarity loss The expression is as follows:

[0058] In the formula, For dynamic time-normalized distance, To generate the velocity sequence of the trajectory, As a reference for the velocity sequence of human trajectories, The length of the generated velocity sequence; It should be noted that the speed pattern similarity loss The dynamic time warping algorithm is used to penalize the discrepancy between the velocity curve of the generated trajectory and the statistical model of human behavior. Dynamic time warping is used to calculate the similarity between the velocity curve of the generated trajectory and the average velocity curve of the human sample database, ensuring the authenticity of macroscopic velocity change patterns (such as "slow at both ends, fast in the middle").

[0059] In the specific implementation process, firstly, the instantaneous velocity sequence is calculated based on the trajectory points. ,in Calculate the velocity sequence of the generated trajectory. and velocity sequences of reference human trajectories .

[0060] Micro-jitter feature loss The expression is as follows:

[0061] In the formula, The trajectory sequence generated for the model, For high-frequency feature extraction functions, The mean of the high-frequency features; It should be noted that the micro-jitter feature loss Frequency domain analysis encourages the generated trajectory's high-frequency components to conform to the characteristic distribution of natural human hand tremors. This is achieved through tremor frequency matching. Fourier transform analysis of the trajectory's high-frequency components is used to match the frequency range of natural human hand tremors. This loss function ensures the generated trajectory contains micro-tremors similar to the frequency of natural human hand tremors.

[0062] In practice, the x and y coordinate sequences of the trajectory are subjected to Fast Fourier Transform (FFT) respectively, and then the sum of energy or dominant frequency of the high-frequency components (e.g., frequencies above a certain threshold) is calculated. This formula calculates the absolute difference between the high-frequency characteristics of the generated trajectory and the statistical average of human behavior; the smaller the difference, the closer the shaking pattern is to that of a real person.

[0063] Task completion loss The expression is as follows:

[0064] In the formula, These are hyperparameters used to control the intensity of the penalty. The last coordinate point of the generated trajectory , These are the actual, desired coordinates of the target point.

[0065] It should be noted that, as Figure 2 As shown, Figure 2 The diagram shows the model training architecture provided in this embodiment of the invention. The core of this invention lies in the design of a hierarchical generator and a multimodal joint discriminator. The hierarchical structure of the generator decomposes the complex video generation task into two steps: controllable geometric path planning and visual rendering. The discriminator, on the other hand, provides collaborative supervision of the generation results from both motion and visual dimensions. Together, they constitute the technical foundation of this invention.

[0066] The training data for this invention originates from recordings of real human actions. Specifically, a large number of test subjects of different ages and genders are recruited to record screen activity on various mainstream devices (such as smartphones and tablets of different brands and sizes). The recordings mainly consist of repeating typical swiping operations such as swiping to unlock verification codes and unlocking phones using pattern recognition. This process aims to collect the most natural and diverse videos of human swiping behavior under different devices and gesture habits. In the data processing and enhancement stage, each frame of the recorded video is labeled using an automated target tracking algorithm supplemented by manual verification. The center coordinates (x, y) of the finger touch point and the corresponding timestamp t are extracted to form the original trajectory sequence. To improve the model's generalization ability and robustness, the collected data is also subjected to large-scale enhancement. Spatial enhancement includes random rotation of the video (simulating screen orientation switching), horizontal flipping (simulating left and right hand operations), scaling, and translation (simulating different screen resolutions and interaction areas). Temporal enhancement involves interpolating or extracting frames from the video to generate samples of different total durations while maintaining the trajectory shape.

[0067] The training method of this invention employs a two-stage course learning strategy. The first stage is a supervised pre-training stage, in which a discriminator is not used, only the generator is trained. The loss function mainly consists of reconstruction loss, enabling the model to learn the basic mapping relationship from input to output and acquire basic generation capabilities. The loss function L_pre aims to teach the model to generate basically correct trajectories and videos from images and time, making the generated trajectories and videos close to real data at the pixel and coordinate levels. The second stage is an adversarial fine-tuning stage, in which a discriminator is introduced to conduct adversarial training with the generator, engaging in an adversarial game between the generator and the discriminator to generate... The generator strives to generate more realistic data to "deceive" the discriminator, while the discriminator works to improve its discrimination ability. This process references the idea of ​​"initial camouflage -> detection challenge -> policy optimization -> iterative loop". During training, the discriminator gradually learns to pay attention to the "clues" of machine simulation, and the gradient information of these failed samples will be backpropagated to the generator, forcing it to optimize these "flaws" in the next iteration, thereby optimizing the generation quality. This dynamic game process drives the generator to spontaneously correct those easily identifiable "machine features", thus continuously approaching human behavior in microscopic details.

[0068] Example 3 This embodiment proposes an application of the anthropomorphic sliding trajectory generation method described in the above embodiments in the real-time detection of human-computer behavior and the replay of generated trajectories on the web page.

[0069] During real-time human-computer behavior detection at the front end, a front-end JavaScript script listens for mousedown, mousemove, and mouseup events (or touchstart, touchmove, and touchend events on the touchscreen) of user swipe operations in real time, capturing and recording trajectory points containing (x, y) coordinates and timestamps. After the swipe ends, the front end sends the collected complete trajectory data array to the back-end detection server in JSON format. The back-end server inputs the trajectory data into a deployed multimodal joint discriminator model. This discriminator, after training, can determine whether the trajectory sequence originates from a human or a machine, outputting a probability score of 0.0 to 1.0 (the higher the score, the closer it is to human operation). Then, based on this score and a preset threshold (such as 0.5), it determines whether the operation is legal. Finally, the back-end returns the "pass / fail" verification result to the front end, and the front end performs corresponding operations such as allowing login or requiring the user to retry based on the result.

[0070] When replaying the generated trajectory on the webpage, the trained generator model is first optimized (e.g., ONNX conversion, TensorRT acceleration, model quantization). For real-time applications, knowledge distillation is used to compress it into a lightweight version before deployment to the server or edge device. When an external application sends a request containing a static image with start and end markers and a time parameter (e.g., 1.8 seconds), the deployed service loads the model and performs a forward inference. The model simultaneously outputs the generated video frame sequence and a precisely synchronized trajectory sequence with timestamps [[x1,y1,t1],[x2,y2,t2],...]. Subsequently, the backend service synthesizes the frame sequence into a video file (e.g., MP4) or returns it as a stream. The system simultaneously sends the trajectory sequence to the front-end webpage in JSON format. After receiving the trajectory data, the front-end creates a movable simulated finger element (such as a dot or slider icon) on the page and uses requestAnimationFrame to drive the animation in a loop. In each frame, it checks the current time, finds the (x, y) coordinates of the corresponding time point from the trajectory array (performing linear interpolation if necessary to ensure smoothness), and then updates the CSS transform property of the simulated finger element (such as translate(x_px, y_px)) to move it to the new position. This process continues until the animation plays the entire trajectory sequence, thus accurately reproducing the sliding behavior generated by the model on the webpage with the correct timing.

[0071] Example 4 like Figure 3As shown, this embodiment proposes an anthropomorphic sliding trajectory generation system, which is applied to the anthropomorphic sliding trajectory generation method described in the above embodiment, including: a preprocessing module 100, a feature extraction and fusion module 200, a hierarchical generator 300, and a discriminator 400.

[0072] The preprocessing module 100 is used to acquire video data containing human sliding operations, preprocess the video data to obtain a static image with start and end point markers, and a scalar time representing the total sliding duration; the feature extraction and fusion module 200 is used to input the static image into an image encoder to extract spatial task features, input the scalar time into a time encoder to extract temporal constraint features, and fuse the spatial task features and the temporal constraint features to obtain fusion condition features; the hierarchical generator 300 is used to, based on the fusion condition features, first output the current trajectory point coordinates from the trajectory generation head at each time step, and then input the current trajectory point coordinates, the fusion condition features, and the previously generated frames into the video generation head to generate a trajectory point sequence and a corresponding video frame sequence; the discriminator 400 is used to perform joint judgment processing on the trajectory point sequence and the video frame sequence, output a true / false judgment result, and return the true / false judgment result to the hierarchical generator to adjust parameters until the trajectory sequence and video frame sequence output by the hierarchical generator reach a preset true / false threshold under the discriminator, thus obtaining a human-like sliding trajectory. It should be noted that the foregoing explanation of the anthropomorphic sliding trajectory generation method embodiment also applies to the anthropomorphic sliding trajectory generation system of this embodiment, and will not be repeated here.

[0073] Example 5 Figure 4 This is a schematic diagram of the structure of the electronic device 500 provided in this embodiment. The electronic device 500 includes: a memory 501, a processor 502, and a computer program stored in the memory 501 and executable on the processor 502.

[0074] When the processor 502 executes the program, it implements the anthropomorphic sliding trajectory generation method provided in the above embodiments.

[0075] Furthermore, the electronic device 500 also includes a communication interface 503 for communication between the memory 501 and the processor 502.

[0076] The memory 501 may include high-speed RAM (Random Access Memory) and may also include non-volatile memory, such as at least one disk storage device.

[0077] If the memory 501, processor 502, and communication interface 503 are implemented independently, then the communication interface 503, memory 501, and processor 502 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0078] Optionally, in a specific implementation, if the memory 501, processor 502, and communication interface 503 are integrated on a single chip, then the memory 501, processor 502, and communication interface 503 can communicate with each other through an internal interface.

[0079] Processor 502 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of the present invention.

[0080] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described anthropomorphic sliding trajectory generation method.

[0081] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0082] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0083] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of the invention pertain.

[0084] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0085] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium. When executed, the program includes one or a combination of the steps of the method embodiments.

[0086] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A humanoid sliding trajectory generation method characterized by, The method comprises the following steps: acquiring video data containing human sliding operation, pre-processing the video data to obtain static images with start point and end point markers, and a scalar time representing the total time length of the sliding; inputting the static images into an image encoder to extract spatial task features, inputting the scalar time into a time encoder to extract time constraint features, and fusing the spatial task features and the time constraint features to obtain fused conditional features; inputting the fused conditional features into a pre-constructed hierarchical generator, which outputs current trajectory point coordinates at each time step by a trajectory generation head, and then inputs the current trajectory point coordinates, the fused conditional features and the generated frame at the previous time step into a video generation head to generate a trajectory point sequence and a corresponding video frame sequence; inputting the trajectory point sequence and the video frame sequence into a pre-constructed discriminator, which performs joint judgment processing on the trajectory point sequence and the video frame sequence, outputs a true-false judgment result, and returns the true-false judgment result to the hierarchical generator for parameter adjustment until the trajectory sequence and the video frame sequence output by the hierarchical generator reach a preset true-false threshold under the discriminator, thereby obtaining a humanized sliding trajectory.

2. The anthropomorphic sliding trajectory generation method according to claim 1, characterized by, The method comprises the following steps: extracting video frames from the video data, decoding the original video into a series of independent image files at a fixed frame rate to obtain a video frame image sequence; performing resolution normalization processing on the video frame image sequence to scale all video frame images to a uniform size to obtain a size-uniform frame image sequence; annotating the finger positions of the size-uniform frame image sequence, automatically predicting the coordinates of the finger tips in each frame using a pre-trained hand key point detection model or a lightweight object detection model, and then adjusting the coordinates of the finger tips in each frame to obtain accurate coordinates of the finger tips at each time step; determining the start point and the end point of the sliding according to the accurate coordinates and marking them on the corresponding frames to form static images with start point and end point markers; extracting time information from the video data to calculate the total time length of the original video, and taking the total time length as a scalar time representing the total time length of the sliding.

3. The anthropomorphic sliding trajectory generation method according to claim 1, characterized by, The method comprises the following steps: inputting the fused conditional features into a pre-constructed hierarchical generator, which outputs current trajectory point coordinates at each time step by a trajectory generation head, and then inputs the current trajectory point coordinates, the fused conditional features and the generated frame at the previous time step into a video generation head to generate a trajectory point sequence and a corresponding video frame sequence, comprising: inputting the fused conditional features into a Transformer decoder of the hierarchical generator, which receives the output of the previous time step as input at each time step, focuses on the generated sequence through a self-attention mechanism, and extracts conditional information from the fused conditional features through a cross-attention mechanism to output the hidden state of the current time step; A trajectory generation head is branched from the hidden state of the current time step, the trajectory generation head processes the hidden state of the current time step through a linear layer, and then outputs the current trajectory point coordinates through an activation function; The current trajectory point coordinates, the fusion condition features, and the video frame generated at the previous time step are input into a video generation head, the hidden state of the current time step, the current trajectory point coordinates, and the fusion condition features are first spliced to form an initial vector, the initial vector is then mapped and reshaped into a spatial feature map using a linear layer, and the spatial feature map is upsampled through a plurality of upsampling blocks, and then a video frame at the current time step is output through a convolutional layer; According to the obtained trajectory point coordinates and video frames, a trajectory point sequence and a video frame sequence are generated.

4. The anthropomorphic sliding trajectory generation method according to claim 1, characterized by, The trajectory point sequence and the video frame sequence are input into a pre-constructed discriminator, the discriminator jointly judges the trajectory point sequence and the video frame sequence, outputs a true or false judgment result, and returns the true or false judgment result to the hierarchical generator for parameter adjustment, including: The video frame sequence is input into the video processing path of the discriminator, and after being input into the Transformer encoder after being divided into three dimensions, the final output of the specific token is used as the compact representation of the video; The trajectory point sequence is input into the trajectory processing path of the discriminator, and after being mapped to a high-dimensional space through a linear layer and adding position encoding, the final output of the specific token is used as the compact representation of the trajectory; The compact representation of the video and the compact representation of the trajectory are spliced to obtain a spliced feature vector; The spliced feature vector is input into a fusion and classification head, and is processed through a plurality of linear layers, activation function layers, and dropout layers in sequence, and then an output value representing the probability of being real is output through an activation function as a true or false judgment result; The true or false judgment result is returned to the hierarchical generator for parameter adjustment.

5. The anthropomorphic sliding trajectory generation method according to claim 4, characterized by, The loss function that adjusts parameters of the hierarchical generator includes a supervised pre-training loss and an adversarial fine-tuning phase total loss; Wherein, Supervised pre-training loss The expression for the supervised pre-training loss is as follows: wherein is a trajectory reconstruction loss, is a video reconstruction loss, is a perceptual loss based on deep neural network features, and is a weight coefficient; The total loss of the adversarial fine-tuning stage includes a discriminator loss and a generator loss The expression of the discriminator loss is as follows: wherein is the mathematical expectation of the real data distribution is the mathematical expectation of the sample x in the real data distribution is the output of the discriminator for the real sample x is the mathematical expectation of the latent noise distribution is the mathematical expectation of the noise z in the latent noise distribution is the fake sample generated by the generator with the noise z as input Generator loss The expression for the generator loss is given as follows: wherein is the adversarial loss, is a weight coefficient of the pre-training loss, is the pre-training loss, is a weight coefficient of the composite anthropomorphism regularization loss, is the composite anthropomorphism regularization loss.

6. The anthropomorphic sliding trajectory generation method according to claim 5, characterized by, Trajectory reconstruction loss The expression of the trajectory reconstruction loss Ltraj is given by: wherein is the total length of the trajectory sequence, is the coordinate of the i-th trajectory point generated, is the coordinate of the i-th real trajectory point. Video reconstruction loss The expression of the video reconstruction loss is as follows: wherein is a structural similarity function for computing a structural similarity between two images, is the i-th generated video image, is the i-th real video image; Perceptual loss based on deep neural network features The expression is as follows: In the formula, is a feature extraction function of a deep neural network.

7. The anthropomorphic sliding trajectory generation method according to claim 5, characterized by, Composite anthropomorphized regularization loss The expression of the loss is as follows: wherein, is a weight coefficient of the speed pattern similarity loss, is a weight coefficient of the micro-jitter feature loss, is a weight coefficient of the task completion degree loss, is the speed pattern similarity loss, is the micro-jitter feature loss, is the task completion degree loss.

8. The anthropomorphic sliding trajectory generation method according to claim 7, characterized by, Speed pattern similarity loss The expression of the speed pattern similarity loss is as follows: wherein is a dynamic time warping distance, is a velocity sequence of the generated trajectory, is a velocity sequence of the reference human trajectory, is a length of the generated velocity sequence; Microscopic jitter feature loss The expression for the loss is given as follows: In the formula, a sequence of trajectories generated for the model, is a high-frequency feature extraction function, is a mean of the high-frequency features; task completion loss The expression of the task completion loss is as follows: wherein is a hyperparameter for controlling the strength of the penalty, is the last coordinate point of the trajectory generated , is the real, desired target point coordinate.

9. A humanoid sliding trajectory generation system characterized by, including: A preprocessing module is configured to obtain video data containing human sliding operations, pre-process the video data, obtain static images with start and end point markers, and a scalar time representing the total sliding time; A feature extraction and fusion module is configured to input the static images into an image encoder to extract spatial task features, input the scalar time into a time encoder to extract time constraint features, and fuse the spatial task features and the time constraint features to obtain fusion condition features; A hierarchical generator is configured to generate a trajectory point sequence and a corresponding video frame sequence based on the fusion condition features, output current trajectory point coordinates at each time step by a trajectory generation head, and input the current trajectory point coordinates, the fusion condition features, and the generated frame at the previous time step into a video generation head. A discriminator is configured to jointly judge the trajectory point sequence and the video frame sequence, output a judgment result, and return the judgment result to the hierarchical generator to adjust parameters until the trajectory sequence and the video frame sequence output by the hierarchical generator reach a preset threshold under the discriminator, and obtain the humanized sliding trajectory.

10. An electronic device, comprising: The control device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the operations performed by the humanized sliding trajectory generation method according to any one of claims 1 to 8 when executing the computer program.

Images