Data sequence generation method and device, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a data sequence generation method, device, equipment and medium, comprising: acquiring initial data, extracting depth features, generating a depth feature map, fusing the initial data and the depth feature map to obtain a joint feature representation, generating an initial data unit based on the joint feature representation, setting a current data unit and a preorder data unit thereof, generating a motion complexity feature, generating a dynamic threshold mask, executing sparse attention processing by applying the dynamic threshold mask, generating a next data unit, updating the preorder data unit and the current data unit, and circulating the process. And performing time sequence processing by combining the initial data unit and the plurality of iterative data units to form a data sequence. According to the method, the attention sparse structure is dynamically controlled by combining the depth feature and the motion complexity feature, and the calculation efficiency and the time sequence continuity in the data sequence generation process are improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a data sequence generation method, apparatus, device, and storage medium. Background Technology

[0002] Against the backdrop of the continuous development of image-to-video (I2V) generation technology, deep learning-based generative models, especially those represented by the Transformer architecture, have become an important direction for improving video generation quality due to their powerful spatiotemporal information modeling capabilities. The Transformer effectively captures long-range spatiotemporal dependencies in image sequences through its self-attention mechanism, greatly enhancing the coherence and visual realism between video frames, and is widely used in content generation, intelligent synthesis, and other fields. However, existing Transformer-based image-to-video generation methods still have significant shortcomings in terms of computational efficiency. Specifically, when processing high-resolution or long sequence inputs, the computational overhead of the standard self-attention mechanism increases quadratically with the sequence length, severely limiting its applicability in scenarios with limited computing resources.

[0003] In the fintech sector, image-to-video generation technology is being explored for applications such as intelligent risk control, virtual customer service, and marketing visualization. For example, videos synthesized from image information can help showcase complex financial product logic or dynamically demonstrate risk warning information. However, financial business scenarios typically have stringent requirements for real-time performance and deployment costs. Existing technologies, especially for high-resolution video generation or long-term output, consume significant computational resources and suffer from high latency, making it difficult to meet the dual demands of efficiency and performance in financial systems. Furthermore, existing sparse attention technologies often rely on fixed sparse structures or simple strategies, lacking the ability to adapt to dynamically changing content in financial scenarios and failing to achieve flexible allocation and optimization of computational resources.

[0004] In the healthcare field, image-to-video generation methods are applied to virtual surgical demonstrations, remote diagnosis and treatment assistance, and dynamic reconstruction of medical images. Medical applications demand extremely high continuity and detail fidelity in video generation, while being constrained by device computing power and system response speed. Although existing technologies have introduced sparse attention mechanisms to alleviate computational pressure, most methods do not consider the dynamic changes in scene structure and motion features within medical images. Especially when faced with medical image inputs of varying complexity, they cannot effectively adjust the sparse strategy of attention computation, leading to decreased consistency in the generated results or unstable real-time system performance. Summary of the Invention

[0005] The main objective of this invention is to provide a data sequence generation method, apparatus, device, and storage medium, aiming to solve the technical problem that the existing technology lacks a mechanism for dynamically and jointly controlling the sparse attention computing structure based on the deep features and motion complexity of the input data, and thus cannot achieve an adaptive balance between computational efficiency and temporal coherence in the generation process.

[0006] To achieve the above objectives, the present invention provides a data sequence generation method, comprising:

[0007] Acquire initial data and extract the depth features from the initial data to obtain a depth feature map;

[0008] The initial data and the deep feature map are fused to generate a joint feature representation;

[0009] A starting data unit is generated based on the joint feature representation, and the current data unit and the preceding data unit of the current data unit are set as the starting data unit;

[0010] Based on the current data unit and the preceding data unit of the current data unit, generate motion complexity features;

[0011] Based on the depth feature map and the motion complexity feature, a dynamic threshold mask is generated;

[0012] Apply the dynamic threshold mask to the attention mechanism to perform sparse attention processing and generate the next data unit;

[0013] Update the preceding data unit of the current data unit to the current data unit, and update the current data unit to the next data unit;

[0014] Repeatedly execute the steps of generating motion complexity features based on the current data unit and its predecessor data unit, generating a dynamic threshold mask based on the motion complexity features, applying the dynamic threshold mask to perform sparse attention processing to generate the next data unit, and updating the predecessor data unit and the current data unit to generate multiple iterative data units;

[0015] The initial data unit and the plurality of iterative data units are subjected to time-series processing to form a data sequence.

[0016] Furthermore, to achieve the above objectives, the present invention provides a data sequence generation apparatus, comprising:

[0017] The deep feature extraction module is used to acquire initial data and extract the depth features of the initial data to obtain a depth feature map;

[0018] The feature fusion module is used to fuse the initial data and the deep feature map to generate a joint feature representation;

[0019] The starting unit generation module is used to generate a starting data unit based on the joint feature representation, and to set the current data unit and the preceding data unit of the current data unit as the starting data unit;

[0020] The motion feature analysis module is used to generate motion complexity features based on the current data unit and the preceding data units of the current data unit;

[0021] A dynamic threshold generation module is used to generate a dynamic threshold mask based on the depth feature map and the motion complexity feature;

[0022] The sparse attention inference module is used to apply the dynamic threshold mask to the attention mechanism, perform sparse attention processing, and generate the next data unit.

[0023] The data unit update module is used to update the preceding data unit of the current data unit to the current data unit, and update the current data unit to the next data unit;

[0024] The iterative sequence construction module is used to repeatedly execute the following steps: generating motion complexity features based on the current data unit and its predecessor data unit; generating a dynamic threshold mask based on the motion complexity features; applying the dynamic threshold mask to perform sparse attention processing to generate the next data unit; and updating the predecessor data unit and the current data unit, thereby generating multiple iterative data units.

[0025] The time sequence generation module is used to perform time sequence processing on the starting data unit and the multiple iterative data units to form a data sequence.

[0026] Furthermore, to achieve the above objectives, the present invention also provides a computer device, the computer device including a memory, a processor, and a data sequence generation program stored in the memory and executable on the processor, wherein when the data sequence generation program is executed by the processor, it implements the steps of the data sequence generation method as described above.

[0027] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a data sequence generation program, which, when executed by a processor, implements the steps of the data sequence generation method described above.

[0028] Beneficial Effects: This invention relates to the field of artificial intelligence technology and can be applied to business scenarios such as fintech and healthcare. It discloses a data sequence generation method, apparatus, device, and medium, comprising: acquiring initial data; extracting depth features from the initial data; obtaining a depth feature map; fusing the initial data and the depth feature map to generate a joint feature representation; generating a starting data unit based on the joint feature representation; setting the current data unit and its preceding data unit as the starting data unit; generating motion complexity features based on the current data unit and its preceding data unit; generating a dynamic threshold mask based on the depth feature map and the motion complexity features; applying the dynamic threshold mask to an attention mechanism; performing sparse attention processing; generating the next data unit; updating the preceding data unit and the current data unit; repeatedly executing the steps of generating motion complexity features, generating a dynamic threshold mask, performing sparse attention processing, and updating data units to generate multiple iterative data units; and performing temporal processing on the starting data unit and the multiple iterative data units to form a data sequence. This invention dynamically generates a threshold mask by jointly considering depth features and motion complexity features, and applies a sparse attention mechanism to the data generation process. It can dynamically optimize the sparse structure of attention calculation without reducing the quality and continuity of the data sequence, thus significantly improving computational efficiency and system stability. Attached Figure Description

[0029] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0030] Figure 1 This is a schematic diagram of an application environment for a data sequence generation method according to an embodiment of the present invention;

[0031] Figure 2 This is a flowchart illustrating an embodiment of the data sequence generation method of the present invention;

[0032] Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of the data sequence generation device of the present invention;

[0033] Figure 4 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention;

[0034] Figure 5 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0035] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0036] The data sequence generation method provided in this embodiment of the invention can be applied to, for example... Figure 1In this application environment, the user terminal communicates with the server via a network. The server can obtain initial data from the user terminal, extract the depth features of the initial data to obtain a depth feature map, fuse the initial data and the depth feature map to generate a joint feature representation, generate a starting data unit based on the joint feature representation, and set the current data unit and its predecessor data unit as the starting data unit. Based on the current data unit and its predecessor data unit, a motion complexity feature is generated. Based on the depth feature map and the motion complexity feature, a dynamic threshold mask is generated. The dynamic threshold mask is applied to an attention mechanism to perform sparse attention processing, generate the next data unit, update the predecessor data unit and the current data unit, and repeat the steps of generating motion complexity features, generating a dynamic threshold mask, performing sparse attention processing, and updating data units to generate multiple iterative data units. The starting data unit and multiple iterative data units are then subjected to temporal processing to form a data sequence. This invention dynamically generates a threshold mask by jointly considering depth features and motion complexity features, applying a sparse attention mechanism to the data generation process. This dynamically optimizes the sparse structure of attention computation without compromising data sequence quality and continuity, significantly improving computational efficiency and system stability. The user end can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server end can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention is described in detail below through specific embodiments.

[0037] Please see Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the data sequence generation method provided by the present invention. It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0038] like Figure 2 As shown, the data sequence generation method proposed in this invention includes the following steps:

[0039] S10, Obtain initial data and extract the depth features of the initial data to obtain a depth feature map;

[0040] In this embodiment, the process of acquiring initial data refers to receiving raw data input from external sources using an information access module or data acquisition system. This initial data can be image data, video data, depth maps, medical images, remote sensing images, or a dataset containing spatial information. The type of initial data is not limited to a specific format; it can be two-dimensional images, three-dimensional point clouds, video frame sequences, volumetric data, or other information sets with spatial structure. Specific acquisition methods include acquiring image information through image acquisition devices, acquiring spatial point cloud information through a sensing system, or importing existing data resources through a data interface. In practical applications, depending on the business scenario, the devices for acquiring initial data can include image sensors, LiDAR, depth cameras, ultrasound equipment, remote sensing satellites, financial information entry systems, or other terminal devices with data acquisition capabilities. To ensure the stability of subsequent processing, the acquired initial data can be temporarily stored through a caching module or data buffer.

[0041] Extracting deep features from initial data specifically refers to extracting structural, spatial, or semantic information from the acquired initial data using feature extraction networks or feature processing units. The meaning of deep features not only refers to the traditional depth estimation result but can also include multi-scale spatial structural information, edge structure information, 3D reconstruction information, or other data representations reflecting spatial relationships. Implementation methods include using multi-layer convolutional networks, residual neural networks, encoder-decoder structures, or Transformer-based encoding modules to perform feature transformations on the initial data. The output deep features are typically tensor data with multi-dimensional structures, and the data dimensions can be dynamically adjusted according to different implementation requirements, such as two-dimensional feature maps, three-dimensional feature tensors, or multi-channel feature matrices. During the deep feature extraction process, spatial pyramid pooling, attention mechanisms, skip connections, and other structures can be combined to enhance the richness of feature representation and spatial hierarchical information.

[0042] Obtaining a deep feature map refers to the process of representing extracted deep features in graph or matrix form through feature organization, dimensionality adjustment, or multi-scale fusion operations. This forms a structured information carrier that facilitates subsequent spatial processing and joint feature analysis. Deep feature maps can include single-scale feature maps, multi-scale fused feature maps, or hierarchical sets of spatial features. Their generation process can involve feature fusion operations at different scales, such as concatenation, multi-channel stitching, feature resampling, or attention weighting mechanisms, to integrate deep feature information from different levels and scales. The structure of a deep feature map can be a two-dimensional matrix, a three-dimensional tensor, or a four-dimensional dataset, depending on the dimensionality of the input data and the design requirements of the network structure. In practical applications, deep feature maps can possess spatial location encoding, scale adaptation capabilities, or cross-modal information fusion capabilities to enhance the representation of spatial information.

[0043] Initial data acquisition can be accomplished through various devices. Depending on the application requirements, image data can be acquired using imaging systems mounted on industrial cameras, consumer cameras, mobile terminals, remote sensing satellites, or drones. 3D point cloud data can be acquired using LiDAR, structured light depth cameras, time-of-flight (ToF) depth sensors, or multi-view stereo reconstruction systems. Video data can be acquired using video surveillance systems, medical image sequence acquisition equipment, or other systems with dynamic data acquisition capabilities. Data import methods are applicable to scenarios with existing data, enabling the import of initial data through network transmission, storage media import, or data sharing platforms.

[0044] In the process of deep feature extraction, spatial hierarchical features can be extracted layer by layer using convolutional neural networks. Specifically, this includes using standard convolutional layers to extract edge and basic texture information, and deep network structures to extract high-level semantic and spatial structural information. Combined with residual connections and batch normalization, the stability and information retention of feature representations are improved. For video data, temporal convolution, 3D convolution, or time-embedded feature encoding methods can be used to extract deep features under dynamic information, taking into account the temporal dimension. For point cloud data, 3D spatial features can be extracted using point cloud feature encoding networks, graph neural networks, or spatial convolutional structures. The output dimension of deep features can be flexibly configured to meet different business needs, for example, through channel adjustment layers, feature compression layers, or feature upsampling structures, adapting to different resolutions and information representation requirements.

[0045] In obtaining deep feature maps, a multi-scale fusion strategy can be employed. This includes jointly fusing spatial detail information from low-level features with structural information from high-level features, preserving spatial localization information through skip connections, and enhancing feature representation in important regions through attention mechanisms. The generation process of deep feature maps can also incorporate positional encoding mechanisms to enhance the ability to represent the relative positional information of spatial features. Regarding the organization of the feature map structure, feature arrangement, dimensionality adjustment operations, spatial resampling, or feature mapping transformations can be used to ensure that the output deep feature maps have a unified data format and good spatial representation capabilities, meeting the structural requirements for subsequent joint feature fusion and dynamic mask generation.

[0046] Example Explanation: In the healthcare business field, the process of acquiring initial data can be applied to medical image analysis scenarios. Specifically, this includes collecting medical image data of patients through CT scanning equipment, MRI imaging systems, ultrasound examination equipment, etc. Targeting the complex anatomical structures and spatial hierarchical information in medical images, depth feature extraction operations are performed to obtain depth feature maps that reflect the relationships between organ structure, tissue hierarchy, and spatial distribution. This facilitates subsequent operations such as 3D reconstruction, lesion detection, and pathological region analysis, thereby improving the accurate representation of spatial information.

[0047] In the fintech business sector, the process of acquiring initial data can be applied to scenario reconstruction and risk monitoring. Specifically, this includes acquiring environmental data or user operation data of business premises through video surveillance systems, remote sensing devices, or 3D imaging systems. Combined with deep feature extraction and deep feature map generation processes, a spatial information expression structure reflecting spatial layout, user behavior, or risk event correlations can be quickly constructed. This facilitates subsequent operations such as site security detection, abnormal behavior identification, and spatial structure analysis, thereby enhancing spatial perception and dynamic monitoring capabilities in financial business scenarios.

[0048] This embodiment effectively captures spatial hierarchy information and structural features in the input data by acquiring initial data and extracting corresponding depth features, enhancing the ability to express spatial structure, depth relationships, and spatial layout in complex scenes. During the acquisition of the depth feature map, multi-scale fusion, positional encoding, and spatial structure optimization operations further improve the completeness and stability of the depth feature map at different scales and spatial locations, facilitating subsequent spatial feature fusion, dynamic feature analysis, and sparse computation optimization operations.

[0049] S20, perform feature fusion on the initial data and the depth feature map to generate a joint feature representation;

[0050] In this embodiment, feature fusion of initial data and depth feature maps refers to forming a joint feature representation with unified expressive capabilities based on data content from different sources through structural adjustments, feature mapping, and information integration operations. Initial data refers to the acquired raw input information, which may include image data, video data, point cloud data, 3D reconstruction data, remote sensing images, or other data sets reflecting spatial scenes, structural layouts, or object states. Specifically, it can be a two-dimensional matrix, a three-dimensional tensor, multi-channel data, or a dataset with structural attributes. Depth feature maps refer to multi-scale spatial structural information obtained through the depth information extraction process. They are typically in the form of a matrix structure, tensor structure, or a hierarchically organized data set, containing spatial location information, depth distribution information, or structural hierarchy information.

[0051] Feature fusion involves several steps. First, channel dimension expansion is performed on the initial data. This involves expanding single-channel or low-channel data into a multi-channel form through channel duplication, feature mapping, or parameter transformation, ensuring consistency with the depth feature map in data dimension. This facilitates subsequent stitching and joint representation. The expansion operation can be achieved through repeated padding, zero padding, or mapping with specific weight parameters. Second, spatial alignment is performed on the depth feature map to ensure that its spatial resolution and size specifications are consistent with the expanded initial data. Spatial alignment can employ upsampling, downsampling, interpolation resampling, or spatial calibration operations based on geometric transformations to guarantee the correspondence between the two sets of data in spatial location and size structure.

[0052] After spatial alignment, the expanded initial data and the aligned depth feature map are combined along the channel or feature dimension through feature concatenation to form a concatenated feature matrix. This concatenation operation not only integrates data from different sources but also preserves their respective structural characteristics, enhancing the diversity and completeness of data representation. To further unify feature representation and numerical distribution, a feature transformation network is used to process the concatenated feature matrix. This network can employ convolutional neural networks, feature mapping networks, or parameter transformation networks, specifically including structures such as convolutional layers, nonlinear activation layers, feature normalization layers, and linear transformation layers. The aim is to optimize the feature representation structure through multiple transformations, adjust information distribution and expression forms, and enhance the feature fusion effect.

[0053] After feature transformation is completed, the transformed feature representation is processed through a feature normalization layer. Feature normalization operations include batch normalization, layer normalization, instance normalization, or other normalization strategies. The purpose is to reduce the numerical differences between different data sources, alleviate gradient instability or feature imbalance, unify the scale and distribution of feature representation, and improve the stability and usability of fused data.

[0054] Finally, a cross-modal attention mechanism is used to fuse the normalized features. The cross-modal attention mechanism refers to dynamically allocating information weights through attention parameters for different data sources or different information modalities, automatically enhancing the expression of key features, suppressing redundant or interfering information, and forming a joint feature representation with global perception and dynamic weight adjustment capabilities. The joint feature representation can be a data set in tensor form, matrix form, or other structural forms, and has the ability to express spatial information, structural information, and the interaction relationship between information from different sources.

[0055] In practical applications, channel dimension expansion can be achieved through copy expansion, which repeatedly fills the target number of channels with single-channel data, or through parameter mapping, which uses a trainable mapping matrix to project the original data into a high-dimensional feature space. Spatial alignment can be combined with upsampling modules, such as transposed convolution, nearest neighbor interpolation, or bilinear interpolation, or with downsampling modules, such as strided convolution or pooling operations, to ensure consistency in spatial dimensions between data from different sources.

[0056] The generation process of the concatenated feature matrix can employ channel-level concatenation, spatial-level concatenation, or feature-level concatenation, combining various fusion strategies to enhance data representation capabilities. The structure of the feature transformation network can be dynamically adjusted according to business needs, including single-layer or multi-layer convolutional structures, parameter-sharing networks, or Transformer-based encoder structures. Specifically, it can employ structures such as 3×3 standard convolutions, 1×1 feature mapping convolutions, or grouped convolutions to achieve efficient feature transformation.

[0057] The implementation of the feature normalization layer can be tailored to the specific application environment, choosing either batch normalization to improve training stability, or layer normalization and instance normalization to adapt to small batches of data or dynamically input data. Cross-modal attention mechanisms can dynamically adjust the fusion ratio of different data sources through dot product attention, weighted attention, or adaptive gating mechanisms, focusing on enhancing the representation of spatial structure information, depth information, or other key features, and outputting a joint feature representation for subsequent processing.

[0058] Example Description: In the healthcare business field, feature fusion operations can be applied to multimodal medical image analysis scenarios. Specifically, it involves using medical image data such as CT images, MRI images, and ultrasound images as initial data, and combining them with deep feature maps extracted from deep learning networks to form joint feature representations. This effectively integrates structural information and spatial layout information, facilitating multimodal image registration, tissue structure recognition, 3D reconstruction, or lesion region analysis, thereby improving the fusion and representation capabilities of multi-source medical data.

[0059] In the fintech business field, feature fusion operations can be applied to risk monitoring, behavior recognition, and environmental perception scenarios. Specifically, this involves using video surveillance data, 3D environmental scanning data, or user operation data as initial data, and combining them with spatial structure information or depth feature maps to form a joint feature representation with multi-source information fusion capabilities. This facilitates dynamic behavior detection, spatial layout analysis, risk event identification, and environmental change perception, thereby improving data integration capabilities and dynamic perception levels in financial business scenarios.

[0060] This embodiment achieves effective integration of information from different sources by structurally fusing initial data with deep feature maps, enhancing the synergy between spatial and structural information in the data representation process. Feature concatenation and feature transformation operations improve the hierarchical nature of data representation and its ability to adapt to multi-dimensional structures. Combined with normalization operations, the numerical stability and representational consistency of the fused features are optimized. The introduction of a cross-modal attention mechanism effectively enhances the dynamic interaction capabilities between different information sources, improving the completeness and information utilization of the joint feature representation in spatial representation, structural recognition, and subsequent dynamic mask generation processes.

[0061] S30, generate a starting data unit based on the joint feature representation, and set the current data unit and the preceding data unit of the current data unit as the starting data unit;

[0062] In this embodiment, the process of generating initial data units based on joint feature representation refers to forming standard data units with the initial state of a data sequence by performing structural analysis, feature transformation, and data reconstruction on the aforementioned joint feature representation. Joint feature representation refers to a high-dimensional structured data set formed by fusing initial data with features from a deep feature map. It is usually represented as a tensor structure, a multidimensional matrix, or a set of feature maps, and has the ability to express spatial information, structural information, and information from different sources.

[0063] In the process of generating the initial data unit, the joint feature representation is first processed by a feature decoder. The feature decoder is a network module with features for feature reconstruction, information parsing, and structural reconstruction. Specifically, it can include deconvolutional networks, upsampling modules, feature mapping layers, or other functional components for dimensionality reduction and structural reconstruction. The feature decoder receives the joint feature representation and, through layer-by-layer reconstruction operations, extracts and generates initial hidden state features. These initial hidden state features are represented as multidimensional tensors, feature matrices, or structured data sets, containing spatial layout information, structural association information, and sequence start-point state information, facilitating subsequent temporal processing and state propagation.

[0064] To adapt the initial hidden state features to the temporal structure requirements of sequence data, a temporal dimension expansion operation needs to be performed on the initial hidden state features. Temporal dimension expansion refers to introducing the original feature data into the time dimension through operations such as dimension insertion, data reshaping, or structure mapping, forming extended hidden state features with the ability to express temporal information. The expansion operation can achieve a complete expression of the time dimension based on zero-padding, repeated padding, parameter mapping, or other forms, ensuring that the generated data units have the basic requirements of a sequence structure.

[0065] The extended latent state features are processed by a starting unit generation network to generate starting data units in a standard format. The starting unit generation network is a network module with functions of structure mapping, information integration, and data reorganization. It can include multilayer perceptrons, convolutional neural networks, recurrent neural networks, or sequence mapping modules based on the Transformer structure. The generated starting data units are datasets with spatial, structural, and temporal information representation capabilities, in the form of tensors, matrices, or structured data structures, serving as the basic units for subsequent data sequence generation.

[0066] To establish the initial state of the data sequence, the current data unit is set as the starting data unit. The current data unit refers to the real-time processing object in the sequence generation process, representing the data content at the current time point or under the current structural state. Simultaneously, the preceding data unit of the current data unit is set as the starting data unit. The preceding data unit refers to the standard data object located before the current data unit in the sequence structure, typically serving as a reference benchmark or dependent data to support subsequent time-series feature calculations and structural correlation analysis.

[0067] In the specific implementation, the feature decoder can adopt a deconvolutional network structure, specifically including a 3×3 deconvolutional layer, an upsampling layer, an activation function layer, and a feature mapping layer, layer by layer restoring the structural information and spatial layout of the joint feature representation. Temporal dimension expansion can be achieved by inserting zero values ​​or repeating existing data in the temporal dimension of the feature tensor, ensuring that the data structure possesses temporal attributes. The starting unit generation network can be based on a lightweight convolutional network or a Transformer sequence encoder structure, outputting starting data units with standard format and structural information through multi-layer mapping and parameter adjustment.

[0068] Setting the current data unit as the starting data unit can be accomplished through memory pointer redirection, data object copying, or structure pointer updating, ensuring the accuracy of data reference relationships and structural integrity. Setting the preceding data unit of the current data unit as the starting data unit can be combined with memory mapping, structure linked list updates, or data reference synchronization operations, ensuring the continuity and stability of the structural relationship and temporal dependency between the preceding and current data units.

[0069] Example Explanation: In the healthcare field, the generation of sequences from multimodal medical imaging data typically requires ensuring that the structural information after fusion of different modalities possesses standardized and continuous expressive capabilities. In this process, generating initial data units based on joint feature representations and simultaneously setting the current data unit and its preceding data units as initial data units ensures the integrity of the data structure and the clarity of the expression logic in the initial stage of the medical imaging sequence. For example, after fusing CT images and depth structural data, a feature decoder restores the multidimensional joint features into medical image units with anatomical structural expressive capabilities. Further, by extending the temporal dimension, temporal attributes are assigned to the data units, ensuring that the initial data units possess spatial expression, structural transfer, and temporal continuity in subsequent sequence generation. Establishing state associations between preceding and current data units ensures structural stability and clear dependencies during sequence data generation, reducing the risk of structural breaks and information loss in medical sequence generation.

[0070] In the fintech business, when constructing risk behavior sequence models based on video surveillance data and in-depth environmental information, generating initial data units and setting dependencies between preceding and current data units helps establish complete and continuous risk behavior data sequences. By forming joint feature representations through feature fusion, and utilizing feature decoders and temporal extensions to generate initial data units with multi-dimensional spatial structures and temporal expression capabilities, and by setting preceding data units as the historical states of the current data unit, the logical relationships between data units are clear and information transmission is stable in the early stages of risk behavior sequence generation. This effectively supports subsequent sequence structure reasoning based on motion features and dynamic information, as well as high-risk behavior identification, thereby improving the structural integrity and dynamic adaptability of financial risk control data sequence generation.

[0071] This embodiment generates starting data units based on joint feature representation, achieving structural unification of multi-source information and standardized expression of the sequence's starting state. Through feature decoding and temporal dimension expansion operations, it ensures the integrity and consistency of data units in spatial layout, structural information, and temporal dimension expression. By setting the state relationship between the current data unit and its predecessor data units, it establishes data dependency chains in the sequence structure, improving the stability and logical continuity of the sequence data generation process and reducing the risk of information loss and structural breakage in subsequent structural calculations and state transmission.

[0072] S40, Based on the current data unit and the preceding data unit of the current data unit, generate motion complexity features;

[0073] In this embodiment, the current data unit refers to the data unit that is in a real-time updated state during the sequence generation process. Specifically, it includes data structure information formed through feature decoder and attention mechanism, and has spatial structure expression and temporal attribute identification. The preceding data unit of the current data unit refers to the data unit of the previous historical state in the sequence before the generation of the current data unit. The two have a logical relationship of temporal sequence and state transmission in the sequence structure, ensuring the continuity and dynamic consistency of data dependencies.

[0074] To generate motion complexity features based on the current and preceding data units, key feature information must first be extracted from both units. This is typically achieved through feature encoders, feature extraction networks, or multi-scale transformation modules, mapping the structural information, spatial representation, and temporal state parameters of each data unit into multi-dimensional feature representations. These representations can be tensors, matrices, or multi-channel feature maps, with the specific dimensions and data structure flexibly adjusted based on the input data type. By comparing the feature representations of the current and preceding data units, and considering spatial location, structural differences, and temporal variation, the feature differences between them are calculated. Common methods include Euclidean distance calculation, cosine similarity analysis, optical flow estimation, or deep learning-based feature matching methods.

[0075] Furthermore, a motion vector field is generated based on the extracted feature difference information. This motion vector field reflects the positional changes, structural shifts, and motion trends of the current data unit compared to its predecessor in both spatial and temporal dimensions. The generation of the vector field can employ optical flow networks, spatial transformation networks, or spatiotemporal difference modules to ensure the continuity and accuracy of motion information representation. Subsequently, gradient magnitude is calculated for the motion vector field. The gradient magnitude reflects the local motion intensity and overall structural changes. Gradient magnitude can be achieved through various techniques such as first-order or higher-order derivative calculations, differential convolution kernels, and edge enhancement operators to ensure the integrity of motion information representation across different scales and regions.

[0076] The gradient magnitude results are spatially averaged to generate an average motion intensity. This average motion intensity reflects the overall motion change and structural dynamics of the data units. Processing methods include global average pooling, regional aggregation, or weighted averaging, which can be flexibly adjusted based on the specific application scenario and computing resources. Finally, the average motion intensity is normalized to adjust the motion intensity values ​​to a standardized range. Commonly used mechanisms include max-min normalization, Z-score normalization, or adaptive normalization to ensure the comparability and stability of motion complexity features between different data units. The final output motion complexity feature is a crucial expression reflecting changes in data unit structure, motion intensity, and spatiotemporal dynamics, and is widely used in subsequent dynamic threshold adjustment, sparse attention mechanism control, and sequence generation optimization processes.

[0077] In the specific implementation process, feature extraction of the current data unit and the preceding data unit can be achieved through convolutional neural networks with shared parameters or multi-layer coding structures, ensuring consistency in feature representation and accuracy in comparative analysis. During motion vector field generation, a dual-frame optical flow network combined with a multi-scale pyramid structure can be used to improve motion estimation accuracy, while introducing structure preservation constraints to avoid loss of detailed information. In the gradient magnitude calculation stage, the Sobel operator, Scharr operator, or adaptive gradient extraction network can be used to enhance local edge information and motion detail representation. During spatial averaging, depending on application requirements, global averaging, partitioned weighting, or hierarchical aggregation methods can be flexibly selected to ensure that the average motion intensity accurately reflects overall dynamic changes. In the normalization process, a standardization method based on statistical distribution can be introduced, or the normalization parameters can be dynamically adjusted in conjunction with deep feature distribution to improve the consistency and discriminative ability of motion complexity features among different data units.

[0078] Example Description: In the healthcare field, to address the need for generating dynamic ultrasound images, sequential CT scans, or cardiac MRI data sequences, motion complexity features are generated using the current data unit and preceding data units. This dynamically reflects the deformation trends, motion intensity, and structural changes of organs and tissues. Through multi-dimensional feature comparison, optical flow vector generation, and dynamic intensity normalization, information such as lung deformation caused by heartbeats or respiration, or vascular pulsation, is accurately captured. This improves the continuity, realism, and structural stability of medical sequence data, and reduces structural distortion and image blurring caused by missing motion information during the generation process.

[0079] In the fintech business, for the data sequence construction needs in security surveillance videos, risk behavior recognition sequences, or remote collaboration processes, generating motion complexity features can reflect the dynamic changes in the state of people, objects, or the environment in real time. Through structural difference analysis and vector field generation, it accurately identifies personnel movement, object movement, or sudden abnormal states in the scene. Combined with motion intensity normalization processing, it improves the data expression accuracy and dynamic change adaptability of risk monitoring sequences, providing an accurate dynamic data foundation for subsequent high-risk behavior recognition and intelligent decision-making.

[0080] This embodiment generates motion complexity features based on the current data unit and the preceding data unit, which can effectively capture local structural changes and overall dynamic trend information in the data sequence. Combined with multidimensional feature difference analysis, motion vector expression and intensity normalization processing, it can accurately quantify and express the motion state of different data units, and further provide data support for dynamic threshold adjustment and sparse attention mechanism, thereby improving the structural stability and dynamic adaptability in the sequence generation process.

[0081] S50, Based on the depth feature map and the motion complexity feature, generate a dynamic threshold mask;

[0082] In this embodiment, the depth feature map is a multi-dimensional spatial representation formed by performing depth information extraction, structural feature encoding, and multi-scale fusion operations on the initial data. It reflects the geometric structure, spatial hierarchy, and local texture information in the data. The depth feature map has clear spatial distribution and structural constraints. The data format is typically a multi-channel tensor or matrix, with the specific number of channels, resolution, and representation dimension flexibly adjusted according to the actual data type and downstream requirements. The motion complexity feature is an index extracted based on the structural changes and dynamic trends of the current data unit and the preceding data units. It comprehensively reflects the overall motion intensity, local change amplitude, and dynamic structural complexity of the data in the time series process. The data format is typically a scalar, vector, or low-dimensional feature representation.

[0083] Generating dynamic threshold masks based on deep feature maps and motion complexity features first requires structural processing of the deep feature maps, specifically including convolution operations, scaling transformations, and spatial feature integration. High-dimensional representation information is extracted from the deep feature maps through multi-layer convolutional structures or feature transformation networks, forming a well-structured and richly layered deep feature representation. This deep feature representation preserves the spatial structure information and local texture features of the original data, providing a stable geometric basis for dynamic threshold generation.

[0084] Simultaneously, linear transformations, scale adaptations, or dynamic adjustments are performed on motion complexity features to ensure that the motion complexity features and deep feature representations have consistent representation scales, channel structures, and spatial mapping relationships. Common techniques include fully connected layers, feature projection modules, or adaptive parameter adjustment mechanisms to form transformed motion features that match the structure of the deep feature representation. By fusing deep feature representations and transformed motion features, a joint representation space is constructed. The fusion operation can employ feature concatenation, element-wise weighting, cross-channel mapping, or multi-layer fusion networks to ensure balanced weights and stable interactions between structural information and dynamic features in the fused representation.

[0085] After the fused feature representation is generated, it is further processed by an activation function. The activation function type is usually chosen as Sigmoid, Tanh, or other nonlinear functions with good continuity to ensure that the fused representation maps to the normalized range, has stable gradient changes and controllable output distribution, forming an activation feature representation. Finally, a dynamic threshold mask is generated based on the activation feature representation. The mask is used to guide the sparsity control, computational resource allocation, and structural region optimization in the subsequent attention mechanism. The dynamic threshold mask has spatial distribution variability, structural adaptability, and dynamic update capability. The mask data structure is usually a multi-channel matrix or tensor of the same size as the depth feature map. The values ​​at different positions in the mask are dynamically adjusted according to the depth information and motion complexity, realizing structural adaptive control of attention resource allocation.

[0086] In practical applications, structural processing of deep feature maps can be achieved through multi-scale convolutional networks, pyramid structures, or hierarchical aggregation modules to improve the integrity of structural representation and the stable fusion of multi-layer information. During the linear transformation of motion complexity features, parameter-sharing fully connected layers or feature channel mapping mechanisms can be used to ensure consistency between the representation space of the transformed motion features and the deep feature representation. During the fusion operation, depending on different application requirements, feature concatenation and element-wise weighting can be used to enhance representational richness, or multi-layer fusion networks can be used to achieve more complex cross-feature interactions and information integration. The choice of activation function needs to be combined with the actual application scenario to ensure that the output mask has stable gradient characteristics and continuous differentiability. Common solutions include mapping Sigmoid to the 0-1 interval, Tanh mapping to a symmetric interval, or adaptive activation functions to achieve dynamic range adjustment. After the dynamic threshold mask is generated, it can directly participate in the sparse matrix construction, weight control, and computational resource allocation in the subsequent attention mechanism, ensuring structural stability, dynamic adaptability, and resource optimization capabilities during sequence generation.

[0087] Example Description: In the healthcare field, for the generation of dynamic ultrasound sequences, cardiac MRI scans, or continuous CT images, dynamic threshold masks are generated based on depth feature maps and motion complexity features. This allows for the adaptive adjustment of the structural focus area and computational resource allocation during sequence generation by combining the spatial geometric information and dynamic motion state of organ structures. This ensures that the deformation expression of the heart, blood vessels, or lungs is stable and the structural details are clear at different time points, reducing the risk of sequence distortion and detail blurring caused by missing motion information, and improving the structural accuracy and clinical application value of medical sequence data.

[0088] In the fintech business, for data generation tasks such as security surveillance videos, risk behavior recognition sequences, or remote collaboration systems, dynamic threshold masks, combined with scene spatial layout and dynamic motion information, adjust the system's computing resource allocation for key areas and high-risk dynamics in real time. This strengthens the structural focus and dynamic tracking of people, objects, or abnormal events, improves the completeness of sequence data representation and the real-time response capability of risk monitoring, and provides high-quality dynamic data support for intelligent security, risk control, and business continuity.

[0089] This embodiment generates a dynamic threshold mask based on depth feature maps and motion complexity features. It can combine the spatial structure information and dynamic motion state of the data to adjust the sparsity mode and computational resource allocation in real time during the attention calculation process. This enables adaptive control of spatial structure and dynamic region attention, improves the structural expression integrity, motion consistency and computational efficiency in the sequence generation process, reduces redundant computational overhead, optimizes the structural region expression ability, and enhances the spatial details and dynamic adaptability of the generated data.

[0090] S60, apply the dynamic threshold mask to the attention mechanism, perform sparse attention processing, and generate the next data unit;

[0091] In this embodiment, the dynamic threshold mask, as a sparse control representation jointly regulated by spatial structure information and motion state, possesses the characteristics of dynamic variability, structural adaptability, and precise regional control. The data format is typically a two-dimensional or three-dimensional tensor structure, and the numerical range is adjusted according to the activation function output. The values ​​at different positions in the mask reflect whether the corresponding spatial position participates in the subsequent attention mechanism calculation process. The attention mechanism dynamically allocates information interaction and computational resources by calculating the correlation weights between positions in the input data. Common representations include attention weight matrices constructed based on query feature representation, key feature representation, and value feature representation.

[0092] In applying dynamic thresholding to the attention mechanism, query feature representation, key feature representation, and value feature representation are first extracted from the input data. Query feature representation is obtained through structural encoding, feature mapping, or multi-layer network transformation of the input data, and it has the ability to express query information, structural requirements, and contextual dependencies at each location in the data. Key feature representation originates from the same or shared data structure and is obtained through independent mapping or shared parameter transformation, expressing the searchable structural information and spatial response at each location in the data. Value feature representation is a dynamically weighted data representation, typically containing spatial structure, content features, and dynamic information, and participates in the final output generation process.

[0093] Based on query feature representation and key feature representation, an attention weight matrix is ​​generated using matrix multiplication, dot product similarity, or other structural relevance calculation methods. The weight matrix reflects the degree of attention each position in the data receives to the overall structure and local features, expressing spatial structural dependence and the strength of information interaction. A dynamic threshold mask is applied to the attention weight matrix process. By combining the spatial distribution and dynamic values ​​of the mask, the participation intensity of different positions in the weight matrix is ​​adjusted, or specific positions are directly masked, forming a sparse attention weight matrix. The sparsity control mechanism can employ element-wise multiplication, threshold truncation, or structural mask filtering to ensure that attention calculations are only retained for key structural positions and dynamically important regions, reducing redundant computational resource consumption and improving the targeting and computational efficiency of structural attention.

[0094] The modulated feature representation is generated by modulating the value feature representation using a sparse attention weight matrix. Specifically, based on the adjusted weight matrix, weighted summation, matrix transformation, or spatial mapping is performed on the value feature representation to generate a modulated feature representation. This modulated feature representation integrates spatial structure information, dynamic region attention, and sparsity control effects, expressing the structural continuity, motion consistency, and efficient computational results of the sequence. Finally, the modulated feature representation is processed by a decoder network, which can employ a multilayer perceptron, convolutional structure, or self-attention module. The decoder network outputs the hidden state of the next data unit, which expresses the structurally complete data feature information, spatial hierarchy, and dynamic adaptive attributes of the next time step. Based on the hidden state of the next data unit, further structural mapping, feature restoration, or output transformation is performed to generate the next data unit. This next data unit possesses a spatial representation capability that is structurally consistent with the current data unit and dynamically continuous, providing dynamically updated basic data for the sequence generation process.

[0095] In practical implementation, the extraction of query feature representations, key feature representations, and value feature representations can be achieved through shared coding networks, multi-branch structures, or parameter-independent feature mapping modules. The specific network structure can be adjusted according to the data type and target task, including convolutional neural networks, Transformer encoders, or other feature extraction frameworks. The attention weight matrix can be generated using scaled dot product attention, additive attention, or structural similarity mapping to ensure the stability and expressiveness of the weight matrix.

[0096] In the application of dynamic thresholding, element-wise multiplication is suitable for sparsity control of dense structures, threshold truncation is suitable for regional dynamic focus, and structure masking filtering is suitable for fine control in irregular structures or complex scenarios. The modulation process of value feature representation can be combined with spatial location coding, hierarchical structure information, or dynamic change trends to improve the expressive completeness and dynamic response capability of the modulation feature representation.

[0097] The decoder network can select a convolutional decoder, a hierarchical reconstruction module, or an attention-enhanced decoding structure based on the target data structure, and output the hidden state of the next data unit. The hidden state generates the next data unit through linear mapping, nonlinear transformation, or structure restoration operations. The data structure maintains consistency with the overall structure of the sequence and supports continuous generation and dynamic updating.

[0098] Example Description: In the healthcare business field, for tasks involving cardiac ultrasound sequences, dynamic MRI images, or continuous CT reconstruction, dynamic threshold masking to attention mechanisms can be applied. By combining the spatial layout and dynamic motion information of organ structures, the attention to structural regions and the allocation of computational resources can be dynamically adjusted to enhance the stability and structural continuity of the expression of dynamic changes in cardiac contraction, vasodilation, or lungs, thereby improving the spatial clarity, motion consistency, and diagnostic reference value of medical sequence data.

[0099] In the fintech business, for dynamic data generation tasks in security monitoring sequences, risk behavior detection, or remote collaboration systems, dynamic threshold masks, combined with scene structure information and dynamic event changes, adjust the structural focus area of ​​the attention mechanism in real time, enhance the ability to express key monitoring targets, abnormal dynamic behaviors, and high-risk areas, improve the structural stability and risk response efficiency of the dynamic data generation process, and provide high-quality dynamic data support for intelligent monitoring, risk identification, and business continuity.

[0100] This embodiment applies dynamic threshold masking to the attention mechanism, which can dynamically control the sparsity mode and computational resource allocation of attention calculation while preserving spatial structure information and dynamic region attention. This effectively reduces redundant computation, enhances the expression intensity of key structures and important dynamic regions, achieves joint optimization of structural information and dynamic attributes, strengthens the structural continuity, spatial integrity and dynamic adaptability of the next data unit, and improves the efficiency and quality of the sequence generation process.

[0101] S70, update the preceding data unit of the current data unit to the current data unit, and update the current data unit to the next data unit;

[0102] In this embodiment, the current data unit serves as a key structural representation dynamically updated during sequence generation. It possesses complete spatial information, structural features, and dynamic attributes. The data structure is typically a tensor, matrix, or feature set, representing the spatial structural state and dynamic evolution result of the sequence at the current moment. The preceding data unit is the historical state information of the current data unit from the previous moment or previous generation round. Its data structure remains consistent with the current data unit, used for dynamic comparison, structural updates, and motion analysis. The next data unit is the structural representation of the next moment in the sequence generated after sparse attention processing and feature modulation. Its data structure maintains isomorphism with the current data unit, possessing spatial integrity and dynamic adaptability.

[0103] In the process of updating the preceding data unit of the current data unit, the contents of the current data unit are first copied or mapped to the storage location of the preceding data unit through deep copy, structure pointer replacement or data mapping. This ensures that the preceding data unit has all the structural information and dynamic attributes of the current data unit. The update process must avoid data redundancy and structural conflicts to ensure the accuracy and structural consistency of the update operation.

[0104] During the process of updating the current data unit to the next data unit, the structural content of the next data unit is completely assigned to the current data unit through methods such as structural replacement, data pointer redirection, or memory mapping. The updated current data unit has the latest generated spatial structure state, dynamic change information, and sequence continuity expression, ensuring the consistency of the data unit structure and the coherence of dynamic evolution at each moment during the sequence generation process.

[0105] In practical implementation, updating previous data units can be achieved through deep copying, suitable for scenarios with complex data structures and high memory isolation requirements, ensuring data content independence and structural integrity during the update process. Alternatively, memory pointer redirection can be used, suitable for scenarios with limited memory resources or high real-time response requirements, improving data update efficiency and system response speed. The update operation of the current data unit can be performed using memory mapping, pointer replacement, or direct assignment, depending on the data structure type, ensuring that the structural information of the next data unit completely covers the storage location of the current data unit, avoiding data residue and structural conflicts.

[0106] During sequence generation, update operations are usually embedded in loop structures or iterative processes. Combined with iteration counters, status indicators, or conditional judgment mechanisms, the synchronous updates of previous data units and current data units are dynamically executed to ensure the continuity and dynamic consistency of the data structure at each moment of the sequence, and to avoid structural misalignment, data loss, or dynamic disorder.

[0107] Example Description: In the healthcare business field, for tasks such as echocardiography sequences, dynamic MRI scans, or continuous CT image reconstruction, by dynamically updating the current data unit and the preceding data unit, the structural stability and dynamic consistency of the heart structure, vascular status, or organ dynamics during the continuous generation process are ensured. This avoids image jumps, dynamic distortions, or structural reconstruction errors caused by data unit update lags or incomplete structural information, thereby improving the spatial representation quality and clinical diagnostic value of medical sequence data.

[0108] In the fintech business, for security monitoring sequences, remote collaborative operations, or dynamic behavior generation tasks, the preceding data units and the current data units are updated in real time. This dynamically maintains the continuous expression of the monitored objects, operational behaviors, or scene structures, avoiding monitoring blind spots, risk identification delays, or business continuity interruptions caused by inconsistent data unit structures. This improves the structural stability, dynamic response efficiency, and data generation quality of the intelligent monitoring system in high-risk scenarios.

[0109] This embodiment updates the preceding data unit of the current data unit to the current data unit and updates the current data unit to the next data unit. This can dynamically maintain the consistency of the data unit structure and the continuity of historical information during the sequence generation process, ensuring the integrity of the data infrastructure on which motion complexity features, dynamic threshold mask generation and sparse attention processing depend. This improves the structural continuity of data units at each time point during the sequence generation process, the coherence of dynamic evolution and the expression stability of the overall sequence.

[0110] S80, repeatedly execute the steps of generating motion complexity features based on the current data unit and the preceding data unit of the current data unit, generating a dynamic threshold mask based on the motion complexity features, applying the dynamic threshold mask to perform sparse attention processing to generate the next data unit, and updating the preceding data unit and the current data unit of the current data unit, to generate multiple iterative data units;

[0111] In this embodiment, the essence of repeated execution is to iteratively process the aforementioned data structure-dependent processing to form a continuous, progressive, multi-stage output. The main loop includes four interrelated operations: motion complexity feature generation, dynamic threshold mask generation, sparse attention processing and next data unit generation, and data unit state update. These operations rely on the relative information difference between the preceding data unit and the current data unit to dynamically adjust the attention calculation mode, thereby improving the overall expressive power of the sequence structure.

[0112] In the specific implementation process, the data change features are first extracted based on the content differences between the current data unit and its predecessor data units to generate motion complexity features. These features characterize the magnitude of change in the spatial structure or numerical distribution of the current data unit compared to its predecessor data units, and typically involve calculating vector offsets, local gradients, or overall structural indices in the feature space.

[0113] Subsequently, this motion complexity feature is combined with existing deep feature maps to dynamically generate a threshold mask. The generation process of the dynamic threshold mask takes into account the degree of change in data content and adjusts the retention or elimination mechanism in the attention weight matrix through an adaptive parameter control strategy to form a dynamic sparsity pattern, thereby controlling the allocation of computing resources.

[0114] Based on dynamic threshold masking, a sparse attention processing method is applied within the attention mechanism. In this process, a sparse structure is directly introduced during the attention matrix calculation stage through masking, preserving attention to important regions or salient features while suppressing computational overhead in redundant or low-relevance regions, ultimately outputting the next data unit. The next data unit typically serves as an incremental information unit in sequence generation, maintaining a consistent structural form with the preceding data unit.

[0115] Finally, the preceding data unit of the current data unit is updated to the existing current data unit, and the current data unit is replaced with the newly generated next data unit, forming a new data structure state. This update mechanism ensures that the next iteration operation is always based on the latest data unit state and its preceding reference, guaranteeing the continuity of the information chain and its dynamic adjustment capability.

[0116] The above process is executed cyclically under preset iterative control conditions. The loop termination condition can be dynamically set according to the number of target data units, the length of the generated sequence, or specific content determination logic. Through multiple iterative operations, multiple iterative data units are gradually generated. These multiple iterative data units together constitute the intermediate stage result of the data sequence, possessing the ability to enrich data representation layer by layer and dynamically balance computing resources.

[0117] In practical applications, the number of iterations can be flexibly configured based on business needs. For example, in video generation, time series modeling, and continuous prediction tasks, the number of iterations can correspond to the target number of frames, time step, or output data size. Iteration progress can be recorded using a dynamic counter, and the overall loop process can be controlled by setting termination conditions.

[0118] Under different implementation methods, the extraction of motion complexity features can employ structured feature comparison, image gradient analysis, vector field calculation, or deep learning feature encoding. Dynamic threshold mask generation can be achieved through various means such as linear mapping, nonlinear transformation, and adaptive network prediction. The sparse attention processing can be flexibly adjusted based on explicit matrix masks, index mapping, soft sparse strategies, or hybrid attention mechanisms.

[0119] For different hardware platforms or computing environments, the above iterative structure can be further improved by introducing asynchronous computing, parallel distributed processing or storage optimization techniques to enhance overall execution efficiency and system response speed.

[0120] Example Description: In the healthcare field, motion complexity features are generated based on the current data unit and its preceding data units. These features are then combined to dynamically generate a threshold mask. This allows for dynamic reflection of the motion state and trends of physiological tissues in tasks such as ultrasound examinations, cardiac function monitoring, and continuous MRI sequence analysis. Through multiple iterative operations, multiple iterative data units are gradually generated, forming a continuous structural representation. This enables adaptive adjustment of attention allocation strategies during sequence modeling of cardiac pulsation, blood flow dynamics, and lesion evolution. It strengthens focus on abnormal tissues or highly dynamic regions, reduces unnecessary computational overhead, and improves the accuracy and stability of sequence generation, supporting the needs of dynamic reconstruction, continuous prediction, and assisted diagnosis of medical images.

[0121] In the fintech business, this iterative mechanism can be leveraged for scenarios such as multi-timestep market behavior modeling, transaction data sequence generation, and asset price volatility prediction. By extracting motion complexity features based on the current data unit and its preceding data units, and combining this with dynamically generated threshold masks to dynamically sparsify the computational range of the attention mechanism, it can effectively capture highly sensitive areas in market data, such as sudden changes, sharp fluctuations, or trend reversals. This improves the timeliness of financial sequence generation and its ability to reflect the true market structure. By generating multiple iterative data units, a coherent financial data sequence with clear dynamic characteristics can be gradually constructed and applied to intelligent risk control, market trend simulation, and high-frequency trading decision support, enhancing the dynamic adaptability and computational efficiency of data-driven systems.

[0122] This embodiment uses the iterative loop mechanism, combined with dynamically generated motion complexity features and dynamic threshold masks, to adaptively adjust the sparse attention processing strategy. While ensuring the continuous generation of data units, it effectively reduces the overall computational resource consumption, improves the structural consistency and dynamic expressive ability of the generated data sequence, and enhances the applicability of the system in high-resolution, long-sequence processing tasks.

[0123] S90, perform timing processing on the starting data unit and the plurality of iterative data units to form a data sequence.

[0124] In this embodiment, the initial data unit and multiple iterative data units undergo temporal processing to form a data sequence containing multi-level operational logic. The initial data unit is the first sequence element obtained from the initial data through a joint feature generation process. The multiple iterative data units are a set of data units generated sequentially through a cyclic generation process and having a temporal relationship. Temporal processing refers to combining the initial data unit and multiple iterative data units according to the generation order or logical order to form a data structure with temporal characteristics. Through structuring or feature enhancement operations, the data units form a continuous, smooth data expression result that conforms to the overall sequence logic in terms of spatial distribution, temporal connection, and dynamic feature expression.

[0125] In practical implementation, the initial data unit is first placed at the beginning of the sequence through data sorting or sequence concatenation. Multiple iterative data units are then concatenated after the initial data unit according to their generation order or other defined order, forming a preliminary sequence structure. Subsequently, based on a 3D convolutional network, a temporal modeling module, or other spatiotemporal feature processing networks, joint feature extraction and temporal relationship enhancement operations are performed on the preliminary sequence structure to generate a data sequence with complete temporal continuity, spatial consistency, and the ability to express dynamic changes.

[0126] To enhance the expressive power of temporal structures, inter-frame compensation mechanisms can be introduced during temporal processing. These mechanisms utilize motion information or trends between adjacent data units to dynamically adjust the features of data units within the sequence, compensating for discontinuities, jumps, or structural misalignments caused by independent generation or missing data. Furthermore, upsampling, feature reconstruction, or continuity optimization operations can improve the expressive quality of the data sequence in terms of resolution, level of detail, and dynamic transitions, ensuring that the final data sequence possesses high accuracy and stability in terms of overall structure, local details, and temporal dynamics.

[0127] Timing processing can be achieved through the following different technical approaches:

[0128] A three-dimensional convolutional network can be used, based on spatiotemporal joint convolution operations, to take as input the initial data unit and multiple iterative data units, extract multi-scale spatiotemporal features, enhance the dynamic correlation of data units within the sequence, and output a data sequence with spatial consistency and temporal continuity.

[0129] Alternatively, by combining sequence feature concatenation with a bidirectional recurrent neural network, the initial data unit and multiple iterative data units can be sequentially combined into a one-dimensional or two-dimensional matrix structure, and then input into the recurrent neural network. By utilizing the forward and backward information flow transmission mechanism, short-term dependencies and long-term dynamic information in the sequence can be captured, generating a more coherent data sequence.

[0130] Furthermore, the adaptive motion compensation module can analyze the motion changes of adjacent data units, dynamically adjust the spatial position and feature distribution of data units in the sequence, correct the inter-frame discontinuity caused by motion changes, structural shifts or generation errors, and improve the dynamic consistency and structural stability of the data sequence.

[0131] To further enhance sequence quality, a multi-scale upsampling module can be used to perform resolution upsampling on the generated data sequence. This optimizes the spatial detail of the sequence while keeping the global structure unchanged, thereby improving the overall expressive effect of the data sequence.

[0132] Example Description: In the healthcare field, during dynamic patient examinations, multiple frames of dynamic medical images are needed to reflect the continuous motion of organs or tissues. Through temporal processing, the initial data unit and multiple iterative data units are structurally combined into a complete data sequence. This sequence can be applied to the generation of ultrasound cardiac sequences, dynamic brain functional sequences, or dynamic blood flow images. It ensures that each frame within the sequence maintains continuity in temporal order and structural representation, presenting a smooth dynamic process, which facilitates accurate analysis of lesion trends or areas of functional abnormality by clinicians.

[0133] In the fintech business, when modeling market trading behavior or tracking risk indicators, it is necessary to construct dynamic data sequences to reflect market conditions or individual indicator changes at multiple time points. Through time series processing, the initial data unit is sequentially combined with multiple iterative data units to form a continuous and stable financial indicator sequence. This is suitable for generating trading volume fluctuation sequences, abnormal user account behavior sequences, or dynamic risk exposure sequences at multiple time points, ensuring the accuracy of the internal time logic of the data sequence and the clarity of indicator change trends. This helps the system execute real-time risk control decisions, predict trading behavior, or analyze risk trends.

[0134] This embodiment achieves efficient temporal fusion of the initial data unit and multiple iterative data units through the above operations. The resulting data sequence has temporal continuity, spatial consistency, and stability of dynamic features, effectively solving the problems of inter-frame breaks, structural jumps, and insufficient dynamic expression in the existing sequence generation process, and improving the overall expression quality and application reliability of the data sequence.

[0135] This invention relates to the field of artificial intelligence technology and can be applied to business scenarios such as fintech and healthcare. It discloses a data sequence generation method, apparatus, device, and medium, comprising: acquiring initial data; extracting depth features from the initial data to obtain a depth feature map; fusing the initial data and the depth feature map to generate a joint feature representation; generating a starting data unit based on the joint feature representation; setting the current data unit and its preceding data unit as the starting data unit; generating motion complexity features based on the current data unit and its preceding data unit; generating a dynamic threshold mask based on the depth feature map and the motion complexity features; applying the dynamic threshold mask to an attention mechanism; performing sparse attention processing; generating the next data unit; updating the preceding data unit and the current data unit; repeatedly executing the steps of generating motion complexity features, generating a dynamic threshold mask, performing sparse attention processing, and updating data units to generate multiple iterative data units; and performing temporal processing on the starting data unit and the multiple iterative data units to form a data sequence. This invention dynamically generates a threshold mask by jointly considering depth features and motion complexity features, and applies a sparse attention mechanism to the data generation process. It can dynamically optimize the sparse structure of attention calculation without reducing the quality and continuity of the data sequence, thus significantly improving computational efficiency and system stability.

[0136] In one embodiment, step S10 includes:

[0137] S101, Receive initial data input;

[0138] S102, Perform color space standardization processing on the initial data to generate standardized initial data;

[0139] S103, Adjust the resolution of the standardized initial data to a preset size;

[0140] S104, The standardized initial data is processed by a deep analysis network to generate initial depth features;

[0141] S105, The initial depth features are processed using a first convolutional layer to generate first-scale depth features;

[0142] S106, use the second convolutional layer to process the first scale depth features to generate the second scale depth features;

[0143] S107, use the third convolutional layer to process the second scale depth features to generate the third scale depth features;

[0144] S108, the depth features at the first scale, the second scale, and the third scale are fused to generate a depth feature map.

[0145] In this embodiment, receiving initial data input refers to obtaining externally transmitted data content through the system interface. The initial data can be a single-frame image, a video keyframe, or other structured visual information. The data type is not limited to two-dimensional images and may also include multi-channel medical images or three-dimensional projection maps to ensure that the input data has complete spatial information.

[0146] Color space standardization of the initial data aims to unify the color representation of the data and avoid color deviations between data generated from different sources or devices affecting subsequent analysis. This process can be based on the standard RGB color space, Lab color space, or other standardized color representation methods, mapping the color distribution of each channel in the original data to a unified reference range. Common implementation methods include histogram equalization, mean-variance normalization, or color space conversion, ensuring consistency in the color dimension of input data from different batches or sources.

[0147] Adjusting the resolution of the standardized initial data to a preset size means standardizing the spatial dimensions of the input data to avoid discrepancies in feature representation caused by inconsistent sizes. This adjustment process can employ image resampling methods such as bilinear interpolation, cubic interpolation, and region scaling to adjust the data width and height to a fixed size range set by the system, such as 256×256 or 512×512 pixels, to meet the input specifications for subsequent deep analysis network structures while preserving key spatial details in the data.

[0148] The initial data is processed by a deep analysis network to generate initial depth features. The deep analysis network typically includes multi-level convolutional structures, non-linear activation units, and necessary downsampling or normalization operations. Its role is to extract structural information and deep expression features from the data. The initial depth features reflect the basic outline, edge information, and local texture patterns of the data in low-dimensional space.

[0149] The initial depth features are processed using the first convolutional layer to generate first-scale depth features. This convolutional operation typically sets a small receptive field, such as a 3×3 or 5×5 convolutional kernel, to focus on capturing local detail changes. The first-scale depth features preserve the basic information of the spatial structure and enhance the local expressiveness of the data.

[0150] The first-scale depth features are processed by a second convolutional layer to generate second-scale depth features. The second-scale features focus on spatial relationships over a larger range. The kernel size or hierarchical structure is expanded compared to the first scale, making it easier to capture spatial relationships and contour structure information over a medium range. The second-scale depth features reflect the mid-level semantic expression of the data.

[0151] The third convolutional layer is used to process the second-scale depth features to generate the third-scale depth features. The third-scale features focus on the global expressive power of the data. They are usually enhanced by further expanding the receptive field or introducing a multi-level stacked structure to improve the perception of the overall shape, structural outline and spatial context, and ensure that the final feature expression has multi-scale fusion capabilities.

[0152] The first-scale depth features, second-scale depth features, and third-scale depth features are fused to generate a depth feature map. The fusion process can be implemented in various ways, including channel-level stitching, weighted summation, or feature attention fusion mechanism. The purpose of the fusion operation is to combine the expressive advantages of features at each scale, retaining local details while strengthening the overall structure and spatial semantics. The resulting depth feature map has multi-level and full-space information expression capabilities, and is suitable for subsequent joint feature extraction, dynamic information analysis, and sparse attention optimization processes.

[0153] This embodiment, by receiving initial data and performing color space standardization, resolution adjustment, depth feature extraction, and multi-scale fusion processing, ensures that data inputs from different sources or of different qualities have a unified and standardized representation, avoiding the impact of input data quality differences on subsequent system stability. Simultaneously, the depth feature map integrates local detail information and global structural information, improving the spatial representation accuracy and structural coherence of subsequently generated data units. This provides a stable, complete, and rich multi-dimensional feature foundation for subsequent dynamic information extraction and sparse computation, significantly enhancing the stability and expressive power of the overall data sequence generation.

[0154] In one embodiment, step S20 above includes:

[0155] S201, Perform channel dimension expansion processing on the initial data to generate expanded initial data;

[0156] S202, Spatial alignment processing is performed on the depth feature map to generate an aligned depth feature map;

[0157] S203, concatenate the extended initial data and the aligned depth feature map to generate a concatenated feature matrix;

[0158] S204, The concatenated feature matrix is ​​processed by a feature transformation network to generate a transformed feature representation;

[0159] S205, The transformed feature representation is processed by a feature normalization layer to generate normalized features;

[0160] S206, The normalized features are fused through a cross-modal attention mechanism to generate a joint feature representation.

[0161] In this embodiment, channel dimension expansion of the initial data refers to increasing the data's expressive dimension in the channel direction by means of channel duplication, zero-padding, or feature mapping transformation, while maintaining the original spatial size. This operation can convert single-channel or low-channel data into a multi-channel structure according to different application scenarios. Common methods include mapping the initial data to a preset high-dimensional feature space based on convolutional transformation, or using a channel repetition strategy to enhance the data's expressive power. The purpose of channel expansion is to compensate for the insufficient dimensionality of the initial data in expressing complex features and adapting to subsequent deep fusion processing, ensuring compatibility with deep feature maps in terms of channel number and data structure.

[0162] Spatial alignment of depth feature maps addresses inconsistencies in spatial scale, location distribution, and boundary dimensions between the initial data and the depth feature map. Techniques such as image resampling, interpolation transformation, space filling, and cropping are employed to ensure complete spatial alignment. Common spatial alignment methods include bilinear interpolation for size adjustment, zero-padding for boundary completion, and cropping of redundant portions. This ensures a one-to-one correspondence between the expanded initial data and the depth feature map in width, height, and spatial location, facilitating the accuracy and stability of subsequent feature fusion processes.

[0163] The process of concatenating extended initial data and aligned depth feature maps to generate a concatenated feature matrix involves merging the two data structures into a unified high-dimensional feature matrix according to channel dimension after channel expansion and spatial alignment. This is typically achieved using a channel-level concatenation method, where the channel features of the extended initial data and the channel features of the aligned depth feature map are jointly stacked in the same spatial location to form a complete concatenated feature matrix. This matrix simultaneously contains both original visual information and depth spatial structure information, possessing richer multimodal expressive capabilities and providing a complete data foundation for subsequent feature transformation and fusion.

[0164] The feature transformation network processes the concatenated feature matrix to generate transformed feature representations. The feature transformation network is generally composed of multi-layer convolutional units, non-linear activation structures, and optional downsampling and normalization modules. It aims to perform deep feature extraction, structural optimization, and information compression on the concatenated feature matrix, eliminate the distribution differences and inconsistencies in feature representation between data from different sources, and ensure that the output transformed feature representation has a unified standard in terms of spatial representation, channel structure, and information density, thereby improving the discriminative ability and generalization effect of feature representation.

[0165] The feature normalization layer processes the transformed feature representation to generate normalized features. The feature normalization layer can adopt different methods such as batch normalization, layer normalization, instance normalization or adaptive normalization. Its function is to normalize and adjust the numerical distribution range of the transformed features, so as to avoid gradient explosion or vanishing problems in subsequent network processing due to unstable feature amplitude and distribution. The normalization operation further enhances the stability and robustness of the system under different input scenarios.

[0166] By fusing normalized features through a cross-modal attention mechanism, a joint feature representation is generated. This mechanism, through dynamic weight allocation and information interaction strategies, combines the complementary advantages of features from different sources in terms of spatial location, channel structure, and expressive content. It focuses on feature regions with strong expressive effects and high information value, while weakening redundant or interfering information, achieving efficient feature information coupling and deep fusion. The final output joint feature representation retains both the basic visual information from the initial data and the spatial structure information from the depth feature map, possessing higher expressive dimensions, stronger discriminative ability, and better adaptability.

[0167] This embodiment effectively addresses the differences in structure, size, and representation of data from different sources by performing channel expansion, spatial alignment, feature stitching, depth transformation, normalization, and cross-modal attention fusion on the initial data and depth feature maps. This improves the data fusion effect, enhances the system's ability to jointly express complex spatial structures and multimodal information, and significantly improves the accuracy, stability, and overall representation quality of subsequent data unit generation and sequence processing.

[0168] In one embodiment, step S30 above includes:

[0169] S301, The joint feature representation is processed by the feature decoder to generate the initial hidden state features;

[0170] S302, extend the initial hidden state features in terms of time dimension to generate extended hidden state features;

[0171] S303, The extended hidden state features are processed by the starting unit generation network to generate the starting data unit;

[0172] S304, Set the current data unit as the starting data unit;

[0173] S305, the preceding data unit of the current data unit is set as the starting data unit;

[0174] S306, initialize the timing state counter to the initial value.

[0175] In this embodiment, the joint feature representation is processed by a feature decoder to generate initial latent state features. The feature decoder typically includes multiple convolutional units, deconvolutional structures, upsampling modules, and nonlinear activation functions, aiming to gradually restore the high-dimensional abstract information in the joint feature representation into an intermediate feature structure with temporal expressive capabilities. During this process, the feature decoder dynamically adjusts network parameters based on the spatial dimensions and channel structure of the joint feature representation, ensuring that the output initial latent state features possess complete spatial information and temporal correlation. The feature decoder can employ convolutional neural networks, transformer structures, or a combination of both, depending on the specific application requirements, to meet the demands for feature restoration accuracy and expressive power in different scenarios.

[0176] Temporal dimension expansion is performed on the initial latent state features to generate extended latent state features. Temporal dimension expansion refers to transforming the original static feature structure into a dynamic feature matrix with temporal expressive capabilities by adding temporal channels, repeating frame structures, or constructing initial time slices, based on the initial latent state features. This expansion operation ensures the initial state of the data in the sequence dimension, providing structural support for subsequent time-series-based inference, evolution, and dynamic generation. Specific implementations of temporal dimension expansion can include replicating the initial latent state features along the time axis to form multiple frames of data, or introducing sequence position information through a time-coding structure.

[0177] The extended latent state features are processed by a starter unit generation network to generate starter data units. This network typically includes multi-layer temporal convolutional structures, dynamic fusion modules, and state prediction units, specifically designed to infer starter data units with complete data structures and sequence initial attributes based on the extended latent state features. By performing spatiotemporal joint modeling, feature compression, and state restoration on the extended latent state features, this network ensures that the output starter data units possess complete spatial information, rich semantic expression, and stable sequence initial states.

[0178] Setting the current data unit as the starting data unit means assigning the overall data structure and state information of the starting data unit to the current data unit, ensuring that the system has a clear source of starting information at the first time point of sequence generation. This operation directly relates to the data dependencies in subsequent processes such as motion complexity feature calculation, dynamic threshold mask generation, and sparse attention processing, ensuring the continuity and logical integrity of the sequence data structure.

[0179] Setting the preceding data unit of the current data unit as the starting data unit means initializing the preceding data unit of the current data unit to be the same as the starting data unit of the current data unit. This constructs the historical data association structure of the first frame of the sequence, satisfying the requirements of subsequent reasoning about the motion complexity characteristics based on the relationship between preceding and following data units. This initialization operation eliminates the reasoning bias and expression instability caused by the lack of historical data at the beginning of the sequence by maintaining the consistency of the data content between the preceding and current data units.

[0180] Initializing the timing state counter to its initial value means setting the timing state counter to a standard initial state based on system preset parameters or dynamic conditions, typically zero or the first frame index value. As a key indicator for tracking the progress and time node changes of data sequence generation within the system, the timing state counter directly affects the logical execution and data synchronization of subsequent iterative processing, dynamic threshold adjustment, and sparse attention updates. This initialization operation ensures that the sequence generation process has a unified and standardized time management mechanism and state tracking capability.

[0181] This embodiment performs feature decoding, temporal expansion, and starter unit generation operations sequentially on the joint feature representation. Combined with the synchronous initialization of the current data unit and the preceding data unit and the standard setting of the temporal state counter, it can effectively construct a starter data unit with complete spatial expression capabilities, sequence logical structure, and time management mechanism. This ensures the data continuity, structural stability, and inference accuracy in the subsequent sequence generation process, and improves the overall system's expression efficiency and processing reliability in multi-frame data modeling and sequence dynamic evolution.

[0182] In one embodiment, step S40 above includes:

[0183] S401, Extract the feature representation of the current data unit and generate the current feature representation;

[0184] S402, extract the feature representation of the preceding data unit of the current data unit, and generate the preceding feature representation;

[0185] S403, Based on the current feature representation and the preceding feature representation, generate a motion vector field;

[0186] S404, Determine the gradient magnitude characteristics of the motion vector field;

[0187] S405, perform spatial dimension averaging on the gradient magnitude feature to obtain the average motion intensity;

[0188] S406, normalize the average motion intensity to generate motion complexity features.

[0189] In this embodiment, motion complexity features are generated based on the current data unit and its preceding data units, involving dynamic change analysis of sequence data and quantification of spatial motion information. This process first requires extracting the feature representation of the current data unit to generate a current feature representation. The current data unit contains complete spatial structure information and sequence dynamic state. Its feature representation refers to the structural extraction and semantic compression of the original data of the current data unit through a feature encoder, convolutional network, or multilayer perceptron module, resulting in a low-dimensional, highly expressive feature matrix. The current feature representation retains the key spatial structure, texture information, and temporal state of the data unit, serving as the basic input for subsequent motion analysis.

[0190] Extract the feature representations of the preceding data units of the current data unit to generate a preceding feature representation. The preceding data unit of the current data unit is the state immediately preceding the current data unit in the sequence, representing the spatial structure and dynamic information of the previous time step. The preceding feature representation is obtained through the same feature extraction process as the current feature representation, ensuring the consistency of the data representation at two time points in terms of structure, dimension, and content, which facilitates subsequent difference analysis and motion calculation.

[0191] Based on the current feature representation and the preceding feature representation, a motion vector field is generated. The motion vector field refers to a set of vectors describing the spatial displacement relationships of data units in a sequence over time, typically implemented using optical flow estimation, temporal matching, or feature similarity calculation. In this process, by comparing the local or global positional changes of the current feature representation and the preceding feature representation within the spatial coordinate system, the direction and amplitude of motion for each spatial location or feature point are inferred, forming a complete motion vector field data structure. The motion vector field reflects the spatial variation trend and motion intensity distribution of data units across consecutive time nodes.

[0192] Determining the gradient magnitude characteristics of a motion vector field involves performing gradient operations on the motion vector field within a spatial range to calculate the absolute value or magnitude of the rate of change of motion at each location. Gradient magnitude characteristics can characterize the intensity and boundaries of motion in different spatial regions, reflecting the dynamic distribution and local motion complexity of sequence data in spatial structure. Specific calculation methods can employ gradient operators, partial derivative matrices, or higher-order difference structures.

[0193] Spatial dimensional averaging is applied to the gradient magnitude features to generate average motion intensity. Spatial dimensional averaging involves weighted or unweighted averaging of the gradient magnitude features across all spatial locations to obtain a scalar index reflecting the overall motion level. Average motion intensity measures the overall motion complexity of the current data unit and its predecessor data units within the global spatial scope, and is an important parameter for evaluating the dynamic properties of the sequence and controlling subsequent sparsity strategies.

[0194] The average motion intensity is normalized to generate motion complexity features. Normalization transforms the average motion intensity values ​​to a standardized range, eliminating inconsistencies caused by differences in resolution, scale, or amplitude between different data units. This ensures that the motion complexity features maintain stable expressive power and comparability under different data conditions. As a key indicator for measuring the dynamic attributes of a sequence, motion complexity features directly participate in subsequent dynamic threshold mask generation and sparse attention adjustment, affecting the expressive efficiency and resource allocation strategies of the sequence data.

[0195] For example, motion complexity estimation: define the complexity c of frame t. t Spatial average of optical flow gradient magnitude:

[0196]

[0197] Among them, c t The motion complexity feature of frame t is represented, which measures the overall motion intensity of that frame; H and W represent the height and width of the input image or feature map, used for normalizing the spatial average; F t This represents the optical flow field of the t-th frame calculated by FlowNet2.0, which represents the pixel-level motion information between the preceding data unit and the current data unit; The gradient of the optical flow field at spatial location (i,j) represents the local motion change at that location; ∥∥2 is the L2 norm, representing the gradient magnitude, which measures the intensity of local motion change; the final result is the average of this magnitude in space, which yields the overall motion complexity index.

[0198] Normalization process:

[0199]

[0200] The motion complexity features of all frames are normalized to ensure that the motion intensity between different frames is within the range of [0,1], which facilitates subsequent dynamic threshold control.

[0201] This embodiment extracts the feature representations of the current data unit and the preceding data unit, and combines them with the generation of motion vector fields, calculation of gradient magnitude, spatial averaging and normalization processing. This enables an accurate and quantitative characterization of the motion intensity and dynamic complexity of sequence data between consecutive time nodes, improving the accuracy of dynamic information expression and the stability of the data structure. It provides a reliable basis for subsequent dynamic resource scheduling and computation optimization based on motion complexity, and improves the processing efficiency and generation quality of sequence data.

[0202] In one embodiment, step S50 above includes:

[0203] S501, Perform convolution processing on the depth feature map to generate a depth feature representation;

[0204] S502, Perform linear transformation on the motion complexity features to generate transformed motion features;

[0205] S503, the depth feature representation and the transformation motion feature are fused to obtain the fused feature representation;

[0206] S504, The fused feature representation is processed by an activation function to generate an activated feature representation;

[0207] S505, Based on the activation feature representation, generate a dynamic threshold mask.

[0208] In this embodiment, a dynamic threshold mask is generated based on depth feature maps and motion complexity features, involving the process of multi-dimensional information fusion and dynamic sparse control parameter generation. First, the depth feature map is convolved to generate a depth feature representation. The depth feature map is a set of features reflecting the spatial structure hierarchy and scene depth distribution. Convolution processing refers to extracting local structural information and depth change features at different locations through the sliding operation of spatial convolution kernels, enhancing the spatial continuity and multi-scale perception capability of feature expression, and generating a depth feature representation as a compact expression of spatial structural information.

[0209] Linear transformation is applied to motion complexity features to generate transformed motion features. Motion complexity features are scalar or low-dimensional vectors that quantify the overall dynamic intensity of sequence data. Linear transformation refers to linearly mapping motion complexity features through parameter matrices or weight coefficients, adjusting their expression dimension and numerical range, improving their compatibility with deep feature representations in terms of data structure and numerical distribution, generating transformed motion features, and providing a unified data foundation for subsequent feature fusion.

[0210] By fusing deep feature representations and transformed motion features, a fused feature representation is generated. The fusion operation can combine spatial structure information and dynamic intensity information in the same feature space through feature concatenation, element-wise weighting, attention weighting, or feature reconstruction, thus generating a fused feature representation. This ensures that the spatial hierarchy information and sequence dynamic attributes contained in the data work together in the generation process of sparse control parameters, improving the adaptability and scene adaptability of the sparse strategy.

[0211] The fused feature representation is processed by an activation function to generate an activated feature representation. The activation function can be the Sigmoid function, the Tanh function, or other nonlinear mapping functions. This nonlinear transformation strengthens key discriminative information in the feature data, suppresses the interference of invalid regions on the sparsity control results, improves the nonlinear distribution capability and data discriminability of the feature representation, and generates an activated feature representation, which serves as an intermediate representation for dynamic threshold mask generation.

[0212] A dynamic threshold mask is generated based on the activation feature representation. The dynamic threshold mask is a sparse control matrix used in the attention mechanism. The dynamic aspect means that the mask value adaptively adjusts according to the spatial structure and dynamic features of the input data, avoiding the limitations of static masks in traditional sparse mechanisms. By setting threshold determination rules, high-weight regions in the activation feature representation are retained as active computational units, while low-weight regions are suppressed or discarded. This achieves dynamic optimization of sparse computational resources, ultimately generating a dynamic threshold mask that participates in subsequent sparsification operations of the attention mechanism.

[0213] For example, the formula for generating a dynamic threshold mask:

[0214] M i =σ(Conv2D(D;θ) d )+Linear(C;θ c ))

[0215] Among them, M i This is a dynamic threshold mask with a value range of [0,1]. N×N ; σ represents the Sigmoid activation function; Conv2D(D; θ d ) indicates that features are extracted from the depth map D through two-dimensional convolution, θ d Here are the convolution parameters; Linear(C; θ) c ) represents the mapping of the motion complexity feature C through a linear transformation, θ c These are the parameters for the linear transformation.

[0216] This embodiment integrates spatial depth information and motion complexity information to dynamically generate a threshold mask that can adaptively control the sparse computation range in the attention mechanism based on the actual spatial structure and dynamic attributes of the input data. This improves the efficiency of computing resource allocation and the accuracy of expressing key regions, reduces unnecessary computational overhead, and enhances the dynamic adaptability and sparse computation effect of sequence data in complex scenarios.

[0217] In one embodiment, step S60 above includes:

[0218] S601, Obtain the query feature representation, key feature representation, and value feature representation of the attention mechanism;

[0219] S602, Based on the query feature representation and the key feature representation, generate an attention weight matrix;

[0220] S603, apply the dynamic threshold mask to the attention weight matrix to generate a sparse attention weight matrix;

[0221] S604, modulate the value feature representation using the sparse attention weight matrix to generate a modulated feature representation;

[0222] S605, The modulation feature representation is processed by the decoder network to generate the hidden state of the next data unit;

[0223] S606, Generate the next data unit based on the hidden state of the next data unit.

[0224] In this embodiment, a dynamic threshold mask is applied to the attention mechanism to perform sparse attention processing and generate the next data unit. This involves dynamic filtering of multi-dimensional feature information, sparse matrix control, and sequence unit update operations. First, the query feature representation, key feature representation, and value feature representation of the attention mechanism are obtained. The query feature representation reflects the target information requirement of the current sequence position or data unit, the key feature representation is used to characterize the feature index information of each data unit in the sequence, and the value feature representation contains the original information content of each data unit in the sequence. The three types of feature representations are derived from different feature transformation paths of the input data, and are usually implemented through a feature projection layer or a multi-head transformation module to ensure that the feature dimension and data structure meet the requirements of matrix calculation.

[0225] An attention weight matrix is ​​generated based on the query feature representation and the key feature representation. The similarity relationship between the query features and key features is calculated through matrix multiplication to form the attention weight matrix, reflecting the distribution of correlation strength among different data units. This process typically employs a scaling factor to standardize the numerical range of the weight matrix and avoid gradient vanishing or exploding problems.

[0226] A dynamic threshold mask is applied to the attention weight matrix to generate a sparse attention weight matrix. The dynamic threshold mask provides a data-driven sparsity control strategy. Based on the dynamic threshold mask generated in the previous steps, some low-weight regions in the attention weight matrix are set to zero or weakened, while high-weight key regions are retained, forming a sparse attention weight matrix. This reduces the overall computational cost and enhances the information interaction between highly correlated data units.

[0227] A modulated feature representation is generated by modulating the value feature representation using a sparse attention weight matrix. The sparse attention weight matrix acts on the value feature representation to perform a weighted combination of information from different data units, generating the modulated feature representation. This process integrates the effective information of each data unit in the sequence and retains high-value data content under sparse constraints, improving information expression efficiency and highlighting important information.

[0228] The decoder network processes the modulation feature representation to generate the hidden state of the next data unit. It performs deep feature transformation and integrates sequence information into the modulation feature representation, generating the hidden state of the next data unit as the foundational information for subsequent sequence inference. The decoder network can be implemented using a self-attention structure, a feedforward network, or a fused multi-layer structure to ensure the integrity of the feature representation and the continuity of the sequence information.

[0229] Based on the hidden state of the next data unit, the next data unit is generated. Through specific state decoding rules or mapping functions, the hidden state information is transformed into a standard data unit format, and the next data unit is output as a new member in the sequence generation chain, participating in the subsequent sequence iteration process.

[0230] For example, the formula for generating the sparse attention weight matrix:

[0231]

[0232] in, M represents the sparse attention weight matrix; i ⊙ represents a dynamic threshold mask; ⊙ represents element-wise multiplication (Hadamard product); For standard attention matrix; Q i K i and V i d is the query, key, and value matrix obtained from the learnable weight mapping; d is the scaling factor for the current feature dimension.

[0233] The weighted update formula for dynamic threshold masks is as follows:

[0234]

[0235] Dynamic threshold mask M i Based on itself and the sparse attention weight matrix Adaptive weighting is applied to further optimize sparsity preservation. The results of sparsified attention are fed back to the dynamic threshold mask itself, achieving interactive fusion between the dynamic threshold mask and the attention mechanism. The aim is to dynamically adjust the mask weights through attention information, making the next attention process more adaptive and sparse.

[0236] Example Description: In the healthcare field, this study employs a data sequence generation method to improve the accuracy of cardiac motion simulation and dynamic prediction of lesion regions for the generation and temporal modeling of dynamic cardiac ultrasound images. First, raw cardiac ultrasound images are input as initial data. A sequence of original cardiac ultrasound images is received, and the input data's color space is standardized through grayscale mapping and brightness equalization to ensure consistency in the data distribution after color space standardization for ultrasound images acquired from different devices. Subsequently, the resolution of the standardized cardiac ultrasound images is adjusted to a preset size, such as uniformly adjusting it to 256×256 pixels, facilitating subsequent depth feature extraction operations.

[0237] The adjusted echocardiogram images are input into a depth analysis network. A cascaded convolutional structure is used to extract hierarchical depth information of cardiac tissue, progressively generating first-scale, second-scale, and third-scale depth features, corresponding to the boundaries of cardiac chambers, the myocardial motion region, and the fine structure of lesions, respectively. Based on the generated depth features at different scales, a feature fusion operation is performed to integrate multi-scale cardiac structural information and obtain a global depth feature map.

[0238] After acquiring the initial data and depth feature map, channel dimension expansion processing is performed on the ultrasound image, converting the single-channel grayscale image into a multi-channel feature matrix. Simultaneously, spatial geometric registration is performed on the depth feature map to ensure consistency with the initial image in spatial distribution. Subsequently, the expanded ultrasound image and the registered depth feature map are fused through feature stitching to form a stitched feature matrix. This stitched feature matrix is ​​then processed by a feature transformation network to generate a transformed feature representation expressing the latent patterns of cardiac motion. A feature normalization layer adjusts the feature value distribution to facilitate feature matching by the subsequent attention mechanism. The cross-modal attention mechanism effectively fuses image information and depth features in this process, generating a joint feature representation and fully extracting the joint spatial and depth structural features of cardiac motion.

[0239] The joint feature representation is input to the feature decoder, which extracts the latent state features of the initial cardiac motion and extends them along the temporal dimension to generate extended latent state features suitable for temporal modeling. The extended latent state features are processed by the starter unit generation network to construct the starter data unit, which serves as the time starting point of the cardiac ultrasound sequence. At the same time, the current data unit and its preceding data unit are set as the starter data unit, and the temporal state counter is initialized.

[0240] Based on the current data unit and the preceding data unit, the feature representations of the current echocardiogram image and the previous echocardiogram image are extracted. A motion vector field is calculated to quantify the spatial displacement between the current and previous cardiac tissue, and the gradient amplitude feature of the motion vector field is determined to measure the intensity of local changes in cardiac tissue motion. Further, the gradient amplitude feature is averaged spatially to calculate the global motion intensity of the current cardiac motion. The average motion intensity is then normalized to generate a motion complexity feature, which dynamically reflects the complexity of the current cardiac motion.

[0241] Based on deep feature maps and motion complexity features, convolutional processing is used to extract deep feature representations of the heart structure. Linear transformation is then used to process the motion complexity features, generating transformed motion features. These deep feature representations and transformed motion features are then fused to generate a fused feature representation. This fused feature representation is processed by an activation function to generate an activated feature representation. Finally, a dynamic threshold mask is generated based on this activated feature representation. This dynamic threshold mask adaptively adjusts the region of focus for attention under the current heart motion state.

[0242] The query feature representation, key feature representation, and value feature representation of the attention mechanism are obtained. An attention weight matrix between regions in the current cardiac motion data is calculated. A dynamic threshold mask is applied to sparsify the attention weight matrix, generating a sparse attention weight matrix. This sparse attention weight matrix is ​​used to modulate the value feature representation, highlighting the feature expressions of key cardiac motion regions and generating a modulation feature representation. The modulation feature representation is input into the decoder network to obtain the hidden state of the next data unit, and then the next data unit is generated, representing the cardiac ultrasound image at the next time step.

[0243] During sequence generation, the preceding data unit of the current data unit is updated to the current data unit, and the current data unit is updated to the next data unit to ensure the continuity of the temporal chain. The process of generating motion complexity features from the current data unit and its preceding data units, generating a dynamic threshold mask based on the motion complexity features, applying the dynamic threshold mask to perform sparse attention processing to generate the next data unit, and updating the data units is repeated continuously to generate multiple iterative data units, dynamically simulating the cardiac motion process.

[0244] We collect initial data units and multiple iterative data units to construct an initial cardiac ultrasound time-series sequence. We then use a 3D convolutional network to process the complete time-series sequence, extract the cardiac motion time-series features across time frames, and use a motion compensation mechanism to perform optical flow correction on the inter-frame displacement during cardiac motion to generate a motion-coherent optimized feature sequence. We use an upsampling network to restore the high-resolution structure of the cardiac ultrasound images, and through inter-frame continuity optimization processing, we remove motion jumps or tomographic phenomena during the simulation process, ultimately forming a complete cardiac ultrasound data sequence, which improves the coherence of cardiac motion simulation and its auxiliary value in clinical diagnosis.

[0245] In the fintech field, data sequence generation methods effectively enhance the temporal representation capabilities of transaction behavior modeling and dynamic risk assessment, particularly for transaction data flow, account behavior sequence generation, and fund flow trend prediction. First, initial account transaction data is acquired as the initial data set. This input includes structured data such as account fund change records, transaction instruction streams, and account behavior tags. The initial data input is received through a unified data parsing interface, and data standardization rules are used to convert the format of the account transaction data, ensuring consistency in the format of fields such as transaction instructions and fund changes across different data sources. Subsequently, data unit conversion and anomaly cleaning are performed, adjusting fields such as amount and timestamp in all transaction records to a unified standard, generating standardized initial data. Finally, resolution adjustment is applied to the standardized initial data, reducing the data granularity to the minute or second level to meet the needs of high-frequency transaction behavior sequence modeling.

[0246] Standardized account transaction data is input into a deep analytics network, which processes three dimensions sequentially: transaction amount fluctuations, account balance changes, and cross-account fund flows. This generates first-scale, second-scale, and third-scale deep features, respectively. The first-scale feature characterizes the intensity of trading orders, the second-scale feature describes account fund dynamics, and the third-scale feature tracks cross-account fund transfer paths. During the generation of these different scales, composite information from transaction intensity, fund changes, and cross-account paths is integrated to generate a deep feature map, which captures the overall fund flow structure of the account.

[0247] Based on the initial data and deep feature maps, the account transaction data is expanded in terms of channel dimension, transforming a single-field sequence into a multi-channel feature matrix. Simultaneously, time alignment and account number mapping are performed on the deep feature maps to ensure consistency between fund flow information and account transaction instructions over time. The expanded account transaction data and the aligned deep feature maps are then concatenated to form a concatenated feature matrix. A feature transformation network extracts transformed feature representations jointly expressing transaction data and account fund flows, and a feature normalization layer adjusts the distribution of feature values ​​such as transaction amount, account balance, and transaction frequency. Based on a cross-modal attention mechanism of account transaction instructions and historical fund paths, transaction behavior data and account structure data are dynamically fused to generate a joint feature representation.

[0248] The joint feature representation is input into the feature decoder to extract the latent state features of the account's current transaction behavior, and then extended along the temporal dimension to generate extended latent state features that can be continuously tracked. The extended latent state features are processed by the starter unit generation network to generate a starter data unit, which serves as the temporal starting point of the account's transaction behavior sequence. The current data unit and the previous data unit are both set to this starter data unit, and the temporal state counter is initialized.

[0249] Based on the current data unit and the preceding data unit, the feature representations of the current and previous trading behaviors of the account are extracted to generate a trading behavior vector field, which measures the difference in fund flow between the current and previous trading orders. Further, the gradient magnitude features of the trading behavior vector field are extracted to quantify the local volatility intensity of the account's trading behavior. The gradient magnitude features are then averaged along the account dimension to calculate the global trading behavior volatility intensity. Normalization is performed on the average trading behavior volatility intensity to generate the account's trading behavior motion complexity features, dynamically reflecting the changes in the account's current fund flow risk and trading frequency.

[0250] Based on deep feature maps and motion complexity features, convolutional processing is used to extract deep feature representations of the account's capital structure, and linear transformation is used to process the motion complexity features of trading behavior, generating transformed motion features. The deep feature representations and transformed motion features are fused to generate a fused feature representation, which is then processed by an activation function to generate an activated feature representation. This activated feature representation is used to generate a dynamic threshold mask, which adaptively adjusts the trading nodes and abnormal capital flow areas that require special attention in the current account's trading behavior sequence.

[0251] The query feature representation, key feature representation, and value feature representation of the attention mechanism are obtained. An attention weight matrix for the time sequence of account transaction behavior is generated based on the query and key features. A dynamic threshold mask is applied to the attention weight matrix to generate a sparse attention weight matrix. The sparse attention weight matrix is ​​used to modulate the value feature representation in the account transaction sequence, highlighting key nodes with abnormal capital fluctuations or sudden increases in transaction frequency, generating a modulated feature representation. The modulated feature representation is input into the decoder network to obtain the hidden state of the next data unit, generating the next data unit representing the account transaction state at the next time point.

[0252] The system continuously updates the preceding data unit of the current data unit and then updates the current data unit to become the next data unit, ensuring the temporal continuity of the account transaction behavior sequence. It repeatedly performs operations such as generating motion complexity features, generating dynamic threshold masks, performing sparse attention processing, generating the next data unit, and updating the data unit, generating multiple iterative data units to dynamically predict future changes in the account transaction behavior sequence.

[0253] The system collects initial data units and multiple iterative data units, combining them to form an initial sequence of account transaction behavior. A 3D convolutional network is then used to process the temporal data of account transaction behavior, extracting cross-temporal features of the account transaction path. A fund flow compensation mechanism is applied to adjust the fund flow deviation between cross-path transaction sequences of accounts, generating an optimized feature sequence. An upsampling network is then used to finely restore the temporal granularity of account transaction records. Through optimization of the continuity between account transaction frames, the abrupt impact of abnormal transaction points in the sequence on the global flow path is eliminated, ultimately forming an account transaction behavior data sequence. This enables the dynamic generation of account transaction trends and temporal early warning of future risk nodes.

[0254] This embodiment controls the sparsity of the attention mechanism through dynamic threshold masking. It can adaptively adjust the allocation of attention computing resources according to the spatial characteristics and dynamic information changes of the input data, effectively reducing the overall computational complexity in the sequence generation process, preserving the key expressions of high-value information regions, and improving the efficiency of sequence generation and the expression accuracy of output data.

[0255] In one embodiment, a data sequence generation apparatus is provided, which corresponds one-to-one with the data sequence generation method in the above embodiments. (Refer to...) Figure 3 , Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of the data sequence generation device of the present invention. The modules include a deep feature extraction module 10, a feature fusion module 20, a starting unit generation module 30, a motion feature analysis module 40, a dynamic threshold generation module 50, a sparse attention inference module 60, a data unit update module 70, an iterative sequence construction module 80, and a time-series sequence generation module 90. Detailed descriptions of each functional module are as follows:

[0256] The depth feature extraction module 10 is used to acquire initial data and extract the depth features of the initial data to obtain a depth feature map;

[0257] Feature fusion module 20 is used to fuse the initial data and the depth feature map to generate a joint feature representation;

[0258] The starting unit generation module 30 is used to generate a starting data unit based on the joint feature representation, and to set the current data unit and the preceding data unit of the current data unit as the starting data unit;

[0259] Motion feature analysis module 40 is used to generate motion complexity features based on the current data unit and the preceding data unit of the current data unit;

[0260] The dynamic threshold generation module 50 is used to generate a dynamic threshold mask based on the depth feature map and the motion complexity feature;

[0261] The sparse attention inference module 60 is used to apply the dynamic threshold mask to the attention mechanism, perform sparse attention processing, and generate the next data unit.

[0262] The data unit update module 70 is used to update the preceding data unit of the current data unit to the current data unit, and update the current data unit to the next data unit;

[0263] The iterative sequence construction module 80 is used to repeatedly execute the steps of generating motion complexity features based on the current data unit and the preceding data unit of the current data unit, generating a dynamic threshold mask based on the motion complexity features, applying the dynamic threshold mask to perform sparse attention processing to generate the next data unit, and updating the preceding data unit and the current data unit of the current data unit, thereby generating multiple iterative data units;

[0264] The time sequence generation module 90 is used to perform time sequence processing on the starting data unit and the multiple iterative data units to form a data sequence.

[0265] In one embodiment, the deep feature extraction module 10 is specifically used for:

[0266] Receive initial data input;

[0267] The initial data is subjected to color space standardization to generate standardized initial data;

[0268] Adjust the resolution of the standardized initial data to a preset size;

[0269] The standardized initial data is processed by a deep analytics network to generate initial depth features;

[0270] The initial depth features are processed using a first convolutional layer to generate first-scale depth features;

[0271] The first-scale depth features are processed using a second convolutional layer to generate second-scale depth features.

[0272] The second-scale depth features are processed using a third convolutional layer to generate third-scale depth features.

[0273] The first-scale depth features, the second-scale depth features, and the third-scale depth features are fused to generate a depth feature map.

[0274] In one embodiment, the feature fusion module 20 is specifically used for:

[0275] The initial data is subjected to channel dimension expansion processing to generate expanded initial data;

[0276] The depth feature map is spatially aligned to generate an aligned depth feature map;

[0277] The extended initial data and the aligned depth feature map are concatenated to generate a concatenated feature matrix;

[0278] The concatenated feature matrix is ​​processed by a feature transformation network to generate a transformed feature representation;

[0279] The transformed feature representation is processed by a feature normalization layer to generate normalized features;

[0280] The normalized features are fused using a cross-modal attention mechanism to generate a joint feature representation.

[0281] In one embodiment, the starting unit generation module 30 is specifically used for:

[0282] The joint feature representation is processed by a feature decoder to generate initial hidden state features;

[0283] The initial hidden state features are extended in a temporal dimension to generate extended hidden state features;

[0284] The extended hidden state features are processed by the starting unit generation network to generate the starting data unit;

[0285] The current data unit is set as the starting data unit;

[0286] The preceding data unit of the current data unit is set as the starting data unit;

[0287] Initialize the timing state counter to its initial value.

[0288] In one embodiment, the motion feature analysis module 40 is specifically used for:

[0289] Extract the feature representation of the current data unit and generate the current feature representation;

[0290] Extract the feature representation of the preceding data unit of the current data unit, and generate the preceding feature representation;

[0291] Based on the current feature representation and the preceding feature representation, a motion vector field is generated;

[0292] Determine the gradient magnitude characteristics of the motion vector field;

[0293] The gradient magnitude features are averaged spatially to obtain the average motion intensity.

[0294] The average motion intensity is normalized to generate motion complexity features.

[0295] In one embodiment, the dynamic threshold generation module 50 is specifically used for:

[0296] The depth feature map is convolved to generate a depth feature representation;

[0297] The motion complexity features are subjected to linear transformation to generate transformed motion features;

[0298] By fusing the depth feature representation and the transformation motion feature, a fused feature representation is obtained;

[0299] The fused feature representation is processed by an activation function to generate an activated feature representation;

[0300] A dynamic threshold mask is generated based on the activation feature representation.

[0301] In one embodiment, the sparse attention inference module 60 is specifically used for:

[0302] Obtain query feature representations, key feature representations, and value feature representations for the attention mechanism;

[0303] Based on the query feature representation and the key feature representation, an attention weight matrix is ​​generated;

[0304] The dynamic threshold mask is applied to the attention weight matrix to generate a sparse attention weight matrix;

[0305] The value feature representation is modulated by the sparse attention weight matrix to generate a modulated feature representation;

[0306] The modulation feature representation is processed by a decoder network to generate the hidden state of the next data unit.

[0307] The next data unit is generated based on the hidden state of the next data unit.

[0308] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used for communication with external user terminals via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a data sequence generation method on the server side.

[0309] In one embodiment, a computer device is provided, which may be a user terminal, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements the functions or steps of a data sequence generation method on the user side.

[0310] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0311] Acquire initial data and extract the depth features from the initial data to obtain a depth feature map;

[0312] The initial data and the deep feature map are fused to generate a joint feature representation;

[0313] A starting data unit is generated based on the joint feature representation, and the current data unit and the preceding data unit of the current data unit are set as the starting data unit;

[0314] Based on the current data unit and the preceding data unit of the current data unit, generate motion complexity features;

[0315] Based on the depth feature map and the motion complexity feature, a dynamic threshold mask is generated;

[0316] Apply the dynamic threshold mask to the attention mechanism to perform sparse attention processing and generate the next data unit;

[0317] Update the preceding data unit of the current data unit to the current data unit, and update the current data unit to the next data unit;

[0318] Repeatedly execute the steps of generating motion complexity features based on the current data unit and its predecessor data unit, generating a dynamic threshold mask based on the motion complexity features, applying the dynamic threshold mask to perform sparse attention processing to generate the next data unit, and updating the predecessor data unit and the current data unit to generate multiple iterative data units;

[0319] The initial data unit and the plurality of iterative data units are subjected to time-series processing to form a data sequence.

[0320] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0321] Acquire initial data and extract the depth features from the initial data to obtain a depth feature map;

[0322] The initial data and the deep feature map are fused to generate a joint feature representation;

[0323] A starting data unit is generated based on the joint feature representation, and the current data unit and the preceding data unit of the current data unit are set as the starting data unit;

[0324] Based on the current data unit and the preceding data unit of the current data unit, generate motion complexity features;

[0325] Based on the depth feature map and the motion complexity feature, a dynamic threshold mask is generated;

[0326] Apply the dynamic threshold mask to the attention mechanism to perform sparse attention processing and generate the next data unit;

[0327] Update the preceding data unit of the current data unit to the current data unit, and update the current data unit to the next data unit;

[0328] Repeatedly execute the steps of generating motion complexity features based on the current data unit and its predecessor data unit, generating a dynamic threshold mask based on the motion complexity features, applying the dynamic threshold mask to perform sparse attention processing to generate the next data unit, and updating the predecessor data unit and the current data unit to generate multiple iterative data units;

[0329] The initial data unit and the plurality of iterative data units are subjected to time-series processing to form a data sequence.

[0330] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and user side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0331] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0332] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0333] It should be noted that if any software tools or components not belonging to this company appear in the embodiments of this application, they are merely illustrative examples and do not represent actual use. The embodiments described above are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for generating a data sequence, characterized in that, Includes the following steps: Acquire initial data and extract the depth features from the initial data to obtain a depth feature map; The initial data and the deep feature map are fused to generate a joint feature representation; A starting data unit is generated based on the joint feature representation, and the current data unit and the preceding data unit of the current data unit are set as the starting data unit; Based on the current data unit and the preceding data unit of the current data unit, generate motion complexity features; Based on the depth feature map and the motion complexity feature, a dynamic threshold mask is generated; Apply the dynamic threshold mask to the attention mechanism to perform sparse attention processing and generate the next data unit; Update the preceding data unit of the current data unit to the current data unit, and update the current data unit to the next data unit; Repeatedly execute the steps of generating motion complexity features based on the current data unit and its predecessor data unit, generating a dynamic threshold mask based on the motion complexity features, applying the dynamic threshold mask to perform sparse attention processing to generate the next data unit, and updating the predecessor data unit and the current data unit to generate multiple iterative data units; The initial data unit and the plurality of iterative data units are subjected to time-series processing to form a data sequence.

2. The data sequence generation method as described in claim 1, characterized in that, Acquire initial data and extract the depth features from the initial data to obtain a depth feature map, including: Receive initial data input; The initial data is subjected to color space standardization to generate standardized initial data; Adjust the resolution of the standardized initial data to a preset size; The standardized initial data is processed by a deep analytics network to generate initial depth features; The initial depth features are processed using a first convolutional layer to generate first-scale depth features; The first-scale depth features are processed using a second convolutional layer to generate second-scale depth features. The second-scale depth features are processed using a third convolutional layer to generate third-scale depth features. The first-scale depth features, the second-scale depth features, and the third-scale depth features are fused to generate a depth feature map.

3. The data sequence generation method as described in claim 1, characterized in that, The initial data and the deep feature map are fused to generate a joint feature representation, including: The initial data is subjected to channel dimension expansion processing to generate expanded initial data; The depth feature map is spatially aligned to generate an aligned depth feature map; The extended initial data and the aligned depth feature map are concatenated to generate a concatenated feature matrix; The concatenated feature matrix is ​​processed by a feature transformation network to generate a transformed feature representation; The transformed feature representation is processed by a feature normalization layer to generate normalized features; The normalized features are fused using a cross-modal attention mechanism to generate a joint feature representation.

4. The data sequence generation method as described in claim 1, characterized in that, The starting data unit is generated based on the joint feature representation, and the current data unit and its predecessor data unit are set as the starting data unit, including: The joint feature representation is processed by a feature decoder to generate initial hidden state features; The initial hidden state features are extended in a temporal dimension to generate extended hidden state features; The extended hidden state features are processed by the starting unit generation network to generate the starting data unit; The current data unit is set as the starting data unit; The preceding data unit of the current data unit is set as the starting data unit; Initialize the timing state counter to its initial value.

5. The data sequence generation method as described in claim 1, characterized in that, Based on the current data unit and its preceding data units, motion complexity features are generated, including: Extract the feature representation of the current data unit and generate the current feature representation; Extract the feature representation of the preceding data unit of the current data unit, and generate the preceding feature representation; Based on the current feature representation and the preceding feature representation, a motion vector field is generated; Determine the gradient magnitude characteristics of the motion vector field; The gradient magnitude features are averaged spatially to obtain the average motion intensity. The average motion intensity is normalized to generate motion complexity features.

6. The data sequence generation method as described in claim 1, characterized in that, Based on the depth feature map and the motion complexity feature, a dynamic threshold mask is generated, including: The depth feature map is convolved to generate a depth feature representation; The motion complexity features are subjected to linear transformation to generate transformed motion features; By fusing the depth feature representation and the transformation motion feature, a fused feature representation is obtained; The fused feature representation is processed by an activation function to generate an activated feature representation; A dynamic threshold mask is generated based on the activation feature representation.

7. The data sequence generation method as described in claim 1, characterized in that, Applying the dynamic threshold mask to the attention mechanism, sparse attention processing is performed to generate the next data unit, including: Obtain query feature representations, key feature representations, and value feature representations for the attention mechanism; Based on the query feature representation and the key feature representation, an attention weight matrix is ​​generated; The dynamic threshold mask is applied to the attention weight matrix to generate a sparse attention weight matrix; The value feature representation is modulated by the sparse attention weight matrix to generate a modulated feature representation; The modulation feature representation is processed by a decoder network to generate the hidden state of the next data unit. The next data unit is generated based on the hidden state of the next data unit.

8. A data sequence generation apparatus, characterized in that, The data sequence generation device includes: The deep feature extraction module is used to acquire initial data and extract the depth features of the initial data to obtain a depth feature map; The feature fusion module is used to fuse the initial data and the deep feature map to generate a joint feature representation; The starting unit generation module is used to generate a starting data unit based on the joint feature representation, and to set the current data unit and the preceding data unit of the current data unit as the starting data unit; The motion feature analysis module is used to generate motion complexity features based on the current data unit and the preceding data units of the current data unit; A dynamic threshold generation module is used to generate a dynamic threshold mask based on the depth feature map and the motion complexity feature; The sparse attention inference module is used to apply the dynamic threshold mask to the attention mechanism, perform sparse attention processing, and generate the next data unit. The data unit update module is used to update the preceding data unit of the current data unit to the current data unit, and update the current data unit to the next data unit; The iterative sequence construction module is used to repeatedly execute the following steps: generating motion complexity features based on the current data unit and its predecessor data unit; generating a dynamic threshold mask based on the motion complexity features; applying the dynamic threshold mask to perform sparse attention processing to generate the next data unit; and updating the predecessor data unit and the current data unit, thereby generating multiple iterative data units. The time sequence generation module is used to perform time sequence processing on the starting data unit and the multiple iterative data units to form a data sequence.

9. A computer device, characterized in that, The computer device includes a memory, a processor, and a data sequence generation program stored in the memory and executable on the processor, wherein the data sequence generation program, when executed by the processor, implements the steps of the data sequence generation method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a data sequence generation program, which, when executed by a processor, implements the steps of the data sequence generation method as described in any one of claims 1-7.

Images