Behavior sequence structuring coding method and system based on high-order nonlinear dependence

Abstract

The application discloses a behavior sequence structured coding method based on high-order nonlinear dependence. First, a time sequence sensing embedding is performed on an original behavior sequence. Then, a network formed by stacking multiple structured coding units is used to perform hierarchical coding on the embedded sequence. Each coding unit comprises parallel processing paths, which are respectively used for processing first-order linear features and high-order nonlinear interaction features, and each order of features is weighted and combined through a learnable transformation module. Finally, the output of the network is aggregated and projected to generate a coding representation suitable for a downstream task. The application constructs a network structure comprising multiple order processing paths, so that the neural network can more effectively encode and utilize the structured high-order nonlinear dependence relationship hidden in the sequence data, and provide a more expressive feature representation for sequence modeling tasks.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and specifically relates to sequence data modeling and deep learning network architecture design. More specifically, it relates to a structured encoding method for behavioral sequences, used to explicitly model and represent high-order nonlinear dependencies existing in sequence data. Background Technology

[0002] The core of personalized recommendation algorithms lies in improving user experience by understanding and predicting user behavior. User behavior sequence data, as an important carrier for revealing user intentions and preferences, has evolved from traditional statistical methods to sequence-aware models based on deep learning. These models have made significant progress in capturing temporal dependencies.

[0003] However, user behavior decision-making processes exhibit complex high-order nonlinear dependencies, meaning that future behavior is influenced by nonlinear, cross-timestep interactions between multiple historical behaviors, rather than being a simple decision of a single behavior. For example, purchasing behavior may be driven by the combined effect of recent browsing and early favorites, forming a high-order association. While existing deep learning models introduce nonlinearity through activation functions, they are insufficient in explicitly capturing specific high-order interactions between sequential elements, such as multiplicative or polynomial dependencies. This limits the models' ability to uncover the deeper intentions behind user behavior.

[0004] Volterra series theory provides a mathematical framework for explicitly describing higher-order interactions in nonlinear systems. Its core lies in expressing the system output as a multi-order functional combination of the input history, thereby capturing complex dependencies that transcend linear relationships. Drawing upon this theory, it is hoped that deep learning models can more directly learn the hidden higher-order nonlinear features in data. However, when classical Volterra series are directly applied to high-dimensional, long-sequence data, the number of parameters and computational complexity increase dramatically with the interaction order and input dimension, limiting its general applicability in modern deep learning. Summary of the Invention

[0005] To overcome the limitations of directly applying classic Volterra series while retaining its core advantage of structured modeling of high-order interactions, this invention proposes a behavioral sequence structured encoding method based on high-order nonlinear dependencies. By designing a specific neural network structure containing parallel multi-order processing paths, it explicitly models and encodes the high-order nonlinear dependencies inherent in sequence data such as user behavior, providing more expressive feature representations for sequence modeling tasks.

[0006] According to one aspect of the present invention, a behavioral sequence structured encoding method based on high-order nonlinear dependencies is provided, comprising:

[0007] The input raw behavioral interaction sequence is converted into an initial sequence embedding representation injected with temporal information;

[0008] The initial sequence embedding representation is subjected to hierarchical feature extraction using a constructed multi-level interactive network. The multi-level interactive network is formed by stacking multiple structured coding units, wherein the output of the previous structured coding unit is the input of the current structured coding unit, and each structured coding unit contains parallel multi-level processing paths and subsequent fusion and activation modules.

[0009] The final sequence representation of the output of the multi-level interactive network is aggregated and projected.

[0010] As a further technical solution, the input original behavioral interaction sequence is converted into an initial sequence embedding representation injected with temporal information, including:

[0011] Map the core entity identifier of each interaction unit in the original behavioral interaction sequence to a feature embedding vector;

[0012] Map the position index of each time step in the original behavioral interaction sequence to a position embedding vector;

[0013] The feature embedding vector at the corresponding time step is fused with the position embedding vector to obtain the initial sequence embedding representation.

[0014] As a further technical solution, a hierarchical feature extraction is performed on the initial sequence embedding representation using a constructed multi-level interactive network, including:

[0015] The initial sequence embedding representation of the input is subjected to a first-order linear transformation and a first-order normalization to extract the output representation of the first-order feature path;

[0016] The initial sequence embedding representation of the input is constructed with multi-order interaction terms to generate a multi-order interaction feature tensor. The generated multi-order interaction feature tensor is then subjected to multi-order feature transformation and multi-level normalization to extract the output representation of the multi-order interaction feature path.

[0017] The output representations of each path are fused, and the fused features are activated to generate the final output of the current structured coding unit.

[0018] As a further technical solution, the method also includes:

[0019] The output of the previous structured coding unit is used as the input of the next structured coding unit. The output representations of the first-order feature path and the output representations of the multi-order interactive feature path are extracted again. The output representations of each path are then fused and activated until the final output of the last structured coding unit is obtained.

[0020] As a further technical solution, a multi-order interaction term construction is performed on the initial sequence embedding representation of the input, including:

[0021] Expand the input in K dimensions, each expansion being in a new dimension, to obtain K tensors that can be broadcast multiplied.

[0022] Perform element-wise multiplication on the K expanded tensors to obtain a high-dimensional tensor containing a combination of K-ary product features of all time steps.

[0023] The high-dimensional tensor is reshaped into a three-dimensional tensor.

[0024] As a further technical solution, the output representations of paths of different orders are fused, including:

[0025] The output representations of paths of different orders are fused by element-wise addition.

[0026] As a further technical solution, the final sequence representation output by the multi-order interactive network is aggregated and projected, including:

[0027] Extract the feature vectors of all batches at the last time step from the final sequence to form an aggregated representation;

[0028] The resulting aggregated representation is taken as input and linearly transformed through a learnable weight matrix to map the aggregated representation to the output dimension required by the target task.

[0029] According to one aspect of the present invention, a behavioral sequence structured coding system based on high-order nonlinear dependencies is provided, comprising:

[0030] The sequence-time-aware embedding module is used to convert the input raw behavioral interaction sequence into an initial sequence embedding representation injected with time information;

[0031] A hierarchical high-order dependency coding module is used to perform hierarchical feature extraction on the initial sequence embedding representation using a constructed multi-order interactive network. The multi-order interactive network is formed by stacking multiple structured coding units, wherein the output of the previous structured coding unit is the input of the current structured coding unit, and each structured coding unit contains parallel multi-order processing paths and subsequent fusion and activation modules.

[0032] The encoding aggregation and task projection module is used to aggregate and project the final sequence representation of the output of the multi-level interactive network.

[0033] According to one aspect of the present invention, a behavioral sequence structured encoding device based on higher-order nonlinear dependencies is provided, comprising a memory and a processor, wherein the memory stores program instructions that are executed by the processor, and the processor invokes the program instructions to execute the behavioral sequence structured encoding method based on higher-order nonlinear dependencies.

[0034] According to one aspect of the present invention, a non-transitory computer-readable storage medium is provided, the non-transitory computer-readable storage medium storing computer instructions that cause the computer to execute the described behavior sequence structured coding method based on high-order nonlinear dependencies.

[0035] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0036] 1. This invention constructs a specific network unit containing multi-order processing paths to neurally simulate the key mechanism of Volterra series combining and weighting input signals of different orders; at the same time, the method of this invention structurally generates and learns first-order linear features and higher-order interaction features in user behavior sequences, and gradually extracts deeper-level dependencies through hierarchical stacking.

[0037] 2. The core innovation of this invention lies in enabling neural networks to more effectively encode the high-order nonlinear dependencies hidden in sequence data in a structured manner. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart illustrating the behavioral sequence structured encoding method based on high-order nonlinear dependencies provided in an embodiment of the present invention.

[0040] Figure 2 This is a schematic diagram of a framework for structured coding of behavioral sequences based on high-order nonlinear dependencies, provided in an embodiment of the present invention. Detailed Implementation

[0041] The terms “comprising” and “having”, and any variations thereof, in the specification, claims, and accompanying drawings of this invention are intended to cover a non-exclusive inclusion, such as a process, method, system, product, or apparatus that includes a series of steps or units, not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined to form new technical solutions. Such combinations are not bound by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0043] The present invention provides a structured encoding method for behavioral sequences based on high-order nonlinear dependencies. Its core process includes three main stages: stage one is used for sequence temporal-aware embedding, stage two is used for hierarchical high-order dependency encoding, and stage three is used for encoding aggregation and task projection.

[0044] In Phase One, the input is defined as the user's original behavioral interaction sequence, which is then converted into an initial sequence embedding representation infused with temporal information. This phase mainly includes entity embedding, location embedding, and embedding fusion.

[0045] The entity embedding is used to map the core entity identifier of each interaction unit in the sequence to a dimension. The feature embedding vector. The position embedding is used to map the position index of each time step in the sequence to a feature embedding vector with dimension . The location embedding vector. After feature and location embedding, the embedding fusion fuses the feature embedding vector and the location embedding vector at the corresponding time step to obtain the initial sequence embedding representation. The vector dimension of each time step is The initial sequence embedding representation It will serve as the input for subsequent multi-stage interactive networks.

[0046] It should be noted that the interaction unit refers to the interaction information contained at that moment. For example, at a certain moment, the interaction behavior of a user a with product b, such as liking or commenting, includes this information and is called an interaction unit.

[0047] In stage two, a multi-level interactive network is constructed by stacking N structured coding units to perform hierarchical feature extraction on the initial sequence embedding representation generated in stage one. Each structured coding unit takes the output of the previous structured coding unit as its input and its internal structure includes parallel multi-level processing paths and subsequent fusion and activation modules.

[0048] The parallel multi-order processing paths include first-order linear feature paths and higher-order interactive feature paths. The first-order linear feature path uses a learnable first-order feature transformation module and a layer normalization module to linearly transform and normalize the input to extract first-order features. Each higher-order interactive feature path generates a K-order interactive feature tensor through a K-order interactive term construction module, and then transforms and normalizes this tensor through a learnable K-order feature transformation module and a K-level layer normalization module to learn K-order nonlinear dependencies.

[0049] After parallel processing, the structured coding unit fuses the output representations of each path through a multi-level feature fusion module, and activates the fused features through a non-linear activation module to generate the final output of the coding unit.

[0050] In the third stage, the final sequence representation of the output of the multi-level interactive network from the second stage is received and mapped to the output space required by the predetermined task through an output projection layer.

[0051] The output projection layer receives the final sequence representation output by stage two. Furthermore, through a linear transformation incorporating a learnable parameter layer, it is converted into the dimensions and form required for the target task, resulting in the final encoded representation. .

[0052] like Figure 1 and Figure 2 As shown in the embodiment of the present invention, the steps of a behavioral sequence structured encoding method based on high-order nonlinear dependencies include:

[0053] Phase 1: Generation of sequence embedding representations.

[0054] In this embodiment, the input data is the user's original behavioral interaction sequence, such as the user's history of watching videos and browsing products. First, the item ID of the input sequence is defined as the entity, and the maximum length of the sequence is defined as L. To ensure consistency in model processing, all sequences with a length less than L will be right-padded, and sequences exceeding L will be truncated.

[0055] Entity embedding module: The entity embedding layer is implemented using the torch.nn.Embedding module in the PyTorch deep learning framework. The input dimension of this embedding layer is the total number of entities in the project. The output embedding dimension is For an input ID sequence of length L, a lookup table operation yields a sequence of size L. Feature Embedding Tensor .

[0056] The position embedding module uses the `torch.nn.Embedding` module in the PyTorch deep learning framework to implement the absolute position embedding layer. The input dimension of this embedding layer is the maximum sequence length `L`, and the output embedding dimension is... For a sequence of length L, its position index sequence is... By looking up a table through this embedding layer, a value of [size missing] is obtained. Position Embedded Tensor .

[0057] Embedding and Fusion Module: To inject temporal information into the feature representation, the embedding and fusion module embeds the features into a tensor. With position embedding tensor The feature embedding dimension is then fused. In this embodiment, the feature embedding dimension is required. With position embedding dimension Equal, that is ,in Let be the hidden layer dimension of the model. The fusion operation uses element-wise addition. For the th in the sequence... Each time step, of which The corresponding fused embedding vector Through formula Perform calculations, where For the first Feature embedding vectors at each time step For the first The position embedding vectors at each time step are fused. The fused embedding vectors from all time steps are then stacked to obtain the complete initial sequence embedding representation. , The shape is This will serve as the input for subsequent multi-stage interactive networks.

[0058] Phase Two: Hierarchical High-Order Dependency Coding.

[0059] In this embodiment, this stage consists of N stacked structured coding units with identical structures. Hierarchical feature extraction is performed on the output of stage one to structurally encode higher-order nonlinear dependencies. Each coding unit processes the input and passes the output to the next coding unit.

[0060] Taking the i-th coding unit as an example, its input is... The coding unit contains parallel first-order linear feature paths and higher-order interactive feature paths.

[0061] First-order linear feature path: This path is used to extract linear features from the sequence. The first-order feature transformation module is implemented using a one-dimensional convolutional layer `torch.nn.Conv1d` in PyTorch, serving as a learnable first-order filter. This convolutional layer has L input channels, 1 output channel, and a kernel size of 1. Input Dimensions Where B is the batch size, and after processing by this module, it outputs a dimension of The tensor. The first-level normalization module uses the torch.nn.LayerNorm module in PyTorch, which operates on the feature dimension. The output of the first-order feature transformation module is normalized to obtain the output representation of the first-order feature path.

[0062] Higher-order interaction feature path: This path is used to explicitly model and extract K-order nonlinear interactions between sequence elements to capture higher-order nonlinear dependencies. The K-order interaction term construction module receives input... K-order interaction terms are generated by combining K elements in the sequence.

[0063] In this embodiment, the construction is achieved through the following steps: First, the input... Perform K dimensional expansions, each expanding in a new dimension, resulting in K tensors that can be broadcast multiplied. Then, perform element-wise multiplication on these K expanded tensors to obtain a high-dimensional tensor containing a combination of K-ary product features from all time steps. Finally, reshape this high-dimensional tensor into a three-dimensional tensor using the reshape operation in PyTorch, denoted as . Its dimensions are The second dimension of this tensor It contains all possible K-ary combinations of L time steps.

[0064] The K-order feature transformation module is implemented using a one-dimensional convolutional layer `torch.nn.Conv1d` in PyTorch, serving as a learnable K-order filter. The number of input channels in this convolutional layer is... The output channel count is 1, and the kernel size is 1. This module handles K-order interaction terms. Linear weighted sums and combinations are performed to learn the weights of K-order interaction patterns. The K-order normalization module uses the torch.nn.LayerNorm module in PyTorch and operates on the feature dimension. The output of the K-order feature transformation module is normalized to obtain the output representation of the K-order interactive feature path.

[0065] Multi-order feature fusion module: Multi-order feature fusion combines the output of the first-order path with the outputs of all other higher-order paths. In this embodiment, element-wise addition is used.

[0066] Output Processing and Residual Connection Module: This module performs final processing on the representation after multi-level feature fusion, generating the output of the encoding unit. Nonlinear activation applies the GELU activation function to the fused feature representation to enhance the model's nonlinear expressive power. To prevent overfitting, a `torch.nn.Dropout` layer from PyTorch is applied after the GELU activation function, randomly setting some activation values ​​to zero with a certain probability. To promote gradient flow and simplify network training, the processed features are combined with the input of the encoding unit. Residual connections are performed. In this embodiment, the connection is performed using element-wise addition. After the above processing, the final output of the i-th coding unit is obtained. .

[0067] Phase 3: Encoding aggregation and projection output.

[0068] This stage represents the final sequence after processing by N coding units. Process the data to generate output suitable for specific downstream tasks.

[0069] Sequence representation aggregation module: This module will The data is aggregated into a fixed-dimensional vector representing the entire sequence information. In this embodiment, the aggregation method involves selecting the representation of the last time step of the sequence, which contains the complete sequence information up to the current time step. Specifically, from... Extract the feature vectors of all batches at the last time step to obtain a dimension of... Aggregation representation .

[0070] Output Projection Module: This module uses the fully connected layer torch.nn.Linear in PyTorch as the output projection layer, which projects the aggregated representation. As input, it undergoes a linear transformation through a learnable weight matrix, mapping it to the output dimension required by the target task. In this embodiment, the output dimension is set to the total number of items. This linear transformation will Dimensional aggregation representation Convert to A dimensional predicted score vector, where each dimension corresponds to the predicted score for an item. The final encoded output dimension is... .

[0071] The implementation of the various embodiments of the present invention is based on programmed processing through a device with processor functionality. Therefore, in practical engineering, the technical solutions and functions of the various embodiments of the present invention are encapsulated into various modules. Based on this reality, and building upon the above embodiments, the embodiments of the present invention provide a behavioral sequence structured coding system based on higher-order nonlinear dependencies. This system is used to execute the behavioral sequence structured coding method based on higher-order nonlinear dependencies in the above method embodiments.

[0072] like Figure 2 As shown, the system includes: a sequence-time-aware embedding module, used to convert the input original behavioral interaction sequence into an initial sequence embedding representation injected with temporal information; a hierarchical high-order dependency coding module, used to perform hierarchical feature extraction on the initial sequence embedding representation using a constructed multi-order interaction network, wherein the multi-order interaction network is formed by stacking multiple structured coding units, wherein the output of the previous structured coding unit is the input of the current structured coding unit, and each structured coding unit contains parallel multi-order processing paths and subsequent fusion and activation modules; and a coding aggregation and task projection module, used to aggregate and project the final sequence representation output by the multi-order interaction network.

[0073] The behavioral sequence structured coding system based on high-order nonlinear dependencies provided in this invention addresses the limitations of directly applying classical Volterra series. Figure 2 Several modules within the system, through the design of specific neural network structures containing parallel multi-stage processing paths, explicitly model and encode the high-order nonlinear dependencies inherent in sequence data such as user behavior, providing more expressive feature representations for sequence modeling tasks.

[0074] It should be noted that the system embodiments provided by this invention, in addition to implementing the methods in the above method embodiments, are also used to implement the methods in other method embodiments provided by this invention. The difference lies only in setting corresponding functional modules, and their principles are basically the same as those of the above system embodiments provided by this invention. As long as those skilled in the art, based on the above system embodiments and referring to the specific technical solutions in other method embodiments, obtain corresponding technical means and technical solutions composed of these technical means by combining technical features, and improve the modules in the above system embodiments while ensuring the practicality of the technical solutions, they can obtain corresponding system-like embodiments for implementing the methods in other method-like embodiments. For example:

[0075] Based on the above system embodiments, as a preferred embodiment, the behavior sequence structured coding system based on high-order nonlinear dependencies provided in this embodiment of the invention, wherein the sequence time-aware embedding module is further configured to execute the following instructions:

[0076] Map the core entity identifier of each interaction unit in the original behavioral interaction sequence to a feature embedding vector;

[0077] Map the position index of each time step in the original behavioral interaction sequence to a position embedding vector;

[0078] The feature embedding vector at the corresponding time step is fused with the position embedding vector to obtain the initial sequence embedding representation.

[0079] Based on the above system embodiments, as a preferred embodiment, the hierarchical higher-order dependency encoding module of the behavioral sequence structured encoding system based on higher-order nonlinear dependencies provided in this embodiment of the invention is further configured to execute the following instructions:

[0080] The initial sequence embedding representation of the input is subjected to a first-order linear transformation and a first-order normalization to extract the output representation of the first-order feature path;

[0081] The initial sequence embedding representation of the input is constructed with multi-order interaction terms to generate a multi-order interaction feature tensor. The generated multi-order interaction feature tensor is then subjected to multi-order feature transformation and multi-level normalization to extract the output representation of the multi-order interaction feature path.

[0082] The output representations of each path are fused, and the fused features are activated to generate the final output of the current structured coding unit.

[0083] Based on the above system embodiments, as a preferred embodiment, the hierarchical higher-order dependency encoding module of the behavioral sequence structured encoding system based on higher-order nonlinear dependencies provided in this embodiment of the invention is further configured to execute the following instructions:

[0084] The output of the previous structured coding unit is used as the input of the next structured coding unit. The output representations of the first-order feature path and the output representations of the multi-order interactive feature path are extracted again. The output representations of each path are then fused and activated until the final output of the last structured coding unit is obtained.

[0085] Based on the above system embodiments, as a preferred embodiment, the hierarchical higher-order dependency encoding module of the behavioral sequence structured encoding system based on higher-order nonlinear dependencies provided in this embodiment of the invention is further configured to execute the following instructions:

[0086] Expand the input in K dimensions, each expansion being in a new dimension, to obtain K tensors that can be broadcast multiplied.

[0087] Perform element-wise multiplication on the K expanded tensors to obtain a high-dimensional tensor containing a combination of K-ary product features of all time steps.

[0088] The high-dimensional tensor is reshaped into a three-dimensional tensor.

[0089] Based on the above system embodiments, as a preferred embodiment, the behavior sequence structured coding system based on high-order nonlinear dependencies provided in this embodiment of the invention, wherein the coding aggregation and task projection module is further configured to execute the following instructions:

[0090] Extract the feature vectors of all batches at the last time step from the final sequence to form an aggregated representation;

[0091] The resulting aggregated representation is taken as input and linearly transformed through a learnable weight matrix to map the aggregated representation to the output dimension required by the target task.

[0092] Based on the same inventive concept as the foregoing embodiments, this embodiment of the invention also provides a behavioral sequence structured encoding device based on higher-order nonlinear dependencies, including a memory and a processor. The memory stores program instructions that are executed by the processor, and the processor calls the program instructions to execute the behavioral sequence structured encoding method based on higher-order nonlinear dependencies.

[0093] In embodiments of the present invention, the memory can be non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), or it can be volatile memory, such as random-access memory (RAM). Memory is any other medium capable of carrying or storing desired program code having an instruction or data structure form and accessible by a computer, but is not limited thereto. The memory in embodiments of the present invention can also be a circuit or any other device capable of implementing a storage function for storing program instructions and / or data.

[0094] In this embodiment of the invention, the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in this embodiment of the invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in this embodiment of the invention can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0095] Based on the same inventive concept as the foregoing embodiments, this embodiment of the invention also provides a non-transitory computer-readable storage medium that stores computer instructions that cause the computer to execute the behavior sequence structured encoding method based on high-order nonlinear dependencies.

[0096] In summary, this invention discloses a structured encoding method for behavioral sequences based on high-order nonlinear dependencies. First, the original behavioral sequence is time-awarely embedded. Then, a network formed by stacking multiple structured encoding units is used to hierarchically encode the embedded sequence. Each encoding unit contains parallel processing paths, respectively processing first-order linear features and high-order nonlinear interaction features. A learnable transformation module is used to weight and combine the features of each order. Finally, the network output is aggregated and projected to generate an encoded representation suitable for downstream tasks. This invention, by constructing a network structure containing multi-order processing paths, enables neural networks to more effectively encode and utilize the structured high-order nonlinear dependencies hidden in sequence data, providing more expressive feature representations for sequence modeling tasks.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A structured coding method for behavioral sequences based on high-order nonlinear dependencies, characterized in that, include: The input raw behavioral interaction sequence is converted into an initial sequence embedding representation injected with temporal information; The initial sequence embedding representation is subjected to hierarchical feature extraction using a constructed multi-level interaction network, including: performing a first-order linear transformation and a first-level normalization on the input initial sequence embedding representation to extract the output representation of the first-order feature path; constructing multi-level interaction terms on the input initial sequence embedding representation to generate a multi-level interaction feature tensor, and performing multi-level feature transformation and multi-level normalization on the generated multi-level interaction feature tensor to extract the output representation of the multi-level interaction feature path; fusing the output representations of each path and activating the fused features to generate the final output of the current structured coding unit; wherein the multi-level interaction network is formed by stacking multiple structured coding units, wherein the output of the previous structured coding unit is the input of the current structured coding unit, and each structured coding unit contains parallel multi-level processing paths and subsequent fusion and activation modules; The final sequence representation of the output of the multi-level interactive network is aggregated and projected.

2. The behavioral sequence structured coding method based on high-order nonlinear dependencies according to claim 1, characterized in that, The input raw behavioral interaction sequence is transformed into an initial sequence embedding representation injected with temporal information, including: Map the core entity identifier of each interaction unit in the original behavioral interaction sequence to a feature embedding vector; Map the position index of each time step in the original behavioral interaction sequence to a position embedding vector; The feature embedding vector at the corresponding time step is fused with the position embedding vector to obtain the initial sequence embedding representation.

3. The behavioral sequence structured coding method based on high-order nonlinear dependencies according to claim 1, characterized in that, The method further includes: The output of the previous structured coding unit is used as the input of the next structured coding unit. The output representations of the first-order feature path and the output representations of the multi-order interactive feature path are extracted again. The output representations of each path are then fused and activated until the final output of the last structured coding unit is obtained.

4. The behavioral sequence structured encoding method based on high-order nonlinear dependencies according to claim 1, characterized in that, The initial sequence embedding representation of the input is used to construct multi-order interaction terms, including: Expand the input in K dimensions, each expansion being in a new dimension, to obtain K tensors that can be broadcast multiplied. Perform element-wise multiplication on the K expanded tensors to obtain a high-dimensional tensor containing a combination of K-ary product features of all time steps. The high-dimensional tensor is reshaped into a three-dimensional tensor.

5. The behavioral sequence structured coding method based on high-order nonlinear dependencies according to claim 1, characterized in that, The output representations of paths of different orders are fused, including: The output representations of paths of different orders are fused by element-wise addition.

6. The behavioral sequence structured coding method based on high-order nonlinear dependencies according to claim 1, characterized in that, The final sequence representation output from the multi-order interactive network is aggregated and projected, including: Extract the feature vectors of all batches at the last time step from the final sequence to form an aggregated representation; The resulting aggregated representation is taken as input and linearly transformed through a learnable weight matrix to map the aggregated representation to the output dimension required by the target task.

7. A structured coding system for behavioral sequences based on high-order nonlinear dependencies, characterized in that, include: The sequence-time-aware embedding module is used to convert the input raw behavioral interaction sequence into an initial sequence embedding representation injected with time information; A hierarchical high-order dependency coding module is used to perform hierarchical feature extraction on the initial sequence embedding representation using a constructed multi-order interaction network. This includes: performing a first-order linear transformation and a first-order normalization on the input initial sequence embedding representation to extract the output representation of the first-order feature path; constructing multi-order interaction terms on the input initial sequence embedding representation to generate a multi-order interaction feature tensor, and performing multi-order feature transformation and multi-order normalization on the generated multi-order interaction feature tensor to extract the output representation of the multi-order interaction feature path; fusing the output representations of each order path and activating the fused features to generate the final output of the current structured coding unit; wherein the multi-order interaction network is formed by stacking multiple structured coding units, where the output of the previous structured coding unit is the input of the current structured coding unit, and each structured coding unit contains parallel multi-order processing paths and subsequent fusion and activation modules. The encoding aggregation and task projection module is used to aggregate and project the final sequence representation of the output of the multi-level interactive network.

8. A behavioral sequence structured coding device based on high-order nonlinear dependencies, characterized in that, The method includes a memory and a processor, wherein the memory stores program instructions that are executed by the processor, and the processor invokes the program instructions to execute the behavior sequence structured encoding method based on any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions that cause the computer to perform the behavior sequence structured coding method based on high-order nonlinear dependencies as described in any one of claims 1 to 6.

Images