Behavior sequence structured coding method and system based on high-order nonlinear dependency

Abstract

The invention discloses a behavior sequence structured coding method based on high-order nonlinear dependency, which comprises the following steps of: firstly, performing time sequence perception embedding on an original behavior sequence; then, hierarchical coding is carried out on the embedded sequence by stacking a network formed by a plurality of structured coding units, each coding unit comprises parallel processing paths which are respectively used for processing first-order linear features and high-order nonlinear interaction features, and the features of all orders are weighted and combined through a learnable transformation module; and finally, carrying out aggregation and projection on the output of the network, and generating a coding representation suitable for a downstream task. According to the method, a network structure containing a multi-order processing path is constructed, so that a neural network can more effectively encode and utilize a hidden structured high-order nonlinear dependency relationship in sequence data, and feature representation with higher expressive power is provided for a sequence modeling task.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of artificial intelligence, and particularly relates to sequence data modeling and deep learning network architecture design. More specifically, it relates to a structured encoding method for behavior sequence, which is used for explicitly modeling and representing high-order nonlinear dependency relationships existing in sequence data. BACKGROUND

[0002] The core of personalized recommendation algorithm is to improve user experience by understanding and predicting user behavior. As an important carrier for revealing user intention and preference, user behavior sequence data has developed from traditional statistical methods to deep learning-based sequence perception models. These models have made significant progress in capturing temporal dependencies.

[0003] However, the user behavior decision-making process exhibits complex high-order nonlinear dependencies, i.e., future behavior is influenced by nonlinear, cross-time step interactions between multiple historical behaviors, rather than the simple decision of a single behavior. For example, a purchase behavior may be driven by the combined effect of recent browsing and early collection, forming a high-order association. Although existing deep learning models introduce nonlinearity through activation functions, they lack the ability to explicitly capture specific high-order interactions between sequence elements, such as multiplicative or polynomial dependencies, which limits the model's ability to mine deep user behavior intentions.

[0004] Volterra series theory provides a mathematical framework for explicitly describing high-order interactions in nonlinear systems. Its core is to express the system output as a combination of multiple orders of input history functions, thereby capturing complex dependencies beyond linear relationships. Drawing on this theory, it is hoped to help deep learning models more directly learn the high-order nonlinear features hidden in the data. However, the classic Volterra series will have a sharp increase in the number of parameters and computational complexity when directly applied to high-dimensional, long sequence data, which limits its general applicability in modern deep learning. SUMMARY

[0005] To overcome the limitations of direct application of the classic Volterra series while retaining its core advantage of structured modeling of high-order interactions, the present application proposes a structured encoding method for behavior sequence based on high-order nonlinear dependencies, which explicitly models and encodes the high-order nonlinear dependencies contained in sequence data such as user behavior, by designing a specific neural network structure containing parallel multi-order processing paths, providing more expressive feature representations for sequence modeling tasks.

[0006] According to one aspect of the present application, a structured encoding method for behavior sequence based on high-order nonlinear dependencies is provided, comprising: converting the input original behavior interaction sequence into an initial sequence embedding representation with injected temporal information; perform hierarchical feature extraction on the initial sequence embedding representation by using the constructed multi-order interaction network, the multi-order interaction network being formed by stacking a plurality of structured encoding units, wherein the output of a previous structured encoding unit is the input of a current structured encoding unit, and each of the structured encoding units comprises parallel multi-order processing paths and a subsequent fusion and activation module; perform aggregation and projection output on the final sequence representation output by the multi-order interaction network.

[0007] As a further technical solution, the input original behavior interaction sequence is converted into an initial sequence embedding representation with injected timing information, including: mapping the core entity identifier of each interaction unit in the original behavior interaction sequence into a feature embedding vector; mapping the position index of each time step in the original behavior interaction sequence into a position embedding vector; fusing the feature embedding vector and the position embedding vector corresponding to the time step to obtain the initial sequence embedding representation.

[0008] As a further technical solution, the initial sequence embedding representation is subjected to hierarchical feature extraction by using the constructed multi-order interaction network, including: performing first-order linear transformation and first-order layer normalization on the input initial sequence embedding representation to extract the output representation of the first-order feature path; performing multi-order interaction term construction on the input initial sequence embedding representation to generate a multi-order interaction feature tensor, and performing multi-order feature transformation and multi-order layer normalization on the generated multi-order interaction feature tensor to extract the output representation of the multi-order interaction feature path; fusing the output representations of the paths of each order, and performing activation on the fused features to generate the final output of the current structured encoding unit.

[0009] As a further technical solution, the method further comprises: taking the output of a previous structured encoding unit as the input of a next structured encoding unit, again extracting the output representation of the first-order feature path and the output representation of the multi-order interaction feature path, and performing fusion and activation of the output representations of the paths of each order until the final output of the last structured encoding unit is obtained.

[0010] As a further technical solution, the input initial sequence embedding representation is subjected to multi-order interaction term construction, including: performing K-dimensional expansion on the input, each expansion being in a new dimension to obtain K tensors capable of broadcast multiplication; performing element-level multiplication operation on the K expanded tensors to obtain a high-dimensional tensor containing K-element product combinations of all time step features; The high-dimensional tensor is reshaped into a three-dimensional tensor.

[0011] As a further technical solution, the output representations of the paths of each order are fused, including: The output representations of the paths of each order are fused in an element-level addition manner.

[0012] As a further technical solution, the final sequence representation output by the multi-order interaction network is aggregated and projected, including: The feature vectors of all batches at the last time step are extracted from the final sequence to form an aggregated representation; The formed aggregated representation is taken as input, linearly transformed through a learnable weight matrix, and mapped to the output dimension required by the target task.

[0013] According to an aspect of the present application, a behavior sequence structured coding system based on high-order nonlinear dependence is provided, including: A sequence timing perception embedding module is configured to convert an input original behavior interaction sequence into an initial sequence embedding representation with injected timing information; A hierarchical high-order dependence coding module is configured to perform hierarchical feature extraction on the initial sequence embedding representation using a constructed multi-order interaction network, wherein the multi-order interaction network is formed by stacking a plurality of structured coding units, the output of a previous structured coding unit is the input of a current structured coding unit, and each structured coding unit includes parallel multi-order processing paths and subsequent fusion and activation modules; An encoding aggregation and task projection module is configured to aggregate and project the final sequence representation output by the multi-order interaction network.

[0014] According to an aspect of the present application, a behavior sequence structured coding device based on high-order nonlinear dependence is provided, including a memory and a processor, the memory stores program instructions executed by the processor, and the processor calls the program instructions to execute the behavior sequence structured coding method based on high-order nonlinear dependence.

[0015] According to an aspect of the present application, a non-transitory computer readable storage medium is provided, which stores computer instructions, and the computer instructions make the computer execute the behavior sequence structured coding method based on high-order nonlinear dependence.

[0016] Compared with the prior art, the present application has the following advantages: 1.The application simulates the key mechanism of Volterra series to combine and weight input signals of different orders in a neuralized manner by constructing a specific network unit containing multiple processing paths.

[0017] 2.The core innovation of the application is to enable neural networks to more effectively structure the coding of high-order nonlinear dependencies hidden in sequence data. BRIEF DESCRIPTION OF DRAWINGS

[0018] To more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0019] Figure 1 The flowchart of the behavior sequence structured coding method based on high-order nonlinear dependencies provided by the embodiments of the present application.

[0020] Figure 2 The framework diagram of the behavior sequence structured coding based on high-order nonlinear dependencies provided by the embodiments of the present application. DETAILED DESCRIPTION

[0021] The terms "include" and "have" and any variations thereof in the specification and claims of the present application and the above drawings are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device containing a series of steps or units does not have to be limited to the steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product or device.

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined to form new technical solutions. Such combinations are not bound by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0023] The present invention provides a structured encoding method for behavioral sequences based on high-order nonlinear dependencies. Its core process includes three main stages: stage one is used for sequence time-aware embedding, stage two is used for hierarchical high-order dependency encoding, and stage three is used for encoding aggregation and task projection.

[0024] In Phase One, the input is defined as the user's original behavioral interaction sequence, which is then converted into an initial sequence embedding representation infused with temporal information. This phase mainly includes entity embedding, location embedding, and embedding fusion.

[0025] The entity embedding is used to map the core entity identifier of each interaction unit in the sequence to a dimension. The feature embedding vector. The position embedding is used to map the position index of each time step in the sequence to a feature embedding vector with dimension . The location embedding vector. After feature and location embedding, the embedding fusion fuses the feature embedding vector and the location embedding vector at the corresponding time step to obtain the initial sequence embedding representation. The vector dimension of each time step is The initial sequence embedding representation It will serve as the input for subsequent multi-stage interactive networks.

[0026] It should be noted that the interaction unit refers to the interaction information contained at that moment. For example, at a certain moment, the interaction behavior of a user a with product b, such as liking or commenting, includes this information and is called an interaction unit.

[0027] In the second stage, a multi-order interaction network is constructed by stacking N structured encoding units to perform hierarchical feature extraction on the initial sequence embedding representation generated in the first stage. The input of each structured encoding unit is the output of the previous structured encoding unit, and the internal structure of each structured encoding unit includes parallel multi-order processing paths and a subsequent fusion and activation module.

[0028] The parallel multi-order processing paths include a first-order linear feature path and high-order interaction feature paths. The first-order linear feature path performs linear transformation and normalization on the input through a learnable first-order feature transformation module and a layer normalization module to extract first-order features. Each high-order interaction feature path generates a K-order interaction feature tensor through a K-order interaction term construction module, and then performs transformation and normalization on the tensor through a learnable K-order feature transformation module and a K-order layer normalization module to learn K-order nonlinear dependencies.

[0029] After parallel processing, the structured encoding unit fuses the output representations of the paths of different orders through a multi-order feature fusion module, and activates the fused features through a nonlinear activation module to generate the final output of the encoding unit.

[0030] In the third stage, the final sequence representation output by the second-stage multi-order interaction network is received, and an output projection layer is used to map the final sequence representation to an output space required by a predetermined task.

[0031] The output projection layer receives the final sequence representation output by the second stage , and converts it into a dimension and form required by the target task through linear transformation including a learnable parameter layer to obtain a final encoding representation .

[0032] As shown in Figure 1 and Figure 2 , the steps of the behavior sequence structured encoding method based on high-order nonlinear dependencies provided by the embodiments of the present application include: Stage one: generation of sequence embedding representation.

[0033] In this embodiment, the input data is the original behavior interaction sequence of a user, for example, the user's history of watching videos and browsing goods. First, the item ID of the input sequence is defined as an entity, and the maximum length of the sequence is defined as L. To ensure the consistency of the model processing, all sequences with a length less than L are right-padded, and sequences longer than L are truncated.

[0034] Entity embedding module: the torch.nn.Embedding module in the deep learning framework PyTorch is used to implement the entity embedding layer. The input dimension of the embedding layer is the total number of entities in the item , the output embedding dimension is For an input ID sequence of length L, a feature embedding tensor of size is obtained through a table lookup operation.

[0035] Position embedding module: the absolute position embedding layer is implemented by using the torch.nn.Embedding module in the deep learning framework PyTorch. The input dimension of the embedding layer is the maximum length L of the sequence, and the output embedding dimension is For a sequence of length L, its position index sequence is obtained through the embedding layer table lookup, and a position embedding tensor of size is obtained.

[0036] Embedding fusion module: in order to inject time sequence information into the feature representation, the embedding fusion module fuses the feature embedding tensor and the position embedding tensor In this embodiment, the feature embedding dimension is required to be equal to the position embedding dimension , that is , where is the hidden layer dimension of the model. The fusion operation adopts element-level addition. For the th time step in the sequence, where , the corresponding fused embedding vector is calculated by formula , where is the feature embedding vector of the th time step, and is the position embedding vector of the th time step. The fused embedding vectors of all time steps are stacked to obtain the complete initial sequence embedding representation , The shape of is , which is used as the input of the subsequent multi-order interaction network.

[0037] Stage two: hierarchical high-order dependency encoding.

[0038] In this embodiment, this stage is stacked by N structurally identical structured encoding units, which perform hierarchical feature extraction on the output of stage one to encode high-order nonlinear dependencies in a structured manner. Each encoding unit processes the input and passes the output to the next encoding unit.

[0039] Take the i th encoding unit as an example, the input of which is The encoding unit internally includes a parallel first-order linear feature path and a high-order interaction feature path.

[0040] First-order linear feature path: This path is used to extract linear features of the sequence. The first-order feature transformation module is implemented using a one-dimensional convolutional layer torch.nn.Conv1d in PyTorch as a learnable first-order filter. The input channel number of this convolutional layer is L, the output channel number is 1, and the kernel size is 1. The input has a dimension of , and the output of this module is a tensor with a dimension of . The first-order layer normalization module is implemented using the torch.nn.LayerNorm module in PyTorch, which acts on the feature dimension . The output of the first-order feature transformation module is normalized to obtain the output representation of the first-order feature path.

[0041] High-order interaction feature path: This path is used to explicitly model and extract K-order nonlinear interactions between sequence elements to capture high-order nonlinear dependencies. The K-order interaction term construction module receives the input , and generates K-order interaction terms by combining elements in the sequence into K-tuples.

[0042] In this embodiment, the construction is implemented by the following steps: first, the input is dimensionally expanded K times, each expansion is in a new dimension, resulting in K tensors that can be broadcast multiplied; then, element-level multiplication is performed on the K expanded tensors to obtain a high-dimensional tensor containing all K-tuple combinations of time step features; finally, the high-dimensional tensor is reshaped into a three-dimensional tensor using the reshape operation in PyTorch, denoted as , which has a dimension of . The second dimension of this tensor contains all possible K-tuple combinations of L time steps.

[0043] The K-order feature transformation module is implemented using a one-dimensional convolutional layer torch.nn.Conv1d in PyTorch as a learnable K-order filter. The input channel number of this convolutional layer is , the output channel number is 1, and the kernel size is 1. This module linearly weights and combines the K-order interaction terms to learn the weights of the K-order interaction patterns. The K-order layer normalization module is implemented using the torch.nn.LayerNorm module in PyTorch, which acts on the feature dimension . The output of the K-order feature transformation module is normalized to obtain the output representation of the K-order interaction feature path.

[0044] Multi-order feature fusion module: The multi-order feature fusion module fuses the output of the first-order path and the outputs of all other high-order paths. In this embodiment, element-wise addition is used.

[0045] Output processing and residual connection module: This module performs the final processing on the representation after multi-order feature fusion to generate the output of the encoding unit. A non-linear activation is applied to the fused feature representation using a GELU activation function to enhance the non-linear representation ability of the model. To prevent overfitting of the model, a torch.nn.Dropout layer in PyTorch is applied after the GELU activation function to randomly set some of the activation values to zero with a certain probability. To facilitate gradient flow and simplify network training, the features after the above processing are connected in residual connection with the input of the encoding unit . In this embodiment, the connection uses element-wise addition. After the above processing, the final output of the i-th encoding unit is obtained .

[0046] Stage three: encoding aggregation and projection output.

[0047] This stage processes the final sequence representation after processing by the N encoding units to generate an output suitable for a specific downstream task.

[0048] Sequence representation aggregation module: This module aggregates the sequence representation into a fixed-dimensional vector representing the information of the entire sequence. In this embodiment, the aggregation method is to select the representation of the last time step, which contains the complete sequence information up to the current time step. Specifically, the feature vectors of all batches at the last time step are extracted from to obtain an aggregated representation with a dimension of .

[0049] Output projection module: This module uses the fully connected layer torch.nn.Linear in PyTorch as the output projection layer, takes the aggregated representation as input, and performs a linear transformation through a learnable weight matrix to map it to the output dimension required by the target task. In this embodiment, the output dimension is set to the total number of items . The linear transformation converts the aggregated representation with a dimension of to a predicted score vector with a dimension of , where each dimension corresponds to the predicted score of an item. The final encoding output dimension is .

[0050] The implementation basis of each embodiment of the present application is achieved by the programmed processing of a device with processor function. Therefore, in engineering practice, the technical solutions and functions of each embodiment of the present application are packaged into various modules. Based on this actual situation, on the basis of each of the above embodiments, the embodiment of the present application provides a behavior sequence structured coding system based on high-order nonlinear dependence, which is used to execute the behavior sequence structured coding method based on high-order nonlinear dependence in the above method embodiment.

[0051] As shown in Figure 2 , the system comprises: a sequence timing perception embedding module, configured to convert an input original behavior interaction sequence into an initial sequence embedding representation with timing information injected; a hierarchical high-order dependence coding module, configured to perform hierarchical feature extraction on the initial sequence embedding representation by using a constructed multi-order interaction network, wherein the multi-order interaction network is formed by stacking a plurality of structured coding units, the output of a previous structured coding unit is the input of a current structured coding unit, and each structured coding unit comprises parallel multi-order processing paths and subsequent fusion and activation modules; and a coding aggregation and task projection module, configured to aggregate and project the final sequence representation output by the multi-order interaction network.

[0052] The behavior sequence structured coding system based on high-order nonlinear dependence provided by the embodiment of the present application faces the limitation of direct application of the classical Volterra series, adopts several modules in Figure 2 , and explicitly models and codes the high-order nonlinear dependence relationship contained in sequence data such as user behavior by designing a specific neural network structure comprising parallel multi-order processing paths, thereby providing more expressive feature representation for sequence modeling tasks.

[0053] It should be noted that the system embodiments provided by the present application are used to implement the methods in the above method embodiments, and are also used to implement the methods in other method embodiments provided by the present application. The difference is only that the corresponding functional modules are set, and the principle is basically the same as that of the above system embodiments provided by the present application. As long as the person skilled in the art, on the basis of the above system embodiments, refers to the specific technical solutions in other method embodiments, obtains the corresponding technical means by combining technical features, and the technical solutions composed of these technical means, as long as the technical solutions have practicality, the modules in the above system embodiments are improved to obtain the corresponding system class embodiments, which are used to implement the methods in other method class embodiments. For example:

[0054] Based on the content of the above system embodiments, as a preferred embodiment, the behavior sequence structured coding system based on high-order nonlinear dependence provided in the embodiment of the present application is also used to execute the following instructions: mapping each core entity identifier in each interaction unit in the original behavior interaction sequence into a feature embedding vector; mapping the position index of each time step in the original behavior interaction sequence into a position embedding vector; fusing the feature embedding vector and the position embedding vector of the corresponding time step to obtain an initial sequence embedding representation.

[0055] Based on the content of the above system embodiment, as a preferred embodiment, the behavior sequence structured coding system based on high-order nonlinear dependence provided in the embodiment of the application, the hierarchical high-order dependence coding module is further used to execute the following instructions: performing first-order linear transformation and first-order layer normalization on the input initial sequence embedding representation to extract the output representation of the first-order feature path; performing multi-order interaction term construction on the input initial sequence embedding representation to generate a multi-order interaction feature tensor, and performing multi-order feature transformation and multi-order layer normalization on the generated multi-order interaction feature tensor to extract the output representation of the multi-order interaction feature path; fusing the output representations of the paths of each order, and performing activation on the fused features to generate the final output of the current structured coding unit.

[0056] Based on the content of the above system embodiment, as a preferred embodiment, the behavior sequence structured coding system based on high-order nonlinear dependence provided in the embodiment of the application, the hierarchical high-order dependence coding module is further used to execute the following instructions: taking the output of the previous structured coding unit as the input of the next structured coding unit, again extracting the output representation of the first-order feature path and the output representation of the multi-order interaction feature path, and performing fusion and activation of the output representations of the paths of each order until the final output of the last structured coding unit is obtained.

[0057] Based on the content of the above system embodiment, as a preferred embodiment, the behavior sequence structured coding system based on high-order nonlinear dependence provided in the embodiment of the application, the hierarchical high-order dependence coding module is further used to execute the following instructions: performing K times dimension expansion on the input, each expansion being performed on a new dimension to obtain K tensors capable of broadcast multiplication; performing element-level multiplication operation on the K expanded tensors to obtain a high-dimensional tensor containing K-element product combinations of all time step features; reshaping the high-dimensional tensor into a three-dimensional tensor.

[0058] Based on the content of the above system embodiment, as a preferred embodiment, the behavior sequence structuring coding system based on high-order nonlinear dependence provided in the embodiment of the application, the encoding aggregation and task projection module are further used to execute the following instructions: extracting feature vectors of all batches at the last time step from the final sequence to form an aggregated representation; performing linear transformation on the formed aggregated representation through a learnable weight matrix to map the aggregated representation to an output dimension required by the target task.

[0059] Based on the same inventive concept as the foregoing embodiments, the embodiment of the application further provides a behavior sequence structuring coding device based on high-order nonlinear dependence, comprising a memory and a processor, the memory stores program instructions executed by the processor, and the processor invokes the program instructions to execute the behavior sequence structuring coding method based on high-order nonlinear dependence.

[0060] In the embodiment of the application, the memory can be a non-volatile memory such as a hard disk drive (HDD) or a solid-state drive (SSD), and can also be a volatile memory such as a random-access memory (RAM). The memory can be any other medium capable of carrying or storing desired program codes in the form of instructions or data structures and capable of being accessed by a computer, but is not limited to this. The memory in the embodiment of the application can also be a circuit or other any device capable of realizing a storage function, used for storing program instructions and / or data.

[0061] In the embodiment of the application, the processor can be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can realize or execute the disclosed methods, steps and logic block diagrams in the embodiment of the application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in combination with the embodiment of the application can be directly embodied as execution completed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0062] Based on the same inventive concept as the foregoing embodiments, the embodiment of the application further provides a non-transitory computer-readable storage medium storing computer instructions, the computer instructions causing the computer to execute the behavior sequence structuring coding method based on high-order nonlinear dependence.

[0063] In summary, the application discloses a behavior sequence structured coding method based on high-order nonlinear dependence, which firstly performs time sequence perception embedding on an original behavior sequence; then performs hierarchical coding on the embedded sequence through a network formed by stacking multiple structured coding units, wherein each coding unit contains parallel processing paths for processing first-order linear features and high-order nonlinear interaction features respectively, and each order of features is weighted and combined through a learnable transformation module; finally, the output of the network is aggregated and projected to generate a coding representation suitable for downstream tasks. The application builds a network structure containing multiple order processing paths, so that the neural network can more effectively encode and utilize the structured high-order nonlinear dependence relationship hidden in the sequence data, and provide more expressive feature representation for sequence modeling tasks.

[0064] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present application.

Claims

1. A behavioral sequence structuring coding method based on high-order nonlinear dependency, characterized in that, The method comprises the following steps: Converting an input original behavior interaction sequence into an initial sequence embedding representation with timing information injected; Performing hierarchical feature extraction on the initial sequence embedding representation by using a constructed multi-order interaction network, wherein the multi-order interaction network is formed by stacking multiple structured encoding units, the output of a previous structured encoding unit is the input of a current structured encoding unit, and each structured encoding unit comprises parallel multi-order processing paths and a subsequent fusion and activation module; Aggregating and projecting the final sequence representation output by the multi-order interaction network.

2. The method of claim 1, wherein the method is based on a high-order nonlinear dependence of the behavior sequence. The method for converting an input original behavior interaction sequence into an initial sequence embedding representation with timing information injected comprises the following steps: Mapping the core entity identifier of each interaction unit in the original behavior interaction sequence into a feature embedding vector; Mapping the position index of each time step in the original behavior interaction sequence into a position embedding vector; Fusing the feature embedding vector and the position embedding vector corresponding to the time step to obtain the initial sequence embedding representation.

3. The method of claim 1, wherein the method is based on a high-order nonlinear dependence of a behavior sequence. The method for performing hierarchical feature extraction on the initial sequence embedding representation by using a constructed multi-order interaction network comprises the following steps: Performing first-order linear transformation and first-order layer normalization on the input initial sequence embedding representation to extract the output representation of the first-order feature path; Performing multi-order interaction term construction on the input initial sequence embedding representation to generate a multi-order interaction feature tensor, and performing multi-order feature transformation and multi-order layer normalization on the generated multi-order interaction feature tensor to extract the output representation of the multi-order interaction feature path; Fusing the output representations of the paths of different orders, and activating the fused features to generate the final output of the current structured encoding unit.

4. The method of claim 3, wherein the method is based on a high-order nonlinear dependence of the behavior sequence. The method further comprises the following steps: Taking the output of a previous structured encoding unit as the input of a next structured encoding unit, and again extracting the output representation of the first-order feature path and the output representation of the multi-order interaction feature path, and fusing and activating the output representations of the paths of different orders until the final output of the last structured encoding unit is obtained.

5. The method of claim 3, wherein the method is based on a high-order nonlinear dependence of the behavior sequence. The method for performing multi-order interaction term construction on the input initial sequence embedding representation comprises the following steps: Performing K times dimension expansion on the input, each expansion being performed in a new dimension to obtain K tensors that can be subjected to broadcast multiplication; Performing element-level multiplication operation on the K expanded tensors to obtain a high-dimensional tensor containing K-element product combinations of features of all time steps; Reshaping the high-dimensional tensor into a three-dimensional tensor.

6. The method of claim 3, wherein the method is based on a high-order nonlinear dependence of the behavior sequence. The method for fusing the output representations of the paths of different orders comprises the following steps: Fusing the output representations of the paths of different orders in an element-level addition manner.

7. The method of claim 1, wherein the method is based on a high-order nonlinear dependence of a behavior sequence. The method for aggregating and projecting the final sequence representation output by the multi-order interaction network comprises the following steps: Extracting the feature vectors of all batches at the last time step from the final sequence to form an aggregated representation; Taking the formed aggregated representation as input, performing linear transformation through a learnable weight matrix, and mapping the aggregated representation to the output dimension required by the target task.

8. A behavioral sequence structuring coding system based on high order non-linear dependencies, characterized in that, The method comprises the following steps: A sequence timing perception embedding module is configured to convert an input original behavior interaction sequence into an initial sequence embedding representation with timing information injected. a hierarchical high-order dependency encoding module for hierarchical feature extraction of the initial sequence embedding representation using a constructed multi-order interaction network, the multi-order interaction network formed by stacking a plurality of structured encoding units, wherein the output of a previous structured encoding unit is the input of a current structured encoding unit, and each of the structured encoding units comprises parallel multi-order processing paths and a subsequent fusion and activation module; an encoding aggregation and task projection module for aggregating and projecting the final sequence representation output by the multi-order interaction network.

9. A behavioral sequence structuring coding device based on high order nonlinear dependency, characterized by, a computer readable storage medium storing computer instructions, the computer instructions causing the computer to perform the high-order nonlinear dependency based behavior sequence structured encoding method of any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, comprising: a computer readable storage medium storing computer instructions, the computer instructions causing the computer to perform the high-order nonlinear dependency based behavior sequence structured encoding method of any one of claims 1 to 7.

Images