Business table data processing method and device, electronic equipment and readable storage medium

Abstract

The invention relates to the technical field of data processing, and provides a business table data processing method and device, electronic equipment and a readable storage medium. The method comprises the following steps: performing feature decomposition processing on service table data to obtain continuous features corresponding to the service table data and category features corresponding to the service table data; performing gating residual processing on the continuous features to obtain continuous depth features; performing cross context coding processing on the category features to obtain cross coding category features; carrying out fusion processing on the continuous depth features and the cross coding category features to obtain fusion features corresponding to the business table data; and performing full-connection mapping processing on the fused features to obtain a business future demand prediction result corresponding to the business table data, thereby improving the information extraction depth of heterogeneous features in a complex business table, enhancing the capability of synchronous modeling of high-order interaction of category features and deep nonlinearity of continuous features, and improving the business future demand prediction accuracy. And the precision and generalization robustness of business future demand prediction are improved.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a method, apparatus, electronic device, and readable storage medium for processing business form data. Background Technology

[0002] In existing technologies, the conventional approach to processing business table data is to embed categorical features, standardize continuous features, and directly concatenate them into a long vector before feeding it into a single multilayer perceptron (MLP) or shallow network to complete training and prediction in one go. This method suffers from poor high-order interactions of categorical features and insufficient information extraction depth. The conventional monotonically decreasing learning rate can cause a misalignment of the convergence rhythm, resulting in a suboptimal trade-off. Furthermore, the lack of a collaborative training mechanism for heterogeneous dual-tower architectures leads to poor generalization robustness of the model on business data and large fluctuations in prediction accuracy.

[0003] It is evident that existing technologies suffer from insufficient feature information mining, weak model generalization ability, and low business prediction accuracy due to the simplistic heterogeneous feature processing methods. Summary of the Invention

[0004] In view of this, the present disclosure provides a business table data processing method, apparatus, electronic device and readable storage medium to solve the problems in the prior art that the single heterogeneous feature processing method leads to insufficient feature information mining, weak model generalization ability and low business prediction accuracy.

[0005] A first aspect of this disclosure provides a method for processing business table data, comprising: performing feature decomposition processing on the business table data to obtain continuous features and categorical features corresponding to the business table data; performing gated residual processing on the continuous features to obtain continuous deep features; performing cross-context encoding processing on the categorical features to obtain cross-coded categorical features; fusing the continuous deep features and the cross-coded categorical features to obtain fused features corresponding to the business table data; and performing fully connected mapping processing on the fused features to obtain a prediction result of future business demand corresponding to the business table data.

[0006] In some embodiments, gating residual processing is performed on continuous features to obtain continuous deep features, including: standardizing the continuous features to obtain normalized continuous features; gating linear mapping processing is performed on the normalized continuous features to obtain gating enhanced continuous features; and residual fusion processing is performed on the gating enhanced continuous features to obtain continuous deep features.

[0007] In some embodiments, cross-contextual encoding is performed on categorical features to obtain cross-encoded categorical features, including: performing value embedding lookup processing on categorical features to obtain value embedding vectors; performing column embedding assignment processing on categorical features to obtain column embedding vectors; concatenating and fusing the value embedding vectors and column embedding vectors to obtain feature embedding vectors; performing sequence combination processing on the feature embedding vectors to obtain feature embedding sequences; performing self-attention context weighting processing on the feature embedding sequences to obtain context enhancement sequences; and performing multi-layer feedforward fusion processing on the context enhancement sequences to obtain cross-encoded categorical features.

[0008] In some embodiments, the feature embedding sequence is subjected to self-attention context weighting to obtain a context-enhanced sequence, including: performing linear mapping on the feature embedding sequence to obtain a query vector, a key vector, and a value vector; calculating attention scores on the query vector and the key vector to obtain an initial attention score; normalizing the initial attention score to obtain an attention weight value; and performing weighted summation on the value vector based on the attention weight value to obtain the context-enhanced sequence.

[0009] In some embodiments, gating linear mapping is performed on normalized continuous features to obtain gated enhanced continuous features, including: segmenting the normalized continuous features to obtain gated sub-vectors and activation vectors; performing nonlinear activation on the activation vectors to obtain transformed activation vectors; performing gated transformation on the gated sub-vectors to obtain gated weight vectors; performing element-wise multiplication on the transformed activation vectors and gated weight vectors to obtain gated fusion vectors; and performing linear recovery on the gated fusion vectors to obtain gated enhanced continuous features.

[0010] In some embodiments, the continuous features are standardized to obtain normalized continuous features, including: centering the continuous features by mean to obtain center-shifted features; and scaling the center-shifted features by standard deviation to obtain normalized continuous features.

[0011] In some embodiments, performing fully connected mapping on the fused features to obtain the business future demand prediction result corresponding to the business table data includes: performing linear mapping on the fused features to obtain an initial mapping vector; performing nonlinear activation on the initial mapping vector to obtain an activated feature vector; performing linear mapping on the activated feature vector to obtain a mapped feature vector; and performing target dimension transformation on the mapped feature vector to obtain the business future demand prediction result.

[0012] A second aspect of this disclosure provides a business table data processing apparatus, comprising: a first processing module for performing feature decomposition processing on the business table data to obtain continuous features and categorical features corresponding to the business table data; a second processing module for performing gated residual processing on the continuous features to obtain continuous deep features; a third processing module for performing cross-context encoding processing on the categorical features to obtain cross-coded categorical features; a fourth processing module for performing fusion processing on the continuous deep features and cross-coded categorical features to obtain fused features corresponding to the business table data; and a fifth processing module for performing fully connected mapping processing on the fused features to obtain a prediction result of future business demand corresponding to the business table data.

[0013] A third aspect of this disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0014] A fourth aspect of this disclosure provides a readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0015] The beneficial effects of this disclosed embodiment compared with the prior art are as follows: By performing feature decomposition processing on business table data, continuous features and category features corresponding to the business table data are obtained; gated residual processing can be performed on continuous features to obtain continuous deep features; cross-context encoding processing can be performed on category features to obtain cross-coded category features; then, continuous deep features and cross-coded category features can be fused to obtain fused features corresponding to the business table data; fully connected mapping processing can be performed on the fused features to obtain the business future demand prediction results corresponding to the business table data; thereby, the information extraction depth of heterogeneous features in complex business tables is improved, the ability to synchronously model high-order interactions of category features and deep nonlinearities of continuous features is enhanced, and the accuracy and generalization robustness of business future demand prediction are improved. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram illustrating an application scenario of an embodiment of this disclosure; Figure 2This is a flowchart illustrating a business table data processing method provided in an embodiment of this disclosure; Figure 3 This is a flowchart illustrating another business table data processing method provided in this embodiment of the disclosure; Figure 4 This is a schematic flowchart of a residual block processing method provided in an embodiment of this disclosure; Figure 5 This is a schematic diagram of the structure of a business form data processing device provided in an embodiment of this disclosure; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0018] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, so as to provide a thorough understanding of the embodiments of this disclosure. However, those skilled in the art will understand that this disclosure may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this disclosure with unnecessary detail.

[0019] It should be noted that the user information (including but not limited to terminal device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0020] A business form data processing method and apparatus according to an embodiment of the present disclosure will now be described in detail with reference to the accompanying drawings.

[0021] Figure 1 This is a schematic diagram illustrating an application scenario of an embodiment of this disclosure. The application scenario may include terminal devices 1, 2, and 3, server 4, and network 5.

[0022] Terminal devices 1, 2, and 3 can be hardware or software. When terminal devices 1, 2, and 3 are hardware, they can be various electronic devices with displays and supporting communication with server 4, including but not limited to smartphones, tablets, laptops, and desktop computers. When terminal devices 1, 2, and 3 are software, they can be installed in the aforementioned electronic devices. Terminal devices 1, 2, and 3 can be implemented as multiple software programs or software modules, or as a single software program or software module; this disclosure does not limit this. Furthermore, various applications can be installed on terminal devices 1, 2, and 3, such as data processing applications, instant messaging tools, social platform software, search applications, shopping applications, etc.

[0023] Server 4 can be a server that provides various services, such as a backend server that receives requests sent by terminal devices with which it has established communication connections. This backend server can receive and analyze the requests sent by the terminal devices and generate processing results. Server 4 can be a single server, a server cluster consisting of several servers, or a cloud computing service center. This disclosure embodiment does not limit this.

[0024] It should be noted that server 4 can be either hardware or software. When server 4 is hardware, it can be various electronic devices that provide various services to terminal devices 1, 2, and 3. When server 4 is software, it can be multiple software programs or software modules that provide various services to terminal devices 1, 2, and 3, or it can be a single software program or software module that provides various services to terminal devices 1, 2, and 3. This disclosure does not limit the scope of the embodiments.

[0025] Network 5 can be a wired network using coaxial cable, twisted pair, and fiber optic connection, or it can be a wireless network that enables interconnection of various communication devices without wiring, such as Bluetooth, Near Field Communication (NFC), and Infrared. This disclosure does not limit the scope of the network.

[0026] Users can establish a communication connection with server 4 via network 5 through terminal devices 1, 2, and 3 to receive or send information. Specifically, server 4 can obtain business table data through terminal devices 1, 2, and 3, and obtain continuous features and categorical features corresponding to the business table data by performing feature decomposition processing on the business table data; it can perform gated residual processing on the continuous features to obtain continuous deep features; it can perform cross-context encoding processing on the categorical features to obtain cross-coded categorical features; then it can fuse the continuous deep features and cross-coded categorical features to obtain the fused features corresponding to the business table data; and it can perform fully connected mapping processing on the fused features to obtain the business future demand prediction results corresponding to the business table data.

[0027] It should be noted that the specific types, quantities, and combinations of terminal devices 1, 2, and 3, server 4, and network 5 can be adjusted according to the actual needs of the application scenario, and this disclosure embodiment does not impose any restrictions on this.

[0028] Figure 2 This is a flowchart illustrating a business table data processing method provided in an embodiment of this disclosure. Figure 2 Business table data processing methods can be provided by Figure 1 The server executes the command. For example... Figure 2As shown, the data processing method for this business table includes: S201, Perform feature decomposition on the business table data to obtain the continuous features and categorical features corresponding to the business table data.

[0029] Specifically, business table data can be a collection of data organized in tabular form. Business table data can contain multiple rows and columns, where each row can represent an independent sample or instance, and each column can represent a specific feature or attribute. Business table data can be used to provide the raw information foundation, enabling subsequent specialized processing flows to deeply mine the feature types of business table data. This business table data can be obtained from a database or real-time data stream through data acquisition.

[0030] Feature decomposition can be used to identify and separate the features in business table data into two groups, continuous features and categorical features, based on the data type. This provides specialized input for the subsequent dual-stream heterogeneous deep network architecture, ensuring that continuous features and categorical features can be fed into the processing stream for deep information extraction.

[0031] Among them, the continuous features corresponding to the business table data can be features that are numerically continuous and measurable, such as age, income, temperature, etc. There is no limitation here, and differences can be represented by numerical magnitude; continuous features can be used as input to continuous feature processing streams, and nonlinear patterns and complex relationships can be mined through deep residual networks; these continuous features can be extracted from the business table data through feature decomposition processing, and can be used to represent the quantitative information of the samples.

[0032] Categorical features can be features that take values ​​from a finite discrete set, such as gender, occupation, product category, etc. There is no limitation here. Categorical features can be used to represent classification or label information. Categorical features can be used as input to the categorical feature processing stream, and the transformer network can capture the high-order interactions and contextual dependencies between features.

[0033] Feature decomposition can be implemented through automated scripts or predefined rules, such as based on data type detection (e.g., integers or floating-point numbers can be identified as continuous features, while strings or enumeration types can be identified as categorical features). This ensures that the input data for subsequent processing is correctly classified, thus laying the foundation for specialized model architectures.

[0034] Furthermore, feature decomposition processing decomposes business table data into continuous features and categorical features. Specialized deep networks can be designed for the properties of each type of feature. For example, a deep residual network for table data can be used to mine deep nonlinear patterns for continuous features, while a transformer deep network can be used to capture higher-order interactions for categorical features. Feature decomposition processing can be implemented through programming interfaces or visualization tools. Moreover, feature decomposition processing can also support dynamic adjustment. For example, the feature type definition can be updated according to changes in data distribution, thereby enhancing the adaptability of the method in streaming data or incremental learning scenarios.

[0035] Furthermore, feature decomposition can be optimized by incorporating domain-specific knowledge. For example, in medical data, laboratory indicators can be used as continuous features, while diagnostic codes can be used as categorical features to better reflect the actual business logic.

[0036] This application embodiment performs feature decomposition on business table data to obtain continuous features and category features, providing a foundation for subsequent specialized deep network processing; it achieves automatic identification and separation of feature types, avoiding information loss; by decomposing into specialized inputs, it improves feature extraction capabilities and prediction accuracy, and enhances generalization and applicability.

[0037] For example, in financial risk control applications, business table data may include user transaction records, where transaction amount and account balance are continuous features, and user occupation and transaction type are categorical features. These features can be identified and separated through feature decomposition. Continuous features such as transaction amount can be analyzed using deep residual network analysis through continuous feature processing flow to detect abnormal patterns, while categorical features such as user occupation can be analyzed using transformer network analysis through categorical feature processing flow to capture the correlation between occupation and fraudulent behavior.

[0038] S202, gated residual processing is performed on continuous features to obtain continuous depth features.

[0039] Specifically, gated residual processing can be a neural network processing method combining gating mechanisms and residual connections. This gated residual processing can control the information flow through gate vectors and combine residual connections to achieve direct gradient propagation, thereby enhancing network depth and stability. Gated residual processing can adaptively filter information and transform features at deeper levels of continuous features, effectively capturing nonlinear relationships. Specifically, gated residual processing can be executed sequentially by multiple innovative residual blocks in the deep residual network of business table data. Each residual block can be implemented through layer normalization, linear transformation, and gated activation. The standardized continuous feature tensor can be input into a tabular data deep residual network, which can be composed of multiple stacked residual blocks. Each residual block can perform layer normalization on the continuous feature tensor to stabilize training. Then, the dimension can be expanded and split into two sub-vectors through the first linear layer. One sub-vector can be used as a gate signal, and the other sub-vector can be processed by an activation function and multiplied element-wise with the gate signal to achieve gated activation. The dimension can be restored through the second linear layer and added to the continuous feature tensor through residual connections to obtain continuous deep features.

[0040] By stacking innovative residual blocks multiple times, continuous features can be progressively transformed into high-dimensional continuous deep features. These continuous deep features can be vector representations that capture the high-level semantic information of the original continuous features. They can also be the output of gated residual processing, serving as the result of the continuous feature processing flow, and are used to integrate with the categorical feature context representation in concatenation fusion.

[0041] Layer normalization can be applied at the beginning of the residual block to normalize continuous features, reduce internal covariate bias, and improve training speed and stability; the normalized vector can be mapped to a higher dimension through the learnable weights in the first linear layer.

[0042] Furthermore, the expanded vector is divided into two equal-length sub-vectors. One sub-vector can be used as a gating signal (denoted as vector a), and the other sub-vector (denoted as vector b) can be processed by a non-linear activation function (such as ReLU). The gating signal vector a and the activated vector b can be multiplied element-wise to form the gated activation result, thereby achieving adaptive weighting of features, highlighting important information and suppressing noise. The gated activation result can be compressed back to the original input dimension through a second linear layer to ensure the additivity of the residual connections. Then, the output of the second linear layer can be added to the original input vector through residual connections to form the output of the residual block, so as to retain the original information and add the newly learned features. By stacking multiple residual blocks, continuous deep features are obtained.

[0043] This application embodiment achieves effective mining of deep nonlinear patterns of continuous features by performing gated residual processing on continuous features and combining a deep residual network structure with a gating mechanism and residual connections. The gated residual processing enhances the network's expressive power and training stability through adaptive information filtering and feature transformation, and avoids gradient vanishing. The gated residual processing, in conjunction with a periodic learning rate scheduler, improves the prediction accuracy and generalization ability in business table data processing.

[0044] S203, perform cross-context encoding on the categorical features to obtain cross-coded categorical features.

[0045] Specifically, categorical features can be converted into a sequence of embedding vectors through embedding lookup and concatenation, where the value embedding and column embedding of each categorical feature are concatenated to form an initial representation. This sequence of embedding vectors can calculate the attention score between features through self-attention, and can be weighted and summed based on the attention score to update the representation of each categorical feature. Through multi-layer stacking, high-order interactions can be mined to obtain cross-encoded categorical features.

[0046] Among them, the cross-encoded category feature can be the output representation obtained after cross-context encoding. The cross-encoded category feature can be used to characterize the context information contained in each category feature vector after interaction with other features. It is a high-dimensional and dense feature representation. The cross-encoded category feature can be obtained by processing the feature embedding sequence through a transformer deep network. For example, in the output of the last layer of the transformer network, each initial feature embedding can be converted into a vector containing context information. The set of these vectors is the cross-encoded category feature.

[0047] Furthermore, cross-context encoding can be performed by converting categorical features into a sequence of embedding vectors through embedding lookup and concatenation. Here, the value embeddings and column embeddings of each categorical feature can be concatenated to obtain an initial representation. This sequence can then be used to perform self-attention calculation through a transformer deep network to obtain attention scores between categorical features. Based on these attention scores, a weighted summation can be performed to update the representation of each categorical feature. Through multi-layer stacking, higher-order interactions can be mined to obtain cross-encoded categorical features.

[0048] This application embodiment performs cross-contextual encoding on category features and deeply mines the high-order interaction relationships between features through the self-attention mechanism of the transformer deep network, generating cross-encoded category features containing contextual information. This transforms the discrete values ​​of category features into continuous representations with enhanced interactions, improving the quality and information content of feature representations, ensuring that category features fully absorb relevant context during processing, and enhancing the ability to learn complex patterns.

[0049] For example, in user profiling analysis, category features can be "city", "occupation type", "purchase preference", etc. Category features can be separated into tensor form. Each feature value can be converted into a vector through an embedding layer, and column embedding can be concatenated to distinguish feature dimensions. The embedding sequence can be used to calculate the association between features through a self-attention mechanism in a deep network of transformers. For example, the feature "occupation type" as "engineer" can get a high attention score with the feature "purchase preference" as "electronic products", thereby fusing information during the encoding process. After processing by multiple transformers, cross-coded category features are obtained.

[0050] S204 fuses continuous depth features and cross-coded category features to obtain the fused features corresponding to the business table data.

[0051] Specifically, continuous deep features and cross-coded categorical features can be combined. By merging the complementary information of the two types of heterogeneous features, a more comprehensive and robust feature representation can be formed. It can receive output features from continuous feature processing streams and categorical feature processing streams, and perform vector concatenation along the feature dimensions to obtain the fused features corresponding to the business table data.

[0052] Among them, the fusion feature can be a combined vector obtained through fusion processing, which can be represented as a high-dimensional feature representation. This fusion feature integrates the information of continuous deep features and cross-coded category features. This fusion feature can be obtained by direct concatenation and fusion. Furthermore, the dimension of the concatenated fusion feature is equal to the sum of the dimensions of the two input features.

[0053] This application embodiment achieves efficient integration of the output of a dual-stream heterogeneous network by fusing continuous deep features and cross-coded category features. It combines the complementary information of the two types of features, avoids the model performance bottleneck caused by insufficient feature interaction, thereby improving the comprehensiveness and discriminative power of feature representation and enhancing the accuracy of downstream prediction tasks. The synergistic supplementation of heterogeneous information improves the generalization ability and stability.

[0054] For example, in a credit scoring system, business form data can contain user categorical features (such as occupation type and educational background) and continuous features (such as income level and debt ratio). These continuous deep features and cross-coded categorical features can be combined along the dimensions through fusion processing to obtain fused features. These fused features can contain categorical interaction semantics and continuous numerical patterns.

[0055] S205, perform fully connected mapping on the fused features to obtain the business future demand prediction results corresponding to the business table data.

[0056] Specifically, fully connected mapping processing can be a neural network processing based on MLP, which can map fused features to the target output space through linear transformation and nonlinear activation functions, thereby converting high-dimensional fused features into prediction results of future business needs. MLP can contain more than two hidden layers, and the number of neurons in each hidden layer can be adaptively adjusted according to the dimension of the fused features and the complexity of the prediction task.

[0057] The business future demand prediction result can refer to the estimated output of future business trends or demands, such as sales forecasts or user growth estimates, without limitation here; the business future demand prediction result can be the result obtained by transforming the fused features through fully connected mapping processing. This process can introduce nonlinearity into the fused features through activation functions (such as ReLU or Sigmoid); then the fused features can be transformed layer by layer, gradually compressing the feature dimensions and extracting abstract patterns related to the prediction target; the business future demand prediction result is obtained according to the task type (such as regression or classification).

[0058] The embodiments of this application transform the fused features through fully connected mapping processing, thereby improving the ability to represent complex tabular data; through the multi-layer nonlinear transformation of fully connected mapping processing, the level of automation and accuracy of prediction are enhanced, and the reliability and generalization ability of the prediction results are improved.

[0059] For example, in demand forecasting on e-commerce platforms, business table data can include users' historical behavior (such as number of clicks and purchase amount) and attribute information (such as region and membership level); through fully connected mapping processing, the predicted value of future product sales is output, providing a basis for inventory management.

[0060] According to the technical solution provided in this disclosure, by performing feature decomposition on business table data, continuous features and categorical features corresponding to the business table data are obtained; gated residual processing can be performed on the continuous features to obtain continuous deep features; cross-context encoding processing can be performed on the categorical features to obtain cross-coded categorical features; then, the continuous deep features and cross-coded categorical features can be fused to obtain fused features corresponding to the business table data; the fused features can be fully connected to obtain the business future demand prediction results corresponding to the business table data; thereby, the information extraction depth of heterogeneous features in complex business tables is improved, the ability to synchronously model high-order interactions of categorical features and deep nonlinearities of continuous features is enhanced, and the accuracy and generalization robustness of business future demand prediction are improved.

[0061] In some embodiments, gating residual processing is performed on continuous features to obtain continuous deep features, including: standardizing the continuous features to obtain normalized continuous features; gating linear mapping processing is performed on the normalized continuous features to obtain gating enhanced continuous features; and residual fusion processing is performed on the gating enhanced continuous features to obtain continuous deep features.

[0062] Specifically, standardization can be achieved by mathematically transforming the distribution of continuous features to a standard normal distribution with a mean of zero and a standard deviation of one, i.e., normalizing continuous features. These normalized continuous features can be the result of standardization or a continuous feature vector with a uniform scale obtained through Z-score transformation.

[0063] Gated linear mapping can be a process that combines linear transformation and gating mechanisms. It maps normalized continuous features to a high-dimensional space through a linear layer, and controls the information flow through a gating function to obtain gated enhanced continuous features, thereby enhancing the representational power of the normalized continuous features. Specifically, the gated enhanced continuous features can be high-dimensional feature vectors obtained through modulation processing using a gated activation function (such as a Gated Linear Unit). These gated enhanced continuous features can be obtained through gated linear mapping; specifically, the vector obtained by linear mapping can be divided into two parts: one part can be used as a gating signal, and the other part can be activated and multiplied by the gating signal.

[0064] Furthermore, gated linear mapping can map normalized continuous features to a higher dimension through a linear layer, thereby dividing the output vector into two sub-vectors. One sub-vector is processed by an activation function (such as ReLU), and the other sub-vector can be used as a gate signal. The two sub-vectors are multiplied element-wise to form gated enhanced continuous features.

[0065] In addition, residual fusion processing can be a processing method that combines residual connections. It can add gated enhanced continuous features to the original input (or the transformed input) element by element to preserve low-level information and promote gradient flow. Residual fusion processing can take gated enhanced continuous features as the main input and fuse them with continuous features or intermediate representations before normalization to obtain continuous depth features.

[0066] For example, in financial risk assessment, business table data can include continuous features such as user income, debt ratio, and transaction frequency. These continuous features can be standardized to obtain normalized continuous features to eliminate the influence of dimensions. Gated linear mapping can be used to enhance the representation of key features such as high debt ratio. Furthermore, residual fusion can be used to fuse the output of multi-layer networks to generate continuous deep features for credit score prediction.

[0067] According to the technical solution provided in this disclosure, by introducing a specialized gated linear mapping and residual fusion mechanism, the importance of features is dynamically adjusted through a gated activation function, and gradient vanishing is alleviated by residual connections. This significantly enhances the representation ability of continuous deep features, improves the fitting accuracy and generalization ability of complex tabular data, and ensures input stability by integrating the normalization process, gated linear mapping and residual fusion process in sequence. The gated mapping enhances feature interaction, and the residual fusion preserves the underlying information, enhances robustness and improves accuracy.

[0068] In some embodiments, cross-contextual encoding is performed on categorical features to obtain cross-encoded categorical features, including: performing value embedding lookup processing on categorical features to obtain value embedding vectors; performing column embedding assignment processing on categorical features to obtain column embedding vectors; concatenating and fusing the value embedding vectors and column embedding vectors to obtain feature embedding vectors; performing sequence combination processing on the feature embedding vectors to obtain feature embedding sequences; performing self-attention context weighting processing on the feature embedding sequences to obtain context enhancement sequences; and performing multi-layer feedforward fusion processing on the context enhancement sequences to obtain cross-encoded categorical features.

[0069] Specifically, the value embedding lookup process can be a computational process that retrieves the corresponding vector representation from a learnable embedding matrix based on the specific value of the category feature. It can map discrete category symbols to a continuous, dense vector space to obtain value embedding vectors. In this way, each category value can be given an optimizable numerical representation, thereby capturing the semantic information of the category value itself. The value embedding vector can be a fixed-dimensional real number vector obtained through the value embedding lookup process. The value embedding vector can be obtained by the value embedding lookup process and used as one of the inputs for subsequent splicing and fusion processing.

[0070] Column embedding assignment processing refers to the process of assigning an independent, learnable embedding vector to each category feature column. It can assign a shared context identifier to all sample values ​​under the same category feature column, thereby obtaining a column embedding vector, which distinguishes the meaning and role of different category feature columns. The column embedding vector can be an optimizable vector obtained through column embedding assignment processing and bound to a specific feature column. This column embedding vector can be used as an identifier to characterize the overall semantics and structure of the category feature column.

[0071] The concatenation and fusion process can be a process of connecting value embedding vectors and column embedding vectors along a specified dimension into a longer vector. It can integrate vector information from different sources, thereby combining value-specific semantic information and column-specific contextual information into a unified feature representation, namely the feature embedding vector. The feature embedding vector can be a combined vector obtained through the concatenation and fusion process, which can be used to represent the complete embedding representation of a single category feature.

[0072] Sequence combination processing can be a process of arranging multiple feature embedding vectors into a sequence structure according to the feature order in the sample. It is used to organize independent feature representations into a processable time series or serialized data format, i.e., feature embedding sequence, to construct a structured input so that subsequent self-attention mechanisms can capture the interaction relationships between features. The feature embedding sequence can be a vector sequence obtained through sequence combination processing. This feature embedding sequence can be used to represent an ordered set of all class features of a sample. The feature embedding sequence can also be used as input to the transformer encoder to provide a basis for context weighting.

[0073] Self-attention context-weighted processing can be a process of calculating the correlation and weighted fusion of each vector in the feature embedding sequence with all other vectors through a self-attention mechanism. It can dynamically adjust the representation of each element according to other elements in the feature embedding sequence to obtain a context-enhanced sequence. This can capture high-order interactions and dependencies between category features. For example, the features of "city" and "occupation" can jointly influence user behavior. The context-enhanced sequence can be obtained through self-attention context-weighted processing, where each vector incorporates global information from the feature embedding sequence.

[0074] Multi-layer feedforward fusion processing can be a process of independently transforming each vector in the context enhancement sequence through multiple fully connected layers and nonlinear activation functions. By further extracting and fusing high-level features at each position, cross-encoded category features are obtained, thereby enhancing the nonlinear representation capability of the model and stabilizing the feature distribution of the self-attention output. Among them, the cross-encoded category features can be the optimized category feature representation obtained through multi-layer feedforward fusion processing, which can be characterized as a high-order abstraction of category features obtained through deep context encoding and nonlinear transformation.

[0075] For example, in an e-commerce platform, business table data can include user profiles and product attributes. Category features can include the user's city, occupation type, product brand, and product category. Value embedding lookup processing can be performed on category features. For example, the corresponding value embedding vector can be retrieved from the embedding matrix based on the city value "City A". At the same time, a column embedding vector can be assigned to the city column to identify the feature dimension. Then, the value embedding vector and the column embedding vector can be concatenated to form a feature embedding vector. For example, the city feature embedding vector can include the semantics of "City A" and the context of the "City" column. All feature embedding vectors can be combined into a feature embedding sequence according to the original column order. For example, the sequence position 1 corresponds to the city, position 2 corresponds to the occupation, etc. This feature embedding sequence can be processed through self-attention context weighting. For example, the attention weights of city features and occupation features can be calculated to capture the combined effect of "City A programmer" to generate a context-enhanced sequence. Then, the sequence can be nonlinearly transformed through multi-layer feedforward fusion processing to obtain cross-coded category features.

[0076] According to the technical solution provided in this disclosure, by separating and concatenating value embedding and column embedding of category features, fine-grained vectorized representation of discrete features is achieved, enhancing the ability to capture the dual semantics of feature values ​​and feature columns. Through sequence combination and self-attention context weighting, high-order interaction relationships between features are dynamically captured, improving the depth of mining hidden patterns in complex tabular data. Through multi-layer feedforward fusion processing, feature representation is further refined and stabilized, ensuring the quality and robustness of cross-encoded category features, improving the sufficiency and specialization of category feature information extraction, and providing richer contextual information for subsequent fusion and prediction, thereby improving the accuracy and generalization ability in business tabular data prediction tasks.

[0077] In some embodiments, the feature embedding sequence is subjected to self-attention context weighting to obtain a context-enhanced sequence, including: performing linear mapping on the feature embedding sequence to obtain a query vector, a key vector, and a value vector; calculating attention scores on the query vector and the key vector to obtain an initial attention score; normalizing the initial attention score to obtain an attention weight value; and performing weighted summation on the value vector based on the attention weight value to obtain the context-enhanced sequence.

[0078] Specifically, linear mapping can be a process of projecting the feature embedding sequence into different vector spaces using a learnable weight matrix. This process can convert the feature embedding sequence into three different representations: a query vector, a key vector, and a value vector. The query vector can be used to represent the current feature as the "interrogator" in the attention mechanism, the key vector can be used to represent other features as the "interrogated" features, and the value vector can be used to represent the original information content of the feature. The query vector, key vector, and value vector can be generated by multiplying the feature embedding sequence by three independent linear transformation matrices.

[0079] Attention score calculation can be a process for measuring the similarity between vectors. It can be implemented through dot product or additive attention to quantify the correlation strength between each feature in the feature embedding sequence and other features, thus obtaining an initial attention score. For example, calculating the correlation between the "city" feature and the "education level" feature can yield a higher-order interaction pattern between features. The initial attention score can be the original similarity value obtained by calculating the attention score, which can be used to characterize the degree of unnormalized association between feature pairs.

[0080] Furthermore, normalization can be achieved by converting the initial attention scores into a probability distribution using the Softmax function, thereby obtaining attention weight values. This ensures that the sum of all attention weights is 1, thus maintaining numerical stability during weighted summation and highlighting the contributions of important features. The attention weight values ​​can be normalized scores, which can be used to characterize the relative importance of each feature in constructing the context representation.

[0081] Furthermore, the weighted summation process can be a linear combination of value vectors based on attention weights, thereby aggregating the information of all features to form a context-aware representation of each feature, i.e., a context-enhanced sequence, thus enhancing the semantic richness of the features.

[0082] For example, in an e-commerce platform, the system receives business table data containing user attributes (such as city and gender) and purchase history (such as recent purchase amount and number of clicks). Categorical features (city and gender) can be converted into feature embedding sequences. Query vectors, key vectors, and value vectors can be obtained through linear mapping. Initial attention scores can be calculated to evaluate the relationships between features (such as the interaction between city and gender). The initial attention scores can then be normalized to obtain attention weight values. Based on the attention weight values, the value vectors can be weighted and summed to generate a context-enhanced sequence.

[0083] According to the technical solution provided in the embodiments of this disclosure, diversified representations of feature embedding sequences are achieved through self-attention context weighting processing, enhancing the ability to capture complex relationships between features; through attention score calculation and normalization processing, the intensity of feature interaction is accurately quantified, and the rationality of weight distribution is ensured; based on the weights, the value vectors are weighted and summed to generate a context-enhanced sequence, so that each feature representation incorporates global context information, improving the modeling accuracy of high-order interactions of categorical features and improving overall prediction performance.

[0084] In some embodiments, gating linear mapping is performed on normalized continuous features to obtain gated enhanced continuous features, including: segmenting the normalized continuous features to obtain gated sub-vectors and activation vectors; performing nonlinear activation on the activation vectors to obtain transformed activation vectors; performing gated transformation on the gated sub-vectors to obtain gated weight vectors; performing element-wise multiplication on the transformed activation vectors and gated weight vectors to obtain gated fusion vectors; and performing linear recovery on the gated fusion vectors to obtain gated enhanced continuous features.

[0085] Specifically, the gated sub-vector can be a partial vector separated from the normalized continuous features through a segmentation process, which can be used for subsequent gated weight calculation. This gated sub-vector can be a half segment of the feature vector, or it can be obtained by equally dividing the normalized continuous features through the output of a linear layer. The activation sub-vector can be another part of the vector obtained at the same time as the gated sub-vector through segmentation, which can be used for nonlinear activation.

[0086] Furthermore, the segmentation process can be a process of uniformly dividing the normalized continuous features output by the linear layer into two equal-length sub-vectors along the feature dimension, thereby ensuring a balance of information content in the gating and activation paths. For example, in the processing of financial risk control table data, the normalized continuous features may contain standardized values ​​such as user age and income, and the gating sub-vector and activation sub-vector can be obtained through segmentation.

[0087] Nonlinear activation processing can be a process of transforming the activation vector through nonlinear functions (such as ReLU, GELU, etc.). Nonlinear activation processing can be a process of introducing nonlinear relationships through mathematical functions, thereby enhancing the ability to capture complex nonlinear patterns in continuous features. Among them, the transformed activation vector can be the output vector obtained through nonlinear activation processing. This transformed activation vector can be the result of nonlinear transformation of the activation vector, and can carry the feature information after nonlinear transformation for use in gating fusion.

[0088] Furthermore, nonlinear activation processing can be based on the ReLU function. By setting the negative values ​​in the activation vector to zero and retaining the positive values, a nonlinear transformation is performed, thereby highlighting important features and suppressing noise. For example, in medical diagnostic table data, the activation vector can be used to represent standardized continuous indicators such as blood pressure and heart rate. After ReLU activation, a transformed activation vector is obtained.

[0089] Gating transformation processing can be a process of applying linear or simple nonlinear transformations to gated subvectors to generate weight coefficients. It can be a process of calculating dynamic gating values ​​through weight matrices or functions, thereby adaptively generating weights based on input features, which can be used to control the information flow of the activation path. The gating weight vector can be the output vector obtained from the gating transformation processing. This gating weight vector can be a weight coefficient vector with values ​​in the range [0,1], and can modulate the contribution of the transformation activation vector element by element.

[0090] Furthermore, gating transformation can be performed by combining linear layer processing with the Sigmoid function to map the gated subvectors to the range [0,1] to obtain dynamic weights, i.e., gated weight vectors. For example, in the table data of an e-commerce recommendation system, the gated subvectors can originate from continuous features such as user browsing time, and the gated weight vectors can be obtained by transforming them with the Sigmoid function.

[0091] Element-wise multiplication can be performed on two vectors of the same dimension (transformation activation vector and gate weight vector) by multiplying them according to their corresponding elements. This process can be achieved by element-wise multiplication between vectors, thereby realizing adaptive modulation of the gate weights. The gate fusion vector can be the result vector obtained by element-wise multiplication. This gate fusion vector can be the modulated feature representation, which can be used to fuse the output of nonlinear transformation and gate mechanism to form a more robust feature representation.

[0092] Furthermore, element-wise multiplication can be performed using dot product to ensure that each feature element can be recalibrated according to its importance. For example, in predictive maintenance table data for industrial equipment, the transformation activation vector can characterize the features of sensor readings after nonlinear activation, and the gating weight vector can be generated according to the equipment status. After multiplication, a gating fusion vector is obtained, which highlights fault-related signals.

[0093] Linear recovery processing can be a process of restoring the dimension of a vector to match the original input through a linear layer. It can also compress the dimension of the gated fusion vector to be the same as the normalized continuous features through a fully connected layer, thus maintaining the coherence of the network structure. For example, in climate change prediction table data, the gated fusion vector integrates the modulation information of features such as temperature and humidity. The gated enhanced continuous features can be obtained through linear recovery processing and can be used for subsequent residual connections and deep network stacking.

[0094] According to the technical solution provided in this disclosure, features are separated into gating and activation vectors through segmentation processing, realizing specialized division of labor in the information path and enhancing the ability to target different patterns in continuous features. By combining nonlinear activation and gating transformation, weights are dynamically generated to modulate the feature flow, improving the efficiency of capturing deep nonlinear relationships and avoiding the limitations of fixed processing. Through element-wise multiplication and linear recovery, the dimensionality consistency and residual compatibility after feature enhancement are ensured, thereby improving the expressive power and training stability of the continuous feature processing flow, and enhancing the collaborative optimization of the dual-stream architecture and the prediction accuracy of future business needs.

[0095] In some embodiments, the continuous features are standardized to obtain normalized continuous features, including: centering the continuous features by mean to obtain center-shifted features; and scaling the center-shifted features by standard deviation to obtain normalized continuous features.

[0096] Specifically, mean centering can be achieved by subtracting the sample mean of a feature, shifting the center of the data distribution to the zero point, thereby eliminating the overall bias of continuous features and making the processed data symmetrically distributed around the zero value. Mean centering can also convert continuous features into center-shifted features, which can be obtained by mean subtraction, with the data distribution center located at the origin.

[0097] Standard deviation scaling can be performed by dividing by the sample standard deviation of the feature to scale the dispersion of the data to unit variance. This can unify the numerical scale of the center-shifted feature, making all features comparable in importance. The center-shifted feature can be obtained by receiving the center-shifted feature from the mean-centering process. The standard deviation parameter required for this standard deviation scaling can be calculated and fixed based on the training dataset.

[0098] For example, in an e-commerce user behavior prediction system, business table data can include features such as user ID, browsing duration, purchase amount, and number of clicks. Browsing duration and purchase amount can be continuous features. Continuous features can be extracted; for example, the original value of browsing duration can be between [0, 3600] seconds, and the purchase amount can be between [0, 10000] yuan, with differences in units. The mean of browsing duration (e.g., 1200 seconds) and the mean of purchase amount (e.g., 500 yuan) in the training set can be calculated using mean centering. This allows for the calculation of the browsing duration for each sample. Subtracting 1200 from the length and 500 from the purchase amount yields the center-shifted feature. At this point, the data distribution center shifts to zero, but the standard deviation of the browsing time is 600 seconds while the standard deviation of the purchase amount is 2000 yuan. Furthermore, standard deviation scaling can be used to calculate the standard deviation using the training set. Dividing the center-shifted browsing time by 600 and the purchase amount by 2000 yields the normalized continuous feature. At this point, all continuous features become distributions with a mean of 0 and a standard deviation of 1. For example, the browsing time value can fall within the range of [-2, 4] while the purchase amount falls within the range of [-0.5, 4.5].

[0099] According to the technical solution provided in this disclosure, by performing mean centering and standard deviation scaling on continuous features, the data distribution is normalized and scaled. The normalization process is decomposed into two explicit sub-steps: centering and scaling, and deeply integrated into the continuous feature processing stream. Mean centering eliminates feature bias and improves the stability and convergence speed of gradient descent. Standard deviation scaling balances the contribution of different features and enhances the adaptability to diverse continuous data. As a prerequisite for deep mining of continuous features, normalization provides a consistent input, improving the training efficiency and generalization performance of the deep residual network for business table data, and enhancing the prediction accuracy and robustness of the dual-stream heterogeneous architecture.

[0100] In some embodiments, performing fully connected mapping on the fused features to obtain the business future demand prediction result corresponding to the business table data includes: performing linear mapping on the fused features to obtain an initial mapping vector; performing nonlinear activation on the initial mapping vector to obtain an activated feature vector; performing linear mapping on the activated feature vector to obtain a mapped feature vector; and performing target dimension transformation on the mapped feature vector to obtain the business future demand prediction result.

[0101] Specifically, linear mapping can be a process of linearly transforming fused features or activation feature vectors to a new vector space using learnable weight matrices and bias vectors. This linear mapping can extract linear combination relationships from the fused features and adjust the feature dimensions to obtain an initial mapping vector or mapped feature vector. The initial mapping vector can be an intermediate result obtained from the linear mapping process, serving as the feature representation obtained through the first linear transformation. Furthermore, in the linear mapping process, the dimension of the weight matrix of the fully connected layer can be adaptively set according to the dimension of the fused features or activation feature vectors and the expected output size, ensuring that key details of the feature information are not lost during the transformation process.

[0102] The mapped feature vector can be the result of linear mapping, which can be obtained by linear mapping of the activation feature vector.

[0103] Target dimension transformation processing can be a process of adjusting the vector dimension. For example, it can be used to convert the mapped feature vector into the output format required by the prediction task through linear mapping or reshaping. High-dimensional feature vectors can be mapped to low-dimensional output space according to business needs to generate the final interpretable prediction result, that is, the prediction result of future business needs, such as scalar values ​​(which can be used for regression tasks) or probability distributions (which can be used for classification tasks).

[0104] According to the technical solution provided in this disclosure, through fully connected mapping processing, deep transformation and dimensional adaptation of fused features are achieved, improving the accuracy and robustness of future business demand prediction results. Through the alternating application of linear mapping and nonlinear activation, more complex linear and nonlinear patterns can be extracted from the fused features, enhancing the adaptability of prediction results to complex business scenarios. Target dimension transformation processing ensures that the output is directly aligned with business requirements, avoiding information loss or distortion, improving the interpretability and practicality of prediction results. Multi-level feature transformation reduces the risk of overfitting and improves the generalization ability to unseen data.

[0105] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0106] Figure 3 This is a schematic diagram of another business table data processing method provided in an embodiment of this disclosure. For example... Figure 3 As shown, the data processing method for this business table includes: After receiving the input table data (business table data), the data splitting module can divide it into categorical features and continuous features, and send them into two parallel processing streams respectively.

[0107] Category Feature Processing Flow: The core of this processing flow is a TabTransformer deep network. After converting discrete values ​​into information-rich embedding vectors through category values ​​and shared column embedding layers, the vectors are fed into the TabTransformer network, which consists of multiple layers of self-attention modules. This network deeply captures the high-order interactions between features and outputs a high-dimensional category feature context representation (cross-encoded category features).

[0108] Continuous Feature Processing Flow: The core of this processing flow is a deep residual network (ResNet-Tabular) for adaptively transforming business tabular data. After being processed by the normalization layer, continuous features are fed into this network, which outputs a high-dimensional deep representation of continuous features (continuous deep features).

[0109] The outputs of the two streams are concatenated by a direct concatenation and fusion module to form a fused feature vector (fused feature). Finally, this fused feature vector is fed into a downstream prediction module (a multilayer perceptron) to obtain the final prediction result (the prediction result of future business demand corresponding to the business table data).

[0110] Specifically, this can be achieved through the following process: Data preparation and splitting: For each input training sample, its features are decomposed into categorical feature tensors and continuous feature tensors.

[0111] Categorical feature processing (using a transformer deep network): This step is specifically for processing categorical features. Its core is the TabTransformer, which can be used to transform a set of independent categorical features into interrelated contextual representations.

[0112] Embedding lookup and concatenation: For each categorical feature in the categorical feature tensor (e.g., the three features "city", "gender", and "education level" in a sample), perform the following operations: Value embedding: Based on the specific value of the feature (such as "City A"), find the corresponding value embedding vector from a learnable embedding matrix.

[0113] Column embedding: Assign a learnable column embedding vector to the column to which the feature belongs (such as the "city" column). The purpose of this column embedding vector is to give all values ​​belonging to the "city" dimension (city A, city B, etc.) a common imprint, enabling the model to distinguish different feature columns.

[0114] Concatenation: The value embedding vector and the column embedding vector are concatenated to form a more complete feature embedding vector. At this point, all the category features of a sample are transformed into a sequence of multiple feature embedding vectors.

[0115] Transformer encoding: The above feature embedding vector sequence is input into the transformer deep network.

[0116] Core principle: This network consists of multiple layers of self-attention modules and feedforward network modules stacked alternately. In each self-attention module, the association strength between each feature embedding in the sequence (e.g., the embedding representing "city") and all other feature embeddings in the sequence (e.g., the embeddings representing "gender" and "education") can be calculated. This strength is called the attention score (initial attention score).

[0117] Information Interaction and Context Construction: Based on these attention scores, a weighted sum of all feature embeddings can be performed, and the result (attention weight value) is added to the current feature embedding. This process can be understood as each feature representation absorbing the "contextual" information of the other features most relevant to it. For example, the representation of "city" as "city A" may, based on the learned attention weights, incorporate more information about "education level" as "master's degree," thus forming an implicit combined concept such as "highly educated people in city A."

[0118] Deep learning: By stacking multiple layers of such self-attention modules, it is possible to learn complex and high-order feature combinations (e.g., the behavioral patterns of male and master's degree users in "City A" during "holidays").

[0119] Output: After processing by the entire transformer network, each initial feature embedding is transformed into a vector rich in contextual information. These vectors together constitute the final class feature context representation (context-enhanced sequence).

[0120] Continuous feature processing (using a deep residual network for tabular data): This step is specifically designed to process continuous features, and its core is the use of a ResNet-Tabular network designed specifically for one-dimensional tabular data.

[0121] Standardization: Z-score standardization is performed on continuous feature tensors.

[0122] Residual network encoding: Standardized continuous feature tensors are input into a deep residual network of tabular data. This network is composed of multiple innovative stacked residual blocks.

[0123] Feature fusion and prediction Direct concatenation: The categorical feature context representation and the continuous feature depth representation are concatenated along the feature dimension to form a fused feature vector.

[0124] Downstream prediction: Input the fused feature vector into the MLP prediction head to obtain the final prediction result.

[0125] Collaborative optimization training: This is a key step that deeply binds training strategies with model architecture.

[0126] Loss calculation: Calculate the loss between the predicted result and the true label based on the task type.

[0127] Define the optimizer and periodic learning rate scheduler: Select an optimizer to manage all parameters of the entire two-stream model. Instantiate a periodic learning rate scheduler and associate it with the optimizer, setting the lower and upper limits of the learning rate and the half-cycle step size.

[0128] Perform the collaborative optimization steps: In each training iteration, backpropagation and weight updates are first performed based on the calculated loss.

[0129] Key step: After each weight update, immediately call the learning rate scheduler to update the learning rate.

[0130] Core Principle and Technical Effect: This operation ensures that the learning rate is no longer monotonically decreasing, but rather oscillates periodically and smoothly between a preset lower and upper limit. This oscillation is key to resolving the training contradictions of heterogeneous networks in this invention. Specifically: High Learning Rate Exploration Phase: As the learning rate oscillates towards its upper limit, a higher learning rate endows the model with stronger exploration capabilities. This is akin to periodically "heating up" the system during training, helping the complex two-stream model effectively escape sharp local optima or saddle points in the loss function surface. For two subnetworks with different learning paces, this phase forces them to conduct synchronous, large-scale parameter space exploration.

[0131] Low learning rate convergence phase: As the learning rate oscillates towards its lower bound, a lower learning rate allows the model to fine-tune its parameters within a more promising and flatter region, thus achieving stable convergence. This phase allows both subnetworks to stabilize together within the new parameter space region.

[0132] Summary of the collaborative mechanism: This periodic alternation of "exploration-convergence" constitutes the core of the collaborative optimization method disclosed herein. It fundamentally solves the technical contradiction that two deep network modules with vastly different structures and convergence characteristics cannot be effectively trained using a single fixed rhythm. By forcing them to undergo synchronized rhythmic changes, it achieves collaboration at the training level, thereby optimizing the performance of the entire model.

[0133] According to the technical solution provided in this disclosure, a specialized and innovative dual-stream heterogeneous deep network architecture is used to match the most suitable and adaptively modified deep network topology for two types of heterogeneous features. The tabular data deep residual network used in the continuous feature processing stream is an innovative improvement on the traditional residual network by incorporating layer normalization and gated activation functions into the residual structure designed for one-dimensional tabular data. This architecture design, which is deep, specialized and innovatively modified for specific data types, improves the information extraction depth of heterogeneous features in complex business tables, enhances the ability to synchronously model high-order interactions of category features and deep nonlinearities of continuous features, and improves the accuracy and generalization robustness of predicting future business needs.

[0134] The collaborative optimization method that deeply binds architecture and training strategy takes the periodic learning rate scheduling strategy as the standard and collaborative optimization method for training the above-mentioned specific dual-stream heterogeneous architecture, and deeply binds the two together. This solves the inherent technical contradiction of "inconsistent learning rhythm" in the joint training of such heterogeneous networks. By oscillating the periodic learning rate, the two sub-networks are forced to perform synchronous alternation of "exploration-convergence", thus realizing collaboration at the training level.

[0135] Figure 4 This is a schematic flowchart illustrating a method for processing residual blocks provided in an embodiment of this disclosure. Figure 4 As shown, the processing method for this residual block includes: An input vector can be normalized through a single layer to enhance training stability. The normalized vector is fed into a first linear layer, where its output dimension is widened. This widened vector is then split into two equal-length vectors, a and b. Vector b is passed through a standard activation function (such as ReLU), with vector a acting as a gate. Vector a is then element-wise multiplied with the activated vector b, resulting in a gated linear unit. This result is fed into a second linear layer, restoring its dimension to the same as the input vector. Finally, the result of this block is element-wise added to the original input vector through a residual connection, forming the output vector of the residual block.

[0136] According to the technical solution provided in the embodiments of this disclosure, by stacking multiple innovative residual blocks, deep patterns in continuous data can be effectively mined, and finally a continuous feature depth representation can be output.

[0137] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0138] Figure 5 This is a schematic diagram of a business form data processing device provided in an embodiment of this disclosure. Figure 5As shown, the business form data processing device includes: The first processing module 501 is used to perform feature decomposition processing on the business table data to obtain the continuous features and category features corresponding to the business table data. The second processing module 502 is used to perform gated residual processing on continuous features to obtain continuous depth features; The third processing module 503 is used to perform cross-context encoding on the category features to obtain cross-encoded category features; The fourth processing module 504 is used to fuse continuous depth features and cross-coded category features to obtain the fused features corresponding to the business table data. The fifth processing module 505 is used to perform fully connected mapping processing on the fused features to obtain the prediction results of future business needs corresponding to the business table data.

[0139] According to the technical solution provided in this disclosure, by performing feature decomposition on business table data, continuous features and categorical features corresponding to the business table data are obtained; gated residual processing can be performed on the continuous features to obtain continuous deep features; cross-context encoding processing can be performed on the categorical features to obtain cross-coded categorical features; then, the continuous deep features and cross-coded categorical features can be fused to obtain fused features corresponding to the business table data; the fused features can be fully connected to obtain the business future demand prediction results corresponding to the business table data; thereby, the information extraction depth of heterogeneous features in complex business tables is improved, the ability to synchronously model high-order interactions of categorical features and deep nonlinearities of continuous features is enhanced, and the accuracy and generalization robustness of business future demand prediction are improved.

[0140] In some embodiments, the second processing module 502 is specifically used to: standardize the continuous features to obtain normalized continuous features; perform gated linear mapping on the normalized continuous features to obtain gated enhanced continuous features; and perform residual fusion on the gated enhanced continuous features to obtain continuous depth features.

[0141] In some embodiments, the third processing module 503 is specifically used to: perform value embedding lookup processing on the category features to obtain a value embedding vector; perform column embedding assignment processing on the category features to obtain a column embedding vector; perform concatenation and fusion processing on the value embedding vector and the column embedding vector to obtain a feature embedding vector; perform sequence combination processing on the feature embedding vector to obtain a feature embedding sequence; perform self-attention context weighting processing on the feature embedding sequence to obtain a context enhancement sequence; and perform multi-layer feedforward fusion processing on the context enhancement sequence to obtain cross-coded category features.

[0142] In some embodiments, performing self-attention context-weighted processing on the feature embedding sequence to obtain a context-enhanced sequence specifically involves: performing linear mapping processing on the feature embedding sequence to obtain a query vector, a key vector, and a value vector; calculating attention scores on the query vector and key vector to obtain an initial attention score; normalizing the initial attention score to obtain an attention weight value; and performing weighted summation processing on the value vector based on the attention weight value to obtain the context-enhanced sequence.

[0143] In some embodiments, the gated linear mapping process for normalized continuous features to obtain gated enhanced continuous features specifically involves: segmenting the normalized continuous features to obtain gated sub-vectors and activation vectors; performing nonlinear activation processing on the activation vectors to obtain transformed activation vectors; performing gated transformation processing on the gated sub-vectors to obtain gated weight vectors; performing element-wise multiplication of the transformed activation vectors and gated weight vectors to obtain gated fusion vectors; and performing linear recovery processing on the gated fusion vectors to obtain gated enhanced continuous features.

[0144] In some embodiments, standardizing continuous features to obtain normalized continuous features specifically involves: centering the continuous features by mean to obtain center-shifted features; and scaling the center-shifted features by standard deviation to obtain normalized continuous features.

[0145] In some embodiments, the fifth processing module 505 is specifically used to: perform linear mapping processing on the fused features to obtain an initial mapping vector; perform nonlinear activation processing on the initial mapping vector to obtain an activated feature vector; perform linear mapping processing on the activated feature vector to obtain a mapped feature vector; and perform target dimension transformation processing on the mapped feature vector to obtain a prediction result of future business needs.

[0146] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.

[0147] Figure 6 This is a schematic diagram of the electronic device 6 provided in an embodiment of this disclosure. Figure 6 As shown, the electronic device 6 of this embodiment includes a processor 601, a memory 602, and a computer program 603 stored in the memory 602 and executable on the processor 601. When the processor 601 executes the computer program 603, it implements the steps in the various method embodiments described above. Alternatively, when the processor 601 executes the computer program 603, it implements the functions of each module / unit in the various device embodiments described above.

[0148] Electronic device 6 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 6 may include, but is not limited to, processor 601 and memory 602. Those skilled in the art will understand that... Figure 6 This is merely an example of electronic device 6 and does not constitute a limitation on electronic device 6. It may include more or fewer components than shown, or different components.

[0149] The processor 601 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0150] The memory 602 can be an internal storage unit of the electronic device 6, such as a hard disk or RAM of the electronic device 6. The memory 602 can also be an external storage device of the electronic device 6, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the electronic device 6. The memory 602 can also include both internal and external storage units of the electronic device 6. The memory 602 is used to store computer programs and other programs and data required by the electronic device.

[0151] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0152] If integrated modules / units are implemented as software functional units and sold or used as independent products, they can be stored in a readable storage medium (e.g., a computer-readable storage medium). Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable storage medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0153] The above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit it. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be included within the protection scope of this disclosure.

Claims

1. A method for processing business table data, characterized in that, include: The business table data is subjected to feature decomposition to obtain the continuous features and category features corresponding to the business table data. Gated residual processing is performed on the continuous features to obtain continuous depth features; The category features are subjected to cross-context encoding to obtain cross-encoded category features; The continuous depth features and the cross-coded category features are fused together to obtain the fused features corresponding to the business table data. The fusion features are processed by a fully connected mapping to obtain the business future demand prediction results corresponding to the business table data.

2. The business form data processing method according to claim 1, characterized in that, The process of performing gated residual processing on the continuous features to obtain continuous depth features includes: The continuous features are standardized to obtain normalized continuous features; The normalized continuous features are subjected to gated linear mapping to obtain gated enhanced continuous features; The gated enhanced continuous features are subjected to residual fusion processing to obtain the continuous depth features.

3. The business form data processing method according to claim 1, characterized in that, The cross-context encoding process performed on the category features to obtain cross-coded category features includes: The category features are subjected to value embedding lookup processing to obtain value embedding vectors; The category features are subjected to column embedding assignment processing to obtain column embedding vectors; The value embedding vector and the column embedding vector are concatenated and fused to obtain the feature embedding vector; The feature embedding vectors are subjected to sequence combination processing to obtain the feature embedding sequence; The feature embedding sequence is subjected to self-attention context weighting to obtain a context-enhanced sequence; The context-enhanced sequence is subjected to multi-layer feedforward fusion processing to obtain the cross-coded category features.

4. The business form data processing method according to claim 3, characterized in that, The step of performing self-attention context-weighted processing on the feature embedding sequence to obtain a context-enhanced sequence includes: The feature embedding sequence is linearly mapped to obtain the query vector, key vector, and value vector. The query vector and key vector are processed to calculate attention scores to obtain initial attention scores; The initial attention score is normalized to obtain the attention weight value; The context enhancement sequence is obtained by performing a weighted summation on the value vector based on the attention weight values.

5. The business form data processing method according to claim 2, characterized in that, The process of performing gated linear mapping on the normalized continuous features to obtain gated enhanced continuous features includes: The normalized continuous features are segmented to obtain gated sub-vectors and activation sub-vectors; The activation vector is subjected to nonlinear activation processing to obtain the transformed activation vector; The gated subvectors are subjected to a gating transformation to obtain a gating weight vector; The transformed activation vector and the gate weight vector are multiplied element-wise to obtain the gated fusion vector. The gated fusion vector is linearly recovered to obtain the gated enhanced continuous feature.

6. The business form data processing method according to claim 2, characterized in that, The standardization process for the continuous features to obtain normalized continuous features includes: The continuous features are subjected to mean centering to obtain center-shifted features; The center translation feature is scaled according to standard deviation to obtain the normalized continuous feature.

7. The business form data processing method according to claim 1, characterized in that, The step of performing a fully connected mapping process on the fused features to obtain the business future demand prediction result corresponding to the business table data includes: The fused features are linearly mapped to obtain an initial mapping vector; The initial mapping vector is subjected to nonlinear activation processing to obtain the activated feature vector; The activated feature vector is linearly mapped to obtain the mapped feature vector. The target dimension is transformed by the mapping feature vector to obtain the prediction result of the future business demand.

8. A business form data processing device, characterized in that, include: The first processing module is used to perform feature decomposition processing on the business table data to obtain the continuous features and category features corresponding to the business table data. The second processing module is used to perform gated residual processing on the continuous features to obtain continuous depth features; The third processing module is used to perform cross-context encoding on the category features to obtain cross-encoded category features; The fourth processing module is used to fuse the continuous depth features and the cross-coding category features to obtain the fused features corresponding to the business table data; The fifth processing module is used to perform fully connected mapping processing on the fused features to obtain the business future demand prediction results corresponding to the business table data.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

10. A readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images