Data classification method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a data classification method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining table data, extracting all text data in the table data, splicing the text data to obtain table text data, carrying out the compression processing of the table text data, obtaining compressed text data, and storing the compressed text data. Inputting the text compressed data into the text large model to obtain a text feature vector, inputting the table data into the deep learning table model to obtain a table feature vector, fusing the text feature vector and the table feature vector to obtain a fusion feature vector, and inputting the fusion feature vector into a classification network to obtain a classification result. And obtaining a classification result corresponding to the table data.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a data classification method, apparatus, electronic device and storage medium. Background Technology

[0002] In data classification scenarios, tabular data, as the core carrier of structured information, is widely used in key areas such as financial risk control, user profiling, and supply chain management. The accuracy of its classification results directly determines the reliability of transaction decisions. Therefore, the efficient processing and accurate classification of tabular data has become an important research topic for the industry.

[0003] Most current mainstream table classification schemes rely on traditional deep learning table models (such as Wide&Deep, TabNet, and DeepFM) to build their classification systems. However, table data in real-world transaction scenarios is often not an ideally regular structure, but rather contains a large amount of irregular text information. This has become the core technical bottleneck of traditional table classification schemes, resulting in insufficient accuracy in existing table classification schemes. Summary of the Invention

[0004] The main purpose of this specification is to provide a data classification method, apparatus, electronic device, and storage medium, aiming to efficiently and accurately classify tabular data. The technical solution is as follows: Firstly, embodiments of this specification provide a data classification method, including: Retrieve table data; Extract all text data from the table data and concatenate the text data to obtain the table text data; The table text data is compressed to obtain compressed text data. The compressed text data is input into the large text model to obtain the text feature vector; The table data is input into a deep learning table model to obtain table feature vectors; The text feature vector and the table feature vector are fused to obtain a fused feature vector; The fused feature vector is input into the classification network to obtain the classification result corresponding to the tabular data. Secondly, embodiments of this specification provide a method for training a data classification model, including: Obtain a first set of table sample data; the first set of table sample data includes first sample table data and first sample classification labels corresponding to the first sample table data. Extract all first sample text data from the first sample table data, and concatenate the first sample text data to obtain the first sample table text data; The first sample table text data is compressed to obtain compressed first sample text data. The compressed data of the first sample text is input into the large text model in the initial data classification model to obtain the feature vector of the sample text; The first sample table data is input into the deep learning table model in the initial data classification model to obtain the sample table feature vector; The sample text feature vector and the sample table feature vector are fused to obtain the sample fused feature vector; The sample fusion feature vector is input into the classification network of the initial data classification model to obtain the sample classification result corresponding to the sample table data; Based on the sample classification results and the sample classification labels, a classification loss value is determined. Based on the classification loss value, the initial model parameters of the initial data classification model are updated to obtain the adjusted target model parameters. Based on the target model parameters, a trained data classification model is generated.

[0005] Thirdly, embodiments of this specification provide a data classification device, including: The retrieval unit is used to retrieve table data; The text extraction unit is used to extract all text data from the table data and concatenate the text data to obtain the table text data. A text compression unit is used to compress the table text data to obtain compressed text data. The text feature extraction unit is used to input the compressed text data into the large text model to obtain text feature vectors; The table feature extraction unit is used to input the table data into a deep learning table model to obtain table feature vectors; A feature fusion unit is used to fuse the text feature vector and the table feature vector to obtain a fused feature vector; The classification prediction unit is used to input the fused feature vector into the classification network to obtain the classification result corresponding to the table data.

[0006] Fourthly, embodiments of this specification provide a data classification device, comprising: A sample acquisition unit is used to acquire a first table sample data set; the first table sample data set includes first sample table data and first sample classification labels corresponding to the first sample table data. The sample text extraction unit is used to extract all the first sample text data in the first sample table data and concatenate the first sample text data to obtain the first sample table text data. The sample compression unit is used to compress the first sample table text data to obtain compressed first sample text data. The sample text feature extraction unit is used to input the first sample text compressed data into the large text model in the initial data classification model to obtain the sample text feature vector; The sample table feature extraction unit is used to input the first sample table data into the deep learning table model in the initial data classification model to obtain the sample table feature vector. A sample fusion unit is used to fuse the sample text feature vector and the sample table feature vector to obtain a sample fusion feature vector; A classification unit is used to input the sample fusion feature vector into the classification network of the initial data classification model to obtain the sample classification result corresponding to the sample table data; The training unit is used to determine the classification loss value based on the sample classification result and the sample classification label, update the initial model parameters of the initial data classification model based on the classification loss value to obtain the adjusted target model parameters, and generate the trained data classification model based on the target model parameters.

[0007] Fifthly, embodiments of this specification provide an electronic device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method described above.

[0008] Sixthly, embodiments of this specification provide a storage medium storing a computer program that, when executed by a processor, implements the steps of the method described above.

[0009] In a seventh aspect, embodiments of this specification provide a computer program product, comprising: a computer program that, when executed by a processor of an electronic device, enables the processor to at least implement the method described in the first or second aspect.

[0010] In the embodiments of this specification, to address the problem of traditional models' inaccurate processing of irregular tabular data, this method involves acquiring tabular data, extracting all text data from the tabular data, concatenating the text data to obtain tabular text data, compressing the tabular text data to obtain compressed text data, inputting the compressed text data into a large text model to obtain text feature vectors, inputting the tabular data into a deep learning tabular model to obtain tabular feature vectors, fusing the text feature vectors and the tabular feature vectors to obtain a fused feature vector, and inputting the fused feature vector into a classification network to obtain the classification result corresponding to the tabular data. This method textifies the table, leveraging the powerful text understanding capabilities and world knowledge of the large model to enhance the understanding of irregular text. Furthermore, text compression reduces the data volume, accelerating the inference of the large model, while fusing the tabular model feature vectors, inputting the same data into different modalities (tables and text), can gain gains and improve classification accuracy. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram illustrating a data classification method provided in the embodiments of this specification; Figure 2 This is a schematic diagram illustrating an example of a data classification method provided in the embodiments of this specification; Figure 3 This is a flowchart illustrating a data classification method provided in the embodiments of this specification; Figure 4 This is a flowchart illustrating a data classification method provided in the embodiments of this specification; Figure 5 This is a flowchart illustrating a data classification method provided in the embodiments of this specification; Figure 6 This is a flowchart illustrating a data classification method provided in the embodiments of this specification; Figure 7 This is a flowchart illustrating a training method for a data classification model provided in an embodiment of this specification. Figure 8 This is a flowchart illustrating a training method for a data classification model provided in an embodiment of this specification. Figure 9 This is a flowchart illustrating the training process of a data classification model provided in the embodiments of this specification. Figure 10 This is a schematic diagram of the structure of a data classification device provided in the embodiments of this specification; Figure 11 This is a schematic diagram of the structure of a data classification device provided in the embodiments of this specification; Figure 12 This is a schematic diagram of the structure of an electronic device provided in the embodiments of this specification. Detailed Implementation

[0013] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0014] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0015] Most related technologies rely on traditional deep learning tabular models (such as Wide&Deep, TabNet, DeepFM, etc.) to construct classification systems. The core logic of this approach is to model the numerical features and discrete category features in the table using a non-linear network structure, and to uncover higher-order interaction relationships between features, demonstrating a certain degree of effectiveness in processing standardized structured tabular data.

[0016] However, in real-world transaction scenarios, tabular data is often not an ideal, well-structured system. Instead, it is typically mixed with a large amount of irregular text information. This has become the core technical bottleneck of traditional table classification schemes. On the one hand, traditional table models lack the ability to deeply understand the semantics of text. For unstructured text fields in tables, such as "product description," "user notes," and "explanation of reasons for anomalies," they can usually only be converted into low-dimensional vectors through simple encoding (such as one-hot encoding and bag-of-words model). They cannot capture the key information such as the contextual logic and transaction semantic relationships contained in the text, resulting in the value of text features not being fully explored. On the other hand, for the key-value pair semantics of feature names and feature values ​​in tables (such as the semantics of "<sales amount>:10000" which carries the meaning of "revenue scale"), traditional models cannot establish effective association mappings. Especially when the table has irregular situations such as dynamic changes in feature names and mixed storage of numerical values ​​and text, the model is very prone to feature misjudgment, which leads to classification bias.

[0017] To address the aforementioned problems, this specification provides a data classification method through its embodiments. Please refer to [link to documentation]. Figure 1 This diagram illustrates a scenario of a data classification method provided in this specification. The data classification method can be implemented using a data classification device. The data classification device is used to acquire tabular data and generate corresponding table classification results. It is understood that the data classification device provided in this specification can be a terminal device such as a mobile phone, computer, tablet computer, smartwatch, or in-vehicle device, or it can be a module within a terminal device used to implement the data classification method.

[0018] Please see also Figure 1 , Figure 1 This specification provides an example illustration of a data classification method. It involves acquiring tabular data, extracting all text data from the table, concatenating the text data to obtain tabular text data, compressing the tabular text data to obtain compressed text data, thus reducing the processing load of the large text model. The compressed text data is then input into the large text model for feature extraction to obtain text feature vectors. Simultaneously, the tabular data is input into a deep learning tabular model for feature extraction to obtain table feature vectors. The text feature vectors from the text modality and the table feature vectors from the table modality are fused to obtain a fused feature vector. This fused feature vector is then input into a classification network to obtain the classification result corresponding to the tabular data. By adopting this method, irregular text in tables can be effectively processed, and deep fusion of text and table features can be achieved, improving the accuracy of table classification.

[0019] It should be noted that the embodiments of this specification also provide a training method for a data classification model. This involves obtaining a first set of table sample data, which includes first sample table data and corresponding first sample classification labels. All first sample text data are extracted from the first sample table data. The first sample text data is concatenated to obtain first sample table text data. The first sample table text data is compressed to obtain compressed first sample text data. The compressed first sample text data is input into the large text model in the initial data classification model to obtain sample text feature vectors. The first sample table data is input into the deep learning table model in the initial data classification model to obtain sample table feature vectors. The sample text feature vectors and sample table feature vectors are fused to obtain a sample fused feature vector. The sample fused feature vector is input into the classification network in the initial data classification model to obtain the sample classification result corresponding to the sample table data. A classification loss value is determined based on the sample classification result and sample classification labels. The initial model parameters of the initial data classification model are updated based on the classification loss value to obtain adjusted target model parameters. A trained data classification model is generated based on the target model parameters. The trained data classification model is used to implement the data classification method in the embodiments of the above specification.

[0020] The data classification method provided in this specification will be described in detail below with reference to specific embodiments.

[0021] Please see Figure 3 This is a flowchart illustrating a data classification method provided in an embodiment of this specification. Figure 3 As shown, the method in the embodiments of this specification may include the following steps S102-S114.

[0022] S102, retrieve table data; In one embodiment of this specification, the original tabular data to be classified is obtained. Tabular data refers to a dataset with a two-dimensional structured organization of rows and columns. Typically, "rows" are data sample units and "columns" are feature attribute units. Each column corresponds to a feature name (e.g., "monthly salary" or "education level"). The content of each column in each row is the feature value of that sample (e.g., "8000" or "Bachelor's Degree"). The feature value type can cover various types such as numerical, categorical, and textual.

[0023] The tabular data can be any type of tabular data. For example, the obtained tabular data can be user credit application form data in a financial risk control scenario, containing numerical features (such as "monthly salary" and "debt amount"), categorical features (such as "education level" and "occupation"), and irregular text features (such as "loan purpose statement" and "remarks on overdue reasons"). Another example is that the obtained tabular data can be from an e-commerce scenario, containing numerical features (such as "price" and "monthly sales"), categorical features (such as "category" and "material"), and irregular text features (such as "product description: slim-fit and versatile jeans, high-waisted design makes legs look longer, suitable for spring and autumn" and "user review summary: good fit, comfortable fabric, standard size").

[0024] It should be noted that different tabular data may have different classification requirements. Categories can be divided according to actual needs, and the category to which the tabular data belongs can be further determined through a model.

[0025] S104, extract all text data from the table data, and concatenate the text data to obtain table text data; In one embodiment of this specification, to address the bottleneck of traditional table models in handling irregular text, text information from the table is extracted and integrated, laying the foundation for subsequent semantic processing using a large text model. Specifically, the acquired table data is traversed, all text-type data is extracted, and concatenated according to a preliminary format of "feature name:feature value", such as "Name: Zhang San, Education: Bachelor's Degree, Occupation: Engineer, Loan Purpose: Buying a House", forming a single sample of table text data.

[0026] S106, Compress the table text data to obtain compressed text data; In one embodiment of this specification, in order to improve the efficiency of model training and inference, the number of tokens in the text data is reduced by compression, which solves the problem of token redundancy and low efficiency when directly using a large text model to process table text, while retaining the core semantic information.

[0027] In one feasible implementation, for feature names, multi-token feature names (such as "Loan Purpose Description" containing 6 tokens) are converted into compressed tokens of a single token, i.e., "Loan Purpose Description" is treated as a complete token. This avoids the semantic fragmentation of feature names caused by traditional word segmentation and reduces the amount of token processing, thus achieving data compression. For feature values, numerical and non-numerical types are distinguished. Numerical feature values ​​are compressed into single tokens by replacing preset numerical tokens, while non-numerical feature values ​​(especially in short texts) are directly retained to avoid semantic loss. Optionally, for numerical features, methods such as binning interpolation (PLE), Fourier transform (FoNE), and periodic frequency can also be used to compress numerical feature values ​​in tabular text data.

[0028] Optionally, the compression process is implemented in the form of "feature name - feature value" to ensure that the correspondence between "feature name - feature value" is not destroyed, thus providing a foundation for the subsequent accurate identification of semantic associations by large text models.

[0029] For example, the string "Name: Zhang San, Education: Bachelor's Degree, Occupation: Engineer, Loan Purpose: Purchase of Wedding House, Monthly Salary: 8000" is compressed, converting feature names such as "Name" and "Education" into single tokens such as "<Name>" and "<Education>", and converting numerical values ​​such as "8000" into single tokens. <num>The obtained compressed text data is: "<Name>: Zhang San <Education>: Bachelor's Degree <Occupation>: Engineer <Loan Purpose>: Purchase of a Wedding Home <Monthly Salary>: " <num>".

[0030] S108, The compressed text data is input into the large text model to obtain the text feature vector; In one embodiment of this specification, the large-scale text model refers to a Transformer architecture model (such as BERT, LLaMA, RoBERTa, etc.) that has been pre-trained on large-scale general text and possesses powerful semantic understanding, contextual association capture, and world knowledge integration capabilities. Its core advantage lies in its ability to convert natural language text into high-dimensional vectors containing semantic information. By mining the semantic value of text through the large-scale text model and utilizing the world knowledge of the large-scale model to understand irregular text, text feature vectors containing semantic information are generated, thus compensating for the shortcomings of traditional table models in semantic understanding.

[0031] For example, BERT-large is selected as the large text model (1024 hidden layer dimensions, 12 Transformer encoder layers). The input is the compressed text data from an e-commerce scenario: "<Product Name>: Slim Fit Versatile Jeans <Category>: Apparel <Material>: Cotton <Price>: <num><Product Description>: Slim-fit, versatile jeans; high-waisted design makes legs look longer; suitable for spring and autumn. During preprocessing, a start token "[CLS]" and an end token "[SEP]" are added. For compressed words such as "<Product Name>", the average word vectors of the original word segments "product" and "name" are initialized. During encoding, the model uses a 12-layer attention mechanism to capture the semantic association between "<Product Name>" and "slim-fit, versatile jeans", as well as the implicit association between "<Material>: cotton" and "suitable for spring and autumn" in the "Product Description". For "<Price>: <num>The value is restored to "99.9", normalized, and then input into a linear layer (input dimension 1, output dimension 1024) to obtain a numerical feature vector. Finally, the encoded sequence vector is averaged and pooled, and then concatenated with the numerical feature vector and projected to 1024 dimensions to obtain the text feature vector.

[0032] S110, The table data is input into a deep learning table model to obtain the table feature vector; In one embodiment of this specification, the deep learning table model is a model specifically designed for the structured features of tabular data. It can uncover high-order relationships between features through nonlinear transformations and feature interaction modeling. Exemplary deep learning table models include TabNet, DeepFM, and Wide&Deep. By directly inputting tabular data into the deep learning table model, the output table feature vector can be obtained. It retains the advantages of traditional deep learning table models in handling structured features, complementing text feature vectors. For example, features such as "Monthly Salary: 8000", "Debt Amount: 2000", "Education: Bachelor's Degree", and "Occupation: Engineer" from the original tabular data are preprocessed (numerical normalization, one-hot encoding of categorical features) and then input into the TabNet deep learning table model. The model uses an attention mechanism to filter key features and model interaction relationships, outputting a table feature vector with a dimension of 512.

[0033] S112, the text feature vector and the table feature vector are fused to obtain a fused feature vector; In one embodiment of this specification, deep fusion of dual-modal features is achieved, combining the semantic understanding capability of the large text model with the structured feature processing capability of the traditional table model, solving the problem of incomplete single-modal feature information and improving classification accuracy.

[0034] In one feasible implementation, the dimensions of the text and table feature vectors are first unified, typically using high-dimensional features as a benchmark. Low-dimensional features are then augmented to the corresponding dimensions through linear projection. Finally, the text and table feature vectors are fused using element-weighted or dimension-weighted methods. Weights are assigned based on the importance of the bimodal features, and dynamic feature integration is achieved through weight adjustment, avoiding interference from irrelevant information.

[0035] In another feasible implementation, an attention mechanism is used to adaptively capture key information related to the classification task from the bimodal features, achieving on-demand fusion. First, a fusion matrix is ​​constructed by concatenating the text feature vector and the table feature vector column by column. Then, the self-attention weights of the fusion matrix are calculated through an attention layer to locate the feature positions that contribute highly to the classification result. Next, the attention weights are multiplied by the fusion matrix to strengthen key features and suppress redundant features. Finally, the fusion feature vector is obtained by compressing the dimensionality through a linear layer.

[0036] S114, The fused feature vector is input into the classification network to obtain the classification result corresponding to the table data; In one embodiment of this specification, a feature vector incorporating bimodal information is input into a classification network, which outputs a classification result. The core function of the classification network is to map the high-dimensional fused feature vector to a specific classification result. Essentially, it uses nonlinear transformations and feature dimensionality reduction to extract discriminative information directly related to the classification task from the fused features, ultimately achieving a precise "feature-category" mapping. The classification network typically includes a feature dimensionality reduction layer, an activation function layer, and an output layer.

[0037] For example, a classification network can adopt a "linear layer + Softmax" structure. The fused feature vector is input into the first linear layer, activated by ReLU, and then input into the second linear layer. Finally, it is processed by Softmax to output the class probability, and the class with the highest probability is taken as the classification result.

[0038] In the embodiments of this specification, table data is acquired, all text data is extracted from the table data, the text data is concatenated to obtain table text data, and the table text data is compressed to obtain compressed text data. Compression reduces the processing load of the large text model. Then, the compressed text data is input into the large text model for feature extraction to obtain text feature vectors. Simultaneously, the table data is input into a deep learning table model for feature extraction to obtain table feature vectors. The text feature vectors from the text modality and the table feature vectors from the table modality are fused to obtain a fused feature vector. This fused feature vector is input into a classification network to obtain the classification result corresponding to the table data. By textifying the table, the powerful text understanding capabilities and world knowledge of the large model are utilized to enhance the understanding of irregular text. Furthermore, reducing the data volume through text compression accelerates the inference of the large model, and using the same data for both the table and text modalities can improve classification accuracy.

[0039] Please see Figure 4 This is a flowchart illustrating a data classification method provided in an embodiment of this specification. Figure 4 As shown, the method described in the embodiments of this specification may include the following steps S202-S206.

[0040] S202, filter feature names and corresponding feature values ​​from the table text data; In one embodiment of this specification, the concatenated tabular text data is subjected to structured parsing to extract "feature name-feature value" key-value pairs.

[0041] S204, convert each of the feature names into a word element, which serves as the feature name compressed word element corresponding to each of the feature names; In one embodiment of this specification, single-token compression is performed on feature names to solve the semantic fragmentation problem caused by splitting feature names into multiple tokens in traditional word segmentation, while reducing the number of tokens and improving efficiency. For example, selected feature names such as "name," "education level," "occupation," "loan purpose description," and "monthly salary" are converted into single-token compressed feature name units such as "<name>," "<education level>," "<occupation>," "<loan purpose description>," and "<monthly salary>," respectively, and these new tokens are added to the vocabulary of the large text model. Only by adding these compressed feature name units to the vocabulary can the large model recognize "<name>" as an independent token during the word segmentation stage. This achieves a sharp reduction in input length and semantic aggregation.

[0042] S206, Concatenate the compressed text data according to the format that each feature name corresponds to a feature value.

[0043] In one embodiment of this specification, after compressing the feature name, a compressed feature name token is obtained. The feature value corresponding to the feature name is then concatenated with the compressed feature name token to obtain compressed text data. It is understood that only the feature name can be compressed to reduce the amount of text data to be processed. Optionally, the feature value can be compressed or not compressed and directly concatenated with the compressed feature name token.

[0044] Optionally, in one embodiment, the feature values ​​are concatenated with the feature name compressed tokens, and the method includes: S2062, if the feature value is a numerical type, then the feature value of the numerical type is compressed into a numerical word according to a preset compression method and then concatenated with the corresponding feature name compressed word to generate the first text compressed data; Specifically, if the feature value is a numeric type, the numeric value of multiple tokens (such as "8000" which includes four tokens "8", "0", "0", "0") is compressed into a single token or a fixed number of tokens, which greatly reduces the number of tokens. At the same time, the mathematical semantics of the numeric value are preserved through subsequent steps, thus balancing efficiency and effectiveness.

[0045] For example, numerical compression can be performed using Piecewise Linear Embedding (PLE). The range of numerical features is divided into multiple continuous intervals (bins), and a unique interval term is assigned to each interval. Then, the offset of the value within the interval is calculated by linear interpolation within the interval, and the "interval term + offset encoding" is used as the compression result, which preserves the interval attribution while taking into account the precision of the numerical value.

[0046] Another example is that Fourier transform (FoNE, Fourier Feature Embedding) is used to map numerical values ​​from the original space to the frequency space, extract the periodic or trend features of the numerical values, and encode the transformed low-frequency components (core information) into a fixed-length word sequence, which is suitable for numerical features containing periodicity (such as time, sales cycle, etc.).

[0047] Another example is numerical compression using periodic encoding. For periodic values ​​(such as months and hours in a date), they are directly mapped to preset periodic terms (such as months 1-12). <mon1> - <mon12>Non-periodic values ​​are statistically divided into high-frequency, medium-frequency, and low-frequency words to achieve semantic compression.

[0048] In one embodiment of this specification, the preset compression method may be to replace feature values ​​with pre-created numerical tokens, specifically including the following steps: S20622, If the feature value is a numerical type, then the feature value of the numerical type is replaced with a pre-created numerical term and concatenated with the corresponding feature name compressed term to generate the first text compressed data.

[0049] In one embodiment, numerical lexical units are pre-created and stored in the vocabulary of a large text model. For example, the numerical lexical units can be... <num>Then, when the feature value is identified as a numeric type, the numeric value is replaced with... <num>It is then concatenated with compressed feature words.

[0050] S2064, if the feature value is not a numerical type, then the feature value is directly concatenated with the corresponding feature name compressed word to generate the second text compressed data.

[0051] In one embodiment of this specification, non-numerical feature values ​​(such as text) are directly concatenated to avoid semantic loss due to excessive compression and to fully utilize the natural language understanding capabilities of the large text model. For example, if the feature values ​​in "Name: Zhang San" and "Education: Bachelor's Degree" are determined to be non-numerical, the feature values ​​are directly concatenated with the corresponding feature name compressed tokens to obtain "<Name>: Zhang San <Education>: Bachelor's Degree <Occupation>: Engineer <Loan Purpose>: Purchase of Wedding House", which serves as the second text compressed data. This second compressed data is then merged with the first text compressed data to form the complete text compressed data. For example, if the "8000" in "Monthly Salary: 8000" is determined to be numeric, a compression method of "uniformly replacing with preset numeric tokens" is used to replace "8000" with... <num>The character "" is concatenated with the feature name compressed word "<monthly salary>" to form "<monthly salary>". <num>", combined with the processing results of other numerical features, generate the first text compressed data containing numerical compression.

[0052] In the embodiments of this specification, key-value pairs of feature names and corresponding feature values ​​are selected from the tabular text data. Each feature name is converted into a single token as a compressed feature name token. The compressed text data is generated by concatenating the tokens according to the format of "one compressed feature name token corresponds to one feature value". This compression method effectively avoids the semantic fragmentation problem caused by the splitting of feature names in traditional word segmentation. It ensures the semantic integrity of feature names while reducing the number of tokens. The fixed concatenation format of S206 provides standardized input for subsequent large-scale text model encoding, enabling the model to quickly identify the semantic relationship between "feature name and feature value", reducing the semantic parsing cost of the model, and ultimately achieving a balance between improving compression efficiency and preserving semantic information.

[0053] Please see Figure 5 This is a flowchart illustrating a data classification method provided in an embodiment of this specification. Figure 5 As shown, the method described in the embodiments of this specification may include the following steps S302-S306.

[0054] S302, input each word element included in the feature name in the compressed text data into the encoder of the large text model and encode them into word vectors respectively, and take the average value of the word vectors as the feature name vector of the feature name compressed word element corresponding to the feature name; In one embodiment of this specification, the encoder of the large text model is briefly introduced. Its core is a multi-layer attention mechanism network based on the Transformer architecture, which has the ability to capture the semantic relationship between word context and can convert the input discrete words into continuous vectors (word vectors) containing semantic information. It is the core module for the large text model to achieve semantic understanding. The word processing logic of the feature name in this step is "segmentation-embedding-averaging": First, the original feature name (such as "loan purpose description") needs to be split into multiple basic words (such as "loan", "fund", "use", "purpose", "explanation" or "loan", "purpose", "explanation") according to the basic word segmentation rules of the large text model (such as BERT's WordPiece segmentation). The specific word depends on the word segmentation rule. These basic words are all tokens that already exist in the vocabulary of the large text model. Then, the multiple basic words after splitting are input into the encoder. Through the embedding layer and attention encoding of the encoder, the word vector corresponding to each basic word is obtained. Finally, the arithmetic mean of these word vectors is calculated as the feature name vector of the feature name compression word. It solves the problem of newly created feature name compression units (single tokens) without pre-trained embeddings, and retains the core semantics of the original feature name through semantic fusion of basic units, ensuring that the semantics of the feature name can be accurately recognized by the model.

[0055] S304, input the feature values ​​in the compressed text data into the encoder of the large text model and encode them into a first feature value vector; In one embodiment of this specification, the feature values ​​in the compressed text data are uniformly encoded. For non-numerical feature values ​​(especially irregular text), they are directly input into the encoder to utilize its semantic understanding capabilities and be converted into vectors to capture key information in texts such as "buying a wedding house". For numerical feature values, there are two processing paths: First, if they are not compressed in the pre-compression step, the numerical value can be directly input into the encoder in text form (such as "8000") and converted into a first feature value vector through the word embedding layer of the encoder. Second, if they have been compressed into numerical words, the numerical words in the first compressed text data are restored to numerical feature values ​​according to the preset compression method of numerical words. The numerical feature values ​​are then input into the linear layer of the large text model to extract numerical feature vectors, which are used as the second feature value vectors of the numerical words.

[0056] S306, the feature name vector and the first feature value vector are used as the text feature vector corresponding to the text compressed data; In one embodiment of this specification, the vector representations of feature names and feature values ​​are integrated to form a complete text feature vector.

[0057] Optionally, for cases where the compressed text data includes the first compressed text data, i.e., cases where the feature values ​​include numerical types, there is a corresponding feature extraction method, which includes the following steps S402-S404: S402, according to the preset compression method of the numerical words, the numerical words in the first text compressed data are restored to numerical feature values, and the numerical feature values ​​are input into the linear layer of the text large model to extract numerical feature vectors, which are used as the second feature value vectors of the numerical words. In one embodiment of this specification, since the feature values ​​of numerical types are compressed when generating compressed text data, the numerical tokens need to be restored.

[0058] For example, a mapping relationship between "numerical lexical units and original numerical values" is established, and this mapping relationship is recorded synchronously when compressing feature values, such as by storing it through a key-value dictionary. <num> -8000”" <num>Different numerical terms, such as "-2000", correspond to different original numerical values. Based on the reverse logic of a preset compression method (e.g., uniform replacement, binning interpolation), numerical terms are located from the first text compressed data and matched to be restored to their original numerical type (e.g., "<monthly salary>: <num>"in" <num>(Restored to 8000). A dedicated linear layer is configured in the large text model. The preprocessed numerical values ​​are input into this linear layer, and a linear transformation is performed to obtain the numerical feature vector, i.e., the second feature value vector. This retains the advantages of reduced token quantity and improved inference efficiency brought about by numerical compression, while also accurately capturing the mathematical semantics of the numerical values.

[0059] S404, input each word element included in the feature name in the first text compressed data into the encoder of the text large model and encode them into word vectors respectively, and take the average value of the word vectors as the feature name vector of the feature name compressed word element corresponding to the feature name; For details, please refer to S302, which will not be elaborated here.

[0060] S406, the feature name vector and the second feature value vector are used as the text feature vector corresponding to the first text compressed data.

[0061] In one embodiment of this specification, the second feature value vector obtained by restoring the numerical lexical units and the feature name vector obtained by averaging the word vectors are combined as the text feature vector corresponding to the first text compressed data.

[0062] When the feature values ​​include both numerical and non-numerical types, the vector extraction of the feature values ​​can be achieved by combining two methods: linear layer feature extraction after numerical word reduction and direct encoding by non-numerical type encoder.

[0063] In the embodiments of this specification, an encoding method for feature name compression units and feature values ​​is provided. After the feature name is split into units, the multiple basic units are input into an encoder to obtain the word vector corresponding to each unit. Finally, the arithmetic mean of these word vectors is calculated as the feature name vector of the feature name compression unit. The feature values ​​are directly encoded and extracted to obtain the first feature value vector. Further, for the first text compression data containing numerical units, numerical units are restored. The numerical feature values ​​are input into the linear layer of the large text model to extract the numerical feature vector, which serves as the second feature value vector of the numerical units, thus balancing compression efficiency and preservation of numerical information.

[0064] Please see Figure 6 This is a flowchart illustrating a data classification method provided in an embodiment of this specification. Figure 6 As shown, the method described in the embodiments of this specification may include the following steps S502-S510.

[0065] S502, Perform projection transformation on the text feature vector and the table feature vector respectively to obtain the text projection feature vector corresponding to the text feature vector and the table projection feature vector corresponding to the table feature vector; In one embodiment of this specification, the original dimensions and semantic characteristics of the text feature vector and table feature vector are clearly defined, and two independent linear projection layers (text projection layer and table projection layer) are constructed. The output dimension of the projection layers is uniformly set to a preset value (usually the median of the original dimensions of the two or a dimension adapted to the classification network, such as 512 dimensions). The input dimension of the text projection layer is the text feature vector dimension, and the output dimension is 512 dimensions; the input dimension of the table projection layer is the table feature vector dimension, and the output dimension is 512 dimensions. The text feature vector is input into the text projection layer and linearly transformed to obtain the text projection feature vector; the table feature vector is input into the table projection layer and linearly transformed to obtain the table projection feature vector. Through the design of independent projection layers, the bimodal features that originally had large differences in dimensions and semantic space are transformed into projection features with consistent dimensions and adapted semantic space.

[0066] S504, the text projection feature vector and the table projection feature vector are concatenated to obtain the initial fused feature vector; In one embodiment of this specification, it is confirmed that the dimensions of the text projection feature vector (e.g., 512-dimensional) and the table projection feature vector (e.g., 512-dimensional) are consistent to avoid splicing failure due to projection errors. Then, the 512 dimensions of the text projection feature vector and the 512 dimensions of the table projection feature vector are concatenated sequentially by splicing according to feature dimensions to form an initial fused feature vector with dimensions equal to the sum of the two.

[0067] S506, Perform a projection transformation on the initial fused feature vector to obtain the fused feature vector; In one embodiment of this specification, a projection layer is pre-created. In this projection layer, the initial fused feature vector is compressed in dimension through a linear transformation, while filtering out duplicate or invalid information introduced by splicing, to obtain the fused feature vector. The input dimension of the projection layer is the dimension of the initial fused feature vector (e.g., 1024 dimensions), and the output dimension is the dimension adapted to the classification network (e.g., 768 dimensions).

[0068] In the embodiments of this specification, a method for fusing text feature vectors and table feature vectors is provided. By projecting the text feature vectors and table feature vectors separately, the dimensions of the two are made the same, which facilitates feature concatenation. Then, a second projection process is performed. The output dimension of the second projection can directly match the input dimension of the classification network. At the same time, by standardizing and optimizing the feature distribution, the fused features are made to better fit the decision boundary of the classification task, thereby improving the classification accuracy.

[0069] S508, The fused feature vector is input into the classification network to obtain the class probability output by the classification network; In one embodiment of this specification, the fused feature vector is mapped to a category probability distribution, quantifying the model's confidence in each classification result and providing a quantitative basis for the final classification result. The classification network typically includes a feature reduction layer, a non-linear activation layer, and a probability output layer. The fused feature vector is input into a linear layer, undergoes linear transformation for dimensionality reduction, and then non-linearity is introduced through the ReLU activation function to filter irrelevant features. The activated features are then input into the linear layer to obtain the original scores corresponding to the number of categories. Finally, the Softmax function converts the original scores into category probabilities between 0 and 1 (the sum of all category probabilities is 1).

[0070] By using a multi-layer network structure and non-linear activation, a precise mapping from high-dimensional fusion features to class probabilities is achieved. The output of class probabilities not only provides a clear basis for determining subsequent results, but also reflects the model's confidence in the classification results. At the same time, the use of the Softmax function makes the probability distribution conform to the statistical norms of probability, which facilitates the calculation of loss during model training.

[0071] S510, determine the classification result corresponding to the table data based on the category probability.

[0072] In one embodiment of this specification, a mapping rule between category probabilities and classification results is determined based on the transaction scenario. Commonly used rules include the "maximum probability rule" (the default rule, which takes the category with the highest probability as the result) and the "confidence threshold rule" (for high-accuracy scenarios, the corresponding category is output when the maximum probability is higher than the threshold; otherwise, "awaiting manual review" is output). Then, the classification result corresponding to the category probability is determined according to this mapping rule. The mapping rule makes the classification results deterministic and reproducible, avoiding differences in results caused by subjective judgment. It also supports adjusting the rules according to transaction needs (such as increasing the threshold to improve accuracy in risk control scenarios), enhancing the practicality and flexibility of the solution.

[0073] Taking the threshold rule (threshold 0.9) as an example, the category probability is 0.85 < 0.9, and the output is "Pending manual review". Optionally, the determined classification result and the corresponding category probability can be output together to improve the interpretability of the result.

[0074] Please see Figure 7 This is a flowchart illustrating a training method for a data classification model provided in an embodiment of this specification. Figure 7 As shown, the method described in the embodiments of this specification may include the following steps S602-S614.

[0075] S602, Obtain the first table sample data set; the first table sample data set includes the first sample table data and the first sample classification label corresponding to the first sample table data; In one embodiment of this specification, a training method for a data classification model is also provided. This data classification model can be used to implement a data classification method. The data classification model includes a large text model, a deep learning table model, a fusion network, and a classification network. When training the data classification model, sample data is first acquired. For example, a data source consistent with the target transaction scenario is selected (e.g., credit application data in a financial risk control scenario) to ensure the representativeness of the samples. Table data containing structured features and irregular text features is collected, along with the corresponding real classification labels (e.g., "Approved" "Rejected"). The collected samples are cleaned, deduplicated, and divided, for example, into a training set and a validation set at an 8:2 ratio, forming a first set of table sample data containing sample data and labels.

[0076] S604, extract all the first sample text data in the first sample table data, and concatenate the first sample text data to obtain the first sample table text data; In one embodiment of this specification, this step is a text processing step for the sample data. It involves traversing each column of the first sample table data, identifying and extracting feature data of all text types, and recording the corresponding feature names. The extracted "feature name-text feature value" pairs are then concatenated according to a fixed format of "feature name: feature value" to form a single sample's first sample table text data. This operation is performed on all samples in the first sample table data set to obtain a batch of first sample table text data.

[0077] S606, Compress the first sample table text data to obtain compressed first sample text data. In one embodiment of this specification, inference efficiency is ensured by compressing the first sample table text data. It should be noted that the compression rules are completely consistent with those used in the inference stage. In one feasible implementation, for the first sample table text data of a single sample, "feature name-feature value" pairs are first filtered, then the feature names are converted into single-token compressed tokens, numerical feature values ​​are replaced with numerical tokens, and non-numerical feature values ​​are directly concatenated to obtain the compressed first sample text data for a single sample. Compression is performed on the table text data of all samples to form a batch of compressed first sample text data.

[0078] Consistent compression rules for training and inference ensure that the model can accurately learn the semantic representation of compressed text, avoiding performance gaps caused by using original text during training and compressed text during inference; at the same time, the number of tokens in the compressed sample text is reduced, which reduces the computational resource consumption and time cost during training.

[0079] S608, input the first sample text compressed data into the text large model in the initial data classification model to obtain the sample text feature vector; In one embodiment of this specification, compressed sample text is converted into vectors using an initial large-scale text model, providing feature input for the model to learn the association between text semantics and labels. An untuned initial large-scale text model (such as a pre-trained BERT-base with randomly initialized parameters or using pre-trained parameters) is loaded; the compressed data of the first sample text is input into the initial large-scale text model to generate a single sample text feature vector. The sample text feature vectors extracted by the initial large-scale text model accurately represent the semantic information of the compressed text.

[0080] S610, the first sample table data is input into the deep learning table model in the initial data classification model to obtain the sample table feature vector; In one embodiment of this specification, sample structured data is processed through an initial deep learning table model to provide feature input for the model to learn the association between structured features and labels. Specifically, an initial deep learning table model (such as TabNet or DeepFM) with randomly initialized parameters is loaded, and the first sample table data is input into the initial deep learning table model to generate sample table feature vectors. The deep learning table model can effectively capture high-order interaction relationships between structured features, such as the ratio of "monthly salary - debt amount".

[0081] S612, the sample text feature vector and the sample table feature vector are fused to obtain the sample fused feature vector; In one embodiment of this specification, the sample text feature vector and the sample table feature vector are fused to obtain a sample fused feature vector. The generated sample fused feature vector contains both textual semantic information and table structural information, thus solving the problem of incomplete single-modal feature information.

[0082] In one feasible implementation, the fusion can be achieved through a fusion network. The sample text feature vector and the sample table feature vector can be input into the fusion network first. The fusion network projects the sample text feature vector and the sample table feature vector to a unified dimension, and then concatenates them into an initial fusion feature. After nonlinear projection, the final sample fusion feature vector is obtained.

[0083] S614, The sample fusion feature vector is input into the classification network of the initial data classification model to obtain the sample classification result corresponding to the sample table data; In one embodiment of this specification, an initial classification network is used to classify the sample fusion features and output the model prediction results, providing a basis for subsequent loss calculation. For example, an initial classification network with randomly initialized parameters is loaded, the sample fusion feature vector is input into the initial classification network, and after multiple transformations, the sample classification results (e.g., class probability distribution) are output. It should be noted that the same network structure is used in both the training and inference phases to ensure consistent performance between training and inference.

[0084] S616, Based on the sample classification result and the sample classification label, determine the classification loss value, update the initial model parameters of the initial data classification model based on the classification loss value to obtain the adjusted target model parameters, and generate the trained data classification model based on the target model parameters.

[0085] In one embodiment of this specification, the difference between the predicted and the true labels is quantified by the loss value, and then the parameters are updated by backpropagation, ultimately generating a model with classification capabilities. For example, a loss function is selected based on the task type; for example, categorical cross-entropy loss is chosen for multi-class classification. The batch of sample classification results and the corresponding first sample classification labels are input into the loss function to calculate the classification loss value. The gradient of the loss value with respect to all parameters of the initial model (text model, table model, classification network) is calculated using an optimizer (such as Adam, SGD). The parameters are updated according to the gradient descent method (e.g., W1 = W1 - learning rate × gradient). Iterative training and parameter saving are performed, repeating steps S604-S616 (i.e., one epoch). After each epoch, the model performance is evaluated using a validation set, and the model parameters with the highest validation set accuracy are saved as the target model parameters. After iterating to a preset number of epochs, a trained data classification model is generated based on the target model parameters.

[0086] In the embodiments of this specification, by acquiring a first set of sample table data, extracting all first sample text data from the first sample table data, concatenating the first sample text data to obtain first sample table text data, compressing the first sample table text data to obtain compressed first sample text data, inputting the compressed first sample text data into the text large model in the initial data classification model to obtain sample text feature vectors, inputting the first sample table data into the deep learning table model in the initial data classification model to obtain sample table feature vectors, fusing the sample text feature vectors and sample table feature vectors to obtain sample fusion feature vectors, inputting the sample fusion feature vectors into the classification network in the initial data classification model to obtain sample classification results corresponding to the sample table data, determining classification loss values ​​based on sample classification results and sample classification labels, updating the initial model parameters of the initial data classification model based on the classification loss values ​​to obtain adjusted target model parameters, and generating a trained data classification model based on the target model parameters. This method ensures that the model has dual-modal feature processing and classification capabilities, providing a guarantee for the accuracy of the inference stage.

[0087] Please see Figure 8 This diagram illustrates a training method for a data classification model provided in an embodiment of this specification. In one embodiment, the method further includes separate parallel training processes for a large text model and a deep learning table model, first optimizing the basic capabilities of the two models separately, and then performing subsequent fusion training. Figure 8 As shown, the method described in the embodiments of this specification may include the following steps S702-S718.

[0088] S702, Obtain the second table sample data set; the second table sample data set includes the second sample table data and the second sample classification labels corresponding to the second sample table data: In one embodiment of this specification, the second table sample data set is a sample dataset used for separate pre-training of the dual towers, containing sample data and corresponding labels. The second sample table data consists of individual sample data entries in the second table sample data set, possessing structured features (numerical and categorical) and textual features; the second sample classification label is the actual classification result corresponding to the second sample table data, providing a supervisory signal for pre-training.

[0089] The second table sample data set can be from the same source as the first table sample data set, that is, they correspond to the same target transaction scenario.

[0090] S704, extract all the second sample text data from the second sample table data, and concatenate the second sample text data to obtain the second sample table text data; In one embodiment of this specification, all text data is extracted from the second sample table data, and the second sample text data is concatenated in a fixed format to form a single sample text data, which is convenient for text tower processing. Specifically, each column of the second sample table data is traversed, and all text feature data and corresponding feature names are identified and extracted. For example, text data such as "Name: Zhang San", "Education: Bachelor's Degree", and "Loan Purpose: Purchase of Wedding House" are extracted from a single credit sample. The data is concatenated in the format of "Feature Name: Feature Value" to form a single sample of the second sample table text data. The above operation is performed on all samples in the second table sample data set to obtain a batch of second sample table text data.

[0091] S706, Compress the text data of the second sample table to obtain compressed text data of the second sample; In one embodiment of this specification, this step is a lightweight processing step in text pyramid pre-training, reducing the computational cost of the text pyramid. Compressing the second sample table text data yields compressed second sample text data, reducing the number of tokens while preserving the core semantics.

[0092] The compression process is the same as that used in the model inference stage. Specifically, for a single second sample table text data, first filter the "feature name-feature value" pairs; convert multi-token feature names (such as "loan purpose description") into single-token compressed terms "<loan purpose description>", and add them to the large text model vocabulary; replace numerical feature values ​​(such as "8000") with preset numerical terms. <num>Non-numerical feature values ​​(such as "Zhang San") are directly retained; they are concatenated in the format of "feature name compressed word: feature value" to obtain the compressed data of a single second sample text.

[0093] S708, input the compressed data of the second sample text into the initial large text model in the initial data classification model to obtain the feature vector of the second sample text; In one embodiment of this specification, the initial large-scale text model is a pre-trained text model that has not undergone fine-tuning, such as BERT or RoBERTa, and possesses basic semantic understanding capabilities. The compressed second-sample text data is encoded by the initial large-scale text model to obtain the second-sample text feature vector, representing the text's semantic information. Corresponding to the inference stage, for the feature names compressed words in the sample, their original feature names are segmented and encoded, and the average of the word vectors is taken as the vector of that compressed word; for numerical words... <num>"For now, we will use ordinary word encoding (the restoration logic will be executed in the subsequent fusion training stage); through the model encoder and pooling operation, the sequence token vector is transformed into a single second sample text feature vector of fixed dimension, and after batch processing, it provides input for the large text model update.

[0094] S710, input the second sample text feature vector into the first prediction network to obtain the first sample classification result corresponding to the second sample text feature vector; In one embodiment of this specification, the first prediction network is a lightweight classification network pre-trained separately by the text tower, which generates prediction results using previously extracted features. The class probability distribution output by the second sample text feature vector after processing by the first prediction network is also the first sample classification result. The first prediction network may contain a "linear layer + Softmax", with the input dimension consistent with the dimension of the second sample text feature vector, the output dimension matching the number of classification categories, and the parameters randomly initialized. A batch of second sample text feature vectors are input into this network, and after linear transformation for dimensionality reduction, the original scores are transformed into a class probability distribution in the 0-1 interval using the Softmax function, obtaining the batch of first sample classification results. It should be noted that the first prediction network is explicitly used only for pre-training of the large text model and does not participate in subsequent fusion training; it is discarded after training.

[0095] S712, the second sample table data is input into the initial deep learning table model in the initial data classification model to obtain the feature vector of the second sample table; In one embodiment of this specification, the initial deep learning table model is an untrained table-specific model, such as TabNet or DeepFM, which excels at processing structured data. After processing the second sample table data using the initial deep learning table model, a feature vector of the second sample table is obtained, representing the structured features and the interaction information between features.

[0096] Specifically, an initial deep learning table model (such as TabNet) with randomly initialized parameters is loaded; second sample table data is input into the model, key features are filtered through an attention feature selector, high-order interactions between features are captured through a residual network, and then integrated into a fixed-dimensional second sample table feature vector through a fully connected layer.

[0097] S714, input the feature vector of the second sample table into the second prediction network to obtain the second sample classification result corresponding to the feature vector of the sample table; In one embodiment of this specification, the second prediction network is a temporary lightweight classification network pre-trained separately by TableTower, with a structure similar to the first prediction network, and is discarded after pre-training. A lightweight network with the same structure as the first prediction network, consisting of a "linear layer + Softmax", is constructed. The input dimension matches the dimension of the feature vector of the second sample table, and the output dimension corresponds to the number of classification categories. The parameters are randomly initialized. The feature vectors of a batch of second sample tables are input into this network, processed by linear transformation and the Softmax function, and the batch of category probability distributions, i.e., the classification results of the second samples, are output.

[0098] S716, determine the text classification loss value based on the first sample classification result and the second sample classification label, update the initial model parameters of the initial large text model based on the text classification loss value to obtain the updated first model parameters, and generate the large text model based on the first model parameters.

[0099] In one embodiment of this specification, a text classification loss value is calculated to quantify the difference between the classification result of the first sample and the classification label of the second sample, reflecting the prediction error of the initial large text model. Using a preset loss function, the classification results of a batch of first samples and the corresponding classification labels of the second samples are input into the function to calculate the text classification loss value. The parameters of the first prediction network are fixed, and only the loss value is backpropagated to the initial large text model to update all model parameters. After training, the updated model parameters are saved as the first model parameters, and a large text model with strong semantic extraction capabilities is generated based on these parameters.

[0100] S718, determine the table classification loss value based on the second sample classification result and the second sample classification label, update the initial model parameters of the initial deep learning table model based on the table classification loss value, obtain the updated second model parameters, and generate a deep learning table model based on the second model parameters.

[0101] In one embodiment of this specification, the table classification loss value is an indicator that quantifies the difference between the classification result and the classification label of the second sample, reflecting the prediction error of the deep learning table model. Using the same loss function as S716, the table classification loss value between the batch of second sample classification results and the second sample classification labels is calculated. Then, the parameters of the second prediction network are fixed, and the loss value is backpropagated to the initial deep learning table model. Because the parameter size of the deep learning table model is small, a full parameter update is performed. After training, the updated parameters are saved as the parameters of the second model. Based on these parameters, a deep learning table model is generated, providing high-quality table features for subsequent fusion training.

[0102] Optionally, in one embodiment of this specification, the method further includes: S802, add low-rank parameters to the large text model, fix the first model parameters of the large text model, update the low-rank parameters of the large text model based on the classification loss value to obtain the adjusted low-rank parameters, and generate the trained large text model based on the first model parameters and the adjusted low-rank parameters. In one embodiment of this specification, parameter updates during the fusion training phase employ LoRA fine-tuning for the large text model and full updates for the table model, thus addressing the high cost of full fine-tuning for large models. Specifically, low-rank matrix pairs (A and B, e.g., A is 768×16, B is 16×768) are inserted into the key layers (such as the Transformer attention layer) of the pre-trained large text model, with the newly added parameters being much smaller than the original model parameters. Then, the first model parameters obtained from pre-training are fixed, and only the calculated classification loss value is backpropagated to the newly added low-rank parameters. The low-rank parameters are iteratively updated to obtain the adjusted low-rank parameters. The fixed first model parameters are combined with the adjusted low-rank parameters to generate a target large text model adapted to the fusion scenario.

[0103] S804, based on the classification loss value, update the second model parameters of the deep learning table model to obtain the adjusted third model parameters, and generate the trained deep learning table model based on the third model parameters; In one embodiment of this specification, for a full update of the configuration of the deep learning table model, the pre-trained second model parameters are not frozen, and the calculated classification loss value is backpropagated to all parameters of the table model. After training, the adjusted parameters are saved as third model parameters, and the target deep learning table model is generated based on these parameters. Since the table model is a deep learning model with a small parameter size (e.g., TabNet has approximately 5 million parameters), the computational cost of a full update is low. The full update enables the table model to fully learn the correlation between structured features and text features in the fusion scenario, improving its collaborative performance with large text models.

[0104] S806, Generate a trained data classification model based on the trained large text model and the trained deep learning table model.

[0105] In one embodiment of this specification, the target text large model, the target deep learning table model, and the trained classification network (with optimized parameters) are chained together according to the inference process to form a complete "text processing-table processing-fusion-classification" chain, which serves as the data classification model. That is, the data classification model includes a parallel target text large model and a target deep learning table model, which respectively extract text features and table features, then input them into a feature fusion layer for feature fusion, and finally, based on the fused feature input values, the classification network outputs the classification result.

[0106] Please see Figure 9 , Figure 9 This specification provides a flowchart of the training process for a data classification model. Training phase 1 (marked by the red line) involves training the large text model and the deep learning table model separately and in parallel before fusion training. The initial model parameters of the large text model are updated based on the text classification loss value to obtain the first model parameters. The initial model parameters of the deep learning table model are updated based on the table classification loss value to obtain the second model parameters. After training phase 1, training phase 2 (marked by the green line) is initiated. The trained large text model and deep learning table model are used to extract the sample text feature vector and sample table feature vector, respectively. These vectors are then mapped through separate mapping layers, fused in a fusion layer, and then fed into the classification network after another mapping layer to obtain the sample classification prediction result and calculate the classification loss value. A low-rank adaptation approach is adopted. Low-rank parameters are added to the large text model, while the first model parameters are fixed. The low-rank parameters are updated based on the classification loss value to obtain adjusted low-rank parameters. A trained large text model is then generated based on the first model parameters and the adjusted low-rank parameters. Similarly, the second model parameters of the deep learning table model are updated based on the classification loss value to obtain adjusted third model parameters. A trained deep learning table model is then generated based on the third model parameters. Training phase 2 is the phase subsequently used to implement the data classification method.

[0107] In the embodiments of this specification, by adding separate parallel training of the large text model and the deep learning table model before fusion training, the processing capabilities of both text modality features and table modality features are improved. Furthermore, the parallel training mode significantly improves training efficiency, allowing the large text model and the deep learning table model to independently occupy computing resources and proceed synchronously, avoiding the time loss of sequential training, which is particularly suitable for large-scale sample scenarios. This approach solves the problems of weak single-tower foundation and poor coordination in direct fusion training. The pre-trained large text model and deep learning table model possess superior feature extraction capabilities, providing high-quality feature input for subsequent fusion training.

[0108] The following will be combined with the appendix Figure 10 This specification provides a detailed description of the data classification device provided in the embodiments. It should be noted that the appendix... Figure 10 The data classification device in this specification is used to perform the functions described herein. Figures 1-9 The methods shown in the embodiments are illustrated for ease of explanation, showing only the parts related to the embodiments of this specification. For specific technical details not disclosed, please refer to this specification. Figures 1-8 The example shown.

[0109] Please see Figure 10 This diagram illustrates a schematic representation of a data classification apparatus provided in an exemplary embodiment of this specification. The data classification apparatus can be implemented as all or part of a device through software, hardware, or a combination of both. The apparatus 1 includes an acquisition unit 11, a text extraction unit 12, a text compression unit 13, a text feature extraction unit 14, a table feature extraction unit 15, a feature fusion unit 16, and a classification prediction unit 17.

[0110] Acquisition unit 11 is used to acquire table data; Text extraction unit 12 is used to extract all text data in the table data and concatenate the text data to obtain table text data; Text compression unit 13 is used to compress the table text data to obtain compressed text data. The text feature extraction unit 14 is used to input the compressed text data into the large text model to obtain the text feature vector; The table feature extraction unit 15 is used to input the table data into a deep learning table model to obtain table feature vectors; The feature fusion unit 16 is used to fuse the text feature vector and the table feature vector to obtain a fused feature vector; The classification prediction unit 17 is used to input the fused feature vector into the classification network to obtain the classification result corresponding to the table data.

[0111] Optionally, the text compression unit 13 is specifically used to filter feature names and corresponding feature values ​​from the table text data; Each of the aforementioned feature names is converted into a word element, which serves as the feature name compression word element corresponding to each of the aforementioned feature names; Text compressed data is generated by concatenating each feature name into a compressed word corresponding to a feature value.

[0112] Optionally, the text compression unit 13 is specifically used to compress the feature value of the numerical type into a numerical word according to a preset compression method if the feature value is a numerical type, and then concatenate it with the corresponding feature name compressed word to generate the first text compressed data. If the feature value is not a numerical type, then the feature value is directly concatenated with the corresponding feature name compressed token to generate the second text compressed data.

[0113] Optionally, the text compression unit 13 is specifically used to replace the feature value of the numeric type with a pre-created numeric lexical and then concatenate it with the corresponding feature name compression lexical to generate the first text compression data if the feature value is of numeric type.

[0114] Optionally, the text feature extraction unit 14 is specifically used to input each word element included in the feature name in the text compressed data into the encoder of the text large model and encode them into word vectors respectively, and take the average value of the word vectors as the feature name vector of the feature name compressed word element corresponding to the feature name; The feature values ​​in the compressed text data are input into the encoder of the large text model and encoded into a first feature value vector; The feature name vector and the first feature value vector are used as the text feature vector corresponding to the text compressed data.

[0115] Optionally, the text feature extraction unit 14 is specifically used to restore the numerical words in the first text compressed data to numerical feature values ​​according to the preset compression method of the numerical words, and input the numerical feature values ​​to the linear layer of the text large model to extract numerical feature vectors, which are used as the second feature value vectors of the numerical words. Each word element included in the feature name in the first text compressed data is input into the encoder of the text large model and encoded into word vectors respectively. The average value of the word vectors is taken as the feature name vector of the feature name compressed word element corresponding to the feature name. The feature name vector and the second feature value vector are used as the text feature vectors corresponding to the first text compressed data.

[0116] Optionally, the feature fusion unit 16 is specifically used to perform projection transformation on the text feature vector and the table feature vector respectively to obtain the text projection feature vector corresponding to the text feature vector and the table projection feature vector corresponding to the table feature vector; The text projection feature vector and the table projection feature vector are concatenated to obtain the initial fused feature vector; The initial fused feature vector is transformed by projection to obtain the fused feature vector.

[0117] Optionally, the classification prediction unit 17 is specifically used to input the fused feature vector into the classification network to obtain the class probability output by the classification network; The classification result corresponding to the table data is determined based on the category probability.

[0118] Please see Figure 11 This diagram illustrates a schematic representation of a data classification apparatus provided in an exemplary embodiment of this specification. The data classification apparatus can be implemented as all or part of the apparatus through software, hardware, or a combination of both. The apparatus 2 includes a sample acquisition unit 21, a sample text extraction unit 22, a sample compression unit 23, a sample text feature extraction unit 24, a sample table feature extraction unit 25, a sample fusion unit 26, a classification unit 27, and a training unit 28.

[0119] The sample acquisition unit 21 is used to acquire a first table sample data set; the first table sample data set includes first sample table data and first sample classification labels corresponding to the first sample table data. The sample text extraction unit 22 is used to extract all the first sample text data in the first sample table data and concatenate the first sample text data to obtain the first sample table text data. The sample compression unit 23 is used to compress the first sample table text data to obtain compressed first sample text data after compression. The sample text feature extraction unit 24 is used to input the first sample text compressed data into the large text model in the initial data classification model to obtain the sample text feature vector; The sample table feature extraction unit 25 is used to input the first sample table data into the deep learning table model in the initial data classification model to obtain the sample table feature vector. The sample fusion unit 26 is used to fuse the sample text feature vector and the sample table feature vector to obtain a sample fusion feature vector; Classification unit 27 is used to input the sample fusion feature vector into the classification network of the initial data classification model to obtain the sample classification result corresponding to the sample table data; Training unit 28 is used to determine a classification loss value based on the sample classification result and the sample classification label, update the initial model parameters of the initial data classification model based on the classification loss value to obtain the adjusted target model parameters, and generate a trained data classification model based on the target model parameters.

[0120] Optionally, the device 2 further includes a pre-training unit 29, used to acquire a second set of table sample data; the second set of table sample data includes second sample table data and second sample classification labels corresponding to the second sample table data; Extract all the second sample text data from the second sample table data, and concatenate the second sample text data to obtain the second sample table text data; The text data of the second sample table is compressed to obtain compressed text data of the second sample. The compressed data of the second sample text is input into the initial large text model in the initial data classification model to obtain the feature vector of the second sample text; The second sample text feature vector is input into the first prediction network to obtain the first sample classification result corresponding to the second sample text feature vector; The second sample table data is input into the initial deep learning table model in the initial data classification model to obtain the feature vector of the second sample table; The feature vector of the second sample table is input into the second prediction network to obtain the classification result of the second sample corresponding to the feature vector of the sample table. Based on the classification results of the first sample and the classification labels of the second sample, the text classification loss value is determined. Based on the text classification loss value, the initial model parameters of the initial large text model are updated to obtain the updated first model parameters. Based on the first model parameters, the large text model is generated. Based on the classification results and labels of the second samples, a table classification loss value is determined. The initial model parameters of the initial deep learning table model are updated based on the table classification loss value to obtain the updated second model parameters. A deep learning table model is then generated based on the second model parameters.

[0121] Optionally, the training unit 28 is specifically used to add low-rank parameters to the large text model, fix the first model parameters of the large text model, update the low-rank parameters of the large text model based on the classification loss value to obtain the adjusted low-rank parameters, and generate a trained large text model based on the first model parameters and the adjusted low-rank parameters. The second model parameters of the deep learning table model are updated based on the classification loss value to obtain the adjusted third model parameters, and the trained deep learning table model is generated based on the third model parameters. A trained data classification model is generated based on the trained large text model and the trained deep learning table model.

[0122] It should be noted that the data classification device provided in the above embodiments is only illustrated by the division of the above functional modules when executing the data classification method or the data classification model training method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the data classification device provided in the above embodiments and the data classification method and data classification model training method embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0123] It is understood that the data classification device provided in the embodiments of this specification can be a terminal device such as a mobile phone, computer, tablet computer, smartwatch or vehicle device, or it can be a module in the terminal device used to implement the data classification method and the training method of the data classification model.

[0124] The embodiment numbers in this specification are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0125] This specification also provides a storage medium storing a computer program, which, when executed by a processor, implements the above-described functionality. Figures 1-9 The data classification method and data classification model training method described in the illustrated embodiment can be found in the following documentation for details. Figures 1-9 The specific details of the illustrated embodiments will not be elaborated here.

[0126] Please refer to Figure 12 This diagram illustrates the structure of an electronic device provided in an exemplary embodiment of this specification. The electronic device in this specification may include one or more components such as a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, memory 120, input device 130, and output device 140 may be connected via the bus 150.

[0127] Processor 110 may include one or more processing cores. Processor 110 connects to various parts of the electronic device using various interfaces and lines, and performs various functions and processes data of electronic device 100 by running or executing instructions, programs, code sets, or instruction sets stored in memory 120, and by calling data stored in memory 120. Optionally, processor 110 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 110 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user page, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 110, but may be implemented separately using a communication chip.

[0128] The memory 120 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 120 may include non-transitory computer-readable storage medium. The memory 120 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the various method embodiments described above, etc. The operating system may be the Android system, including systems deeply developed based on the Android system, the iOS system developed by Apple Inc., including systems deeply developed based on the iOS system, or other systems.

[0129] The memory 120 can be divided into operating system space and user space. The operating system runs in the operating system space, while native and third-party applications run in user space. To ensure that different third-party applications can achieve good running performance, the operating system allocates corresponding system resources for each application. However, different application scenarios within the same third-party application have different requirements for system resources. For example, in local resource loading scenarios, third-party applications have high requirements for disk read speed; in animation rendering scenarios, third-party applications have high requirements for GPU performance. Since the operating system and third-party applications are independent of each other, the operating system often cannot promptly perceive the current application scenario of a third-party application, resulting in the operating system's inability to adapt system resources accordingly.

[0130] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to establish data communication between the third-party applications and the operating system. This would allow the operating system to obtain the current scenario information of the third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0131] The input device 130 is used to receive input instructions or data, and includes, but is not limited to, a keyboard, mouse, camera, microphone, or touch device. The output device 140 is used to output instructions or data, and includes, but is not limited to, a display device and a speaker. In one example, the input device 130 and the output device 140 can be combined, and the input device 130 and the output device 140 can be a touch display screen.

[0132] The touch display screen can be designed as a full-screen, curved screen, or irregularly shaped screen. It can also be designed as a combination of a full-screen and a curved screen, or a combination of an irregularly shaped screen and a curved screen; however, this specification does not limit the specific design of the embodiments.

[0133] In addition, those skilled in the art will understand that the structure of the electronic device shown in the above figures does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the electronic device may also include radio frequency circuits, input units, sensors, audio circuits, WiFi modules, power supplies, Bluetooth modules, etc., which will not be described in detail here.

[0134] exist Figure 11 In the illustrated electronic device, the processor 110 can be used to call computer applications stored in the memory 120 and specifically perform the following operations: Retrieve table data; Extract all text data from the table data and concatenate the text data to obtain the table text data; The table text data is compressed to obtain compressed text data. The compressed text data is input into the large text model to obtain the text feature vector; The table data is input into a deep learning table model to obtain table feature vectors; The text feature vector and the table feature vector are fused to obtain a fused feature vector; The fused feature vector is input into the classification network to obtain the classification result corresponding to the table data.

[0135] In one embodiment, when the processor 110 performs compression processing on the table text data to obtain compressed text data, it specifically performs the following operations: Filter feature names and corresponding feature values ​​from the text data in the table; Each of the aforementioned feature names is converted into a word element, which serves as the feature name compression word element corresponding to each of the aforementioned feature names; Text compressed data is generated by concatenating each feature name into a compressed word corresponding to a feature value.

[0136] In one embodiment, when the processor 110 performs the following operations to generate compressed text data by concatenating the data according to the format that each feature name compresses a word corresponding to a feature value: If the feature value is a numerical type, then the numerical feature value is compressed into a numerical word according to a preset compression method and then concatenated with the corresponding feature name compressed word to generate the first text compressed data; If the feature value is not a numerical type, then the feature value is directly concatenated with the corresponding feature name compressed token to generate the second text compressed data.

[0137] In one embodiment, when the processor 110 performs the operation of compressing the feature value into a numeric term according to a preset compression method if the feature value is a numeric type, and then concatenating it with the corresponding feature name compressed term to generate the first text compressed data, the processor 110 specifically performs the following operations: If the feature value is a numeric type, then the numeric feature value is replaced with a pre-created numeric lexicon and concatenated with the corresponding feature name compressed lexicon to generate the first text compressed data.

[0138] In one embodiment, the compressed text data includes first compressed text data, and when the processor 110 executes the operation of inputting the compressed text data into a large text model to obtain a text feature vector, it specifically performs the following operations: According to the preset compression method of the numerical lexical units, the numerical lexical units in the first text compressed data are restored to numerical feature values. The numerical feature values ​​are then input into the linear layer of the large text model to extract numerical feature vectors, which are used as the second feature value vectors of the numerical lexical units. Each word element included in the feature name in the first text compressed data is input into the encoder of the text large model and encoded into word vectors respectively. The average value of the word vectors is taken as the feature name vector of the feature name compressed word element corresponding to the feature name. The feature name vector and the second feature value vector are used as the text feature vectors corresponding to the first text compressed data.

[0139] In one embodiment, when the processor 110 performs the fusion of the text feature vector and the table feature vector to obtain the fused feature vector, it specifically performs the following operations: Perform projection transformations on the text feature vector and the table feature vector respectively to obtain the text projection feature vector corresponding to the text feature vector and the table projection feature vector corresponding to the table feature vector; The text projection feature vector and the table projection feature vector are concatenated to obtain the initial fused feature vector; The initial fused feature vector is transformed by projection to obtain the fused feature vector.

[0140] In one embodiment, when the processor 110 inputs the fused feature vector into the classification network to obtain the classification result corresponding to the tabular data, it specifically performs the following operations: The fused feature vector is input into the classification network to obtain the class probability output by the classification network; The classification result corresponding to the table data is determined based on the category probability.

[0141] In one embodiment, processor 110 can be used to invoke a computer application stored in memory 120 and specifically perform the following operations: Obtain a first set of table sample data; the first set of table sample data includes first sample table data and first sample classification labels corresponding to the first sample table data. Extract all first sample text data from the first sample table data, and concatenate the first sample text data to obtain the first sample table text data; The first sample table text data is compressed to obtain compressed first sample text data. The compressed data of the first sample text is input into the large text model in the initial data classification model to obtain the feature vector of the sample text; The first sample table data is input into the deep learning table model in the initial data classification model to obtain the sample table feature vector; The sample text feature vector and the sample table feature vector are fused to obtain the sample fused feature vector; The sample fusion feature vector is input into the classification network of the initial data classification model to obtain the sample classification result corresponding to the sample table data; Based on the sample classification results and the sample classification labels, a classification loss value is determined. Based on the classification loss value, the initial model parameters of the initial data classification model are updated to obtain the adjusted target model parameters. Based on the target model parameters, a trained data classification model is generated.

[0142] In one embodiment, before the processor 110 executes the operation of inputting the first sample text compressed data into the large text model in the initial data classification model to obtain the sample text feature vector, it is also configured to perform the following operations: Obtain the second table sample data set; the second table sample data set includes the second sample table data and the second sample classification labels corresponding to the second sample table data; Extract all the second sample text data from the second sample table data, and concatenate the second sample text data to obtain the second sample table text data; The text data of the second sample table is compressed to obtain compressed text data of the second sample. The compressed data of the second sample text is input into the initial large text model in the initial data classification model to obtain the feature vector of the second sample text; The second sample text feature vector is input into the first prediction network to obtain the first sample classification result corresponding to the second sample text feature vector; The second sample table data is input into the initial deep learning table model in the initial data classification model to obtain the feature vector of the second sample table; The feature vector of the second sample table is input into the second prediction network to obtain the classification result of the second sample corresponding to the feature vector of the sample table. Based on the classification results of the first sample and the classification labels of the second sample, the text classification loss value is determined. Based on the text classification loss value, the initial model parameters of the initial large text model are updated to obtain the updated first model parameters. Based on the first model parameters, the large text model is generated. Based on the classification results and labels of the second samples, a table classification loss value is determined. The initial model parameters of the initial deep learning table model are updated based on the table classification loss value to obtain the updated second model parameters. A deep learning table model is then generated based on the second model parameters.

[0143] In one embodiment, when the processor 110 updates the initial model parameters of the initial data classification model based on the classification loss value to obtain the adjusted target model parameters, and generates a trained data classification model based on the target model parameters, it specifically performs the following operations: A low-rank parameter is added to the large text model. The first model parameter of the large text model is fixed. The low-rank parameter of the large text model is updated based on the classification loss value to obtain the adjusted low-rank parameter. The trained large text model is generated based on the first model parameter and the adjusted low-rank parameter. The second model parameters of the deep learning table model are updated based on the classification loss value to obtain the adjusted third model parameters, and the trained deep learning table model is generated based on the third model parameters. A trained data classification model is generated based on the trained large text model and the trained deep learning table model.

[0144] In the embodiments of this specification, table data is acquired, all text data is extracted from the table data, the text data is concatenated to obtain table text data, the table text data is compressed to obtain compressed text data, the compressed text data is input into a large text model to obtain text feature vectors, the table data is input into a deep learning table model to obtain table feature vectors, the text feature vectors and table feature vectors are fused to obtain fused feature vectors, and the fused feature vectors are input into a classification network to obtain the classification result corresponding to the table data. This method textifies the table, utilizes the powerful text understanding ability and world knowledge of the large model to enhance the understanding of irregular text. In addition, reducing the amount of data through text compression accelerates the inference of the large model, while fusing the table model feature vectors, inputting the same data into different modalities (table, text), can obtain gains and improve classification accuracy.

[0145] Furthermore, by filtering key-value pairs of feature names and corresponding feature values ​​from the tabular text data, each feature name is converted into a single token as a compressed feature name token. These tokens are then concatenated according to the format "one compressed feature name token corresponds to one feature value" to generate compressed text data. This compression method effectively avoids the semantic fragmentation problem caused by feature name splitting in traditional word segmentation, ensuring the semantic integrity of feature names while reducing the number of tokens. The fixed concatenation format of S206 provides standardized input for subsequent large-scale text model encoding, enabling the model to quickly identify the semantic relationship between "feature name and feature value," reducing the semantic parsing cost of the model, and ultimately achieving a balance between improved compression efficiency and preservation of semantic information.

[0146] Furthermore, after splitting the feature names into tokens, the resulting basic tokens are input into an encoder to obtain word vectors for each token. Finally, the arithmetic mean of these word vectors is calculated and used as the feature name vector for the compressed token. Feature values ​​are directly encoded and extracted to obtain the first feature value vector. Further, for the first text compression data containing numerical tokens, numerical token restoration is performed. The numerical feature values ​​are input into the linear layer of the large text model to extract numerical feature vectors, which serve as the second feature value vector for the numerical tokens, thus balancing compression efficiency with the preservation of numerical information.

[0147] Furthermore, a method for fusing text feature vectors and table feature vectors is provided. By projecting the text feature vectors and table feature vectors separately, the dimensions of the two are made the same, which facilitates feature concatenation. Then, a second projection process is performed. The output dimension of the second projection can directly match the input dimension of the classification network. At the same time, the feature distribution is optimized by standardization, so that the fused features are more in line with the decision boundary of the classification task, thereby improving the classification accuracy.

[0148] Furthermore, by acquiring the first set of sample table data, all first sample text data are extracted from the first sample table data. The first sample text data are concatenated to obtain the first sample table text data. The first sample table text data is compressed to obtain compressed first sample text data. The compressed first sample text data is input into the large text model in the initial data classification model to obtain sample text feature vectors. The first sample table data is input into the deep learning table model in the initial data classification model to obtain sample table feature vectors. The sample text feature vectors and sample table feature vectors are fused to obtain sample fused feature vectors. The sample fused feature vectors are input into the classification network in the initial data classification model to obtain the sample classification results corresponding to the sample table data. The classification loss value is determined based on the sample classification results and sample classification labels. The initial model parameters of the initial data classification model are updated based on the classification loss value to obtain the adjusted target model parameters. The trained data classification model is generated based on the target model parameters. This method can ensure that the model has dual-modal feature processing and classification capabilities, providing a guarantee for the accuracy of the inference stage.

[0149] Furthermore, by adding separate parallel training of the large text model and the deep learning table model before fusion training, the processing capabilities of both text modality features and table modality features are improved. In addition, the parallel training mode significantly improves training efficiency, allowing the large text model and the deep learning table model to independently occupy computing resources and proceed synchronously, avoiding the time loss of sequential training, which is particularly suitable for large-scale sample scenarios. This approach solves the problems of weak single-tower foundation and poor coordination in direct fusion training. The pre-trained large text model and deep learning table model possess superior feature extraction capabilities, providing high-quality feature input for subsequent fusion training.

[0150] Additionally, embodiments of this specification provide a computer program product comprising a computer program that, when executed by a processor of an electronic device, enables the processor to at least perform the functions described above. Figures 1 to 8 The method provided in the illustrated embodiment.

[0151] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0152] The above-disclosed embodiments are merely preferred embodiments of this specification and should not be construed as limiting the scope of this specification. Therefore, any equivalent variations made in accordance with the claims of this specification shall still fall within the scope of this specification.< / num> < / num> < / num> < / num> < / num> < / num> < / num> < / num> < / num> < / num> < / mon1> < / num> < / num> < / num> < / num>

Claims

1. A data classification method, the method comprising: Retrieve table data; Extract all text data from the table data and concatenate the text data to obtain the table text data; The table text data is compressed to obtain compressed text data. The compressed text data is input into the large text model to obtain the text feature vector; The table data is input into a deep learning table model to obtain table feature vectors; The text feature vector and the table feature vector are fused to obtain a fused feature vector; The fused feature vector is input into the classification network to obtain the classification result corresponding to the table data.

2. The method as described in claim 1, wherein compressing the table text data to obtain compressed text data includes: Filter feature names and corresponding feature values ​​from the text data in the table; Each of the aforementioned feature names is converted into a word element, which serves as the feature name compression word element corresponding to each of the aforementioned feature names; Text compressed data is generated by concatenating each feature name compressed word in the format that corresponds to a feature value.

3. The method as described in claim 2, wherein concatenating text compressed data according to the format of each feature name compressed token corresponding to a feature value, includes: If the feature value is a numerical type, then the numerical feature value is compressed into a numerical word according to a preset compression method and then concatenated with the corresponding feature name compressed word to generate the first text compressed data; If the feature value is not a numerical type, then the feature value is directly concatenated with the corresponding feature name compressed token to generate the second text compressed data.

4. The method as described in claim 3, wherein if the feature value is a numerical type, then compressing the numerical feature value into numerical tokens according to a preset compression method and concatenating them with the corresponding feature name compressed tokens to generate first text compressed data includes: If the feature value is a numeric type, then the numeric feature value is replaced with a pre-created numeric lexicon and concatenated with the corresponding feature name compressed lexicon to generate the first text compressed data.

5. The method as described in claim 2, wherein inputting the compressed text data into a large text model to obtain text feature vectors includes: Each word element included in the feature name in the compressed text data is input into the encoder of the large text model and encoded into word vectors respectively. The average value of the word vectors is taken as the feature name vector of the feature name compressed word element corresponding to the feature name. The feature values ​​in the compressed text data are input into the encoder of the large text model and encoded into a first feature value vector; The feature name vector and the first feature value vector are used as the text feature vector corresponding to the text compressed data.

6. The method of claim 3, wherein the compressed text data includes first compressed text data, and the step of inputting the compressed text data into a large text model to obtain a text feature vector includes: According to the preset compression method of the numerical lexical units, the numerical lexical units in the first text compressed data are restored to numerical feature values. The numerical feature values ​​are then input into the linear layer of the large text model to extract numerical feature vectors, which are used as the second feature value vectors of the numerical lexical units. Each word element included in the feature name in the first text compressed data is input into the encoder of the text large model and encoded into word vectors respectively. The average value of the word vectors is taken as the feature name vector of the feature name compressed word element corresponding to the feature name. The feature name vector and the second feature value vector are used as the text feature vectors corresponding to the first text compressed data.

7. The method as described in claim 1, wherein fusing the text feature vector and the table feature vector to obtain the fused feature vector comprises: Perform projection transformations on the text feature vector and the table feature vector respectively to obtain the text projection feature vector corresponding to the text feature vector and the table projection feature vector corresponding to the table feature vector; The text projection feature vector and the table projection feature vector are concatenated to obtain the initial fused feature vector; The initial fused feature vector is transformed by projection to obtain the fused feature vector.

8. The method as described in claim 1, wherein inputting the fused feature vector into a classification network to obtain the classification result corresponding to the tabular data includes: The fused feature vector is input into the classification network to obtain the class probability output by the classification network; The classification result corresponding to the table data is determined based on the category probability.

9. A method for training a data classification model, the method comprising: Obtain the first set of sample data; The first set of table sample data includes first sample table data and first sample classification labels corresponding to the first sample table data; Extract all first sample text data from the first sample table data, and concatenate the first sample text data to obtain the first sample table text data; The first sample table text data is compressed to obtain compressed first sample text data. The compressed data of the first sample text is input into the large text model in the initial data classification model to obtain the feature vector of the sample text; The first sample table data is input into the deep learning table model in the initial data classification model to obtain the sample table feature vector; The sample text feature vector and the sample table feature vector are fused to obtain the sample fused feature vector; The sample fusion feature vector is input into the classification network of the initial data classification model to obtain the sample classification result corresponding to the sample table data; Based on the sample classification results and the sample classification labels, a classification loss value is determined. Based on the classification loss value, the initial model parameters of the initial data classification model are updated to obtain the adjusted target model parameters. Based on the target model parameters, a trained data classification model is generated.

10. The method of claim 9, further comprising, before inputting the first sample text compressed data into the large text model in the initial data classification model to obtain the sample text feature vector: Obtain the sample data set for the second table; The second table sample data set includes the second sample table data and the second sample classification labels corresponding to the second sample table data; Extract all the second sample text data from the second sample table data, and concatenate the second sample text data to obtain the second sample table text data; The text data of the second sample table is compressed to obtain compressed text data of the second sample. The compressed data of the second sample text is input into the initial large text model in the initial data classification model to obtain the feature vector of the second sample text; The second sample text feature vector is input into the first prediction network to obtain the first sample classification result corresponding to the second sample text feature vector; The second sample table data is input into the initial deep learning table model in the initial data classification model to obtain the feature vector of the second sample table; The feature vector of the second sample table is input into the second prediction network to obtain the classification result of the second sample corresponding to the feature vector of the sample table. Based on the classification results of the first sample and the classification labels of the second sample, the text classification loss value is determined. Based on the text classification loss value, the initial model parameters of the initial large text model are updated to obtain the updated first model parameters. Based on the first model parameters, the large text model is generated. Based on the classification results and labels of the second samples, a table classification loss value is determined. The initial model parameters of the initial deep learning table model are updated based on the table classification loss value to obtain the updated second model parameters. A deep learning table model is then generated based on the second model parameters.

11. The method of claim 10, wherein updating the initial model parameters of the initial data classification model based on the classification loss value to obtain adjusted target model parameters, and generating a trained data classification model based on the target model parameters, comprises: A low-rank parameter is added to the large text model. The first model parameter of the large text model is fixed. The low-rank parameter of the large text model is updated based on the classification loss value to obtain the adjusted low-rank parameter. The trained large text model is generated based on the first model parameter and the adjusted low-rank parameter. The second model parameters of the deep learning table model are updated based on the classification loss value to obtain the adjusted third model parameters, and the trained deep learning table model is generated based on the third model parameters. A trained data classification model is generated based on the trained large text model and the trained deep learning table model.

12. A data classification apparatus, the apparatus comprising: The retrieval unit is used to retrieve table data; The text extraction unit is used to extract all text data from the table data and concatenate the text data to obtain the table text data. A text compression unit is used to compress the table text data to obtain compressed text data. The text feature extraction unit is used to input the compressed text data into the large text model to obtain text feature vectors; The table feature extraction unit is used to input the table data into a deep learning table model to obtain table feature vectors; A feature fusion unit is used to fuse the text feature vector and the table feature vector to obtain a fused feature vector; The classification prediction unit is used to input the fused feature vector into the classification network to obtain the classification result corresponding to the table data.

13. A data classification apparatus, the apparatus comprising: The sample acquisition unit is used to acquire the first table of sample data. The first set of table sample data includes first sample table data and first sample classification labels corresponding to the first sample table data; The sample text extraction unit is used to extract all the first sample text data in the first sample table data and concatenate the first sample text data to obtain the first sample table text data. The sample compression unit is used to compress the first sample table text data to obtain compressed first sample text data. The sample text feature extraction unit is used to input the first sample text compressed data into the large text model in the initial data classification model to obtain the sample text feature vector; The sample table feature extraction unit is used to input the first sample table data into the deep learning table model in the initial data classification model to obtain the sample table feature vector. A sample fusion unit is used to fuse the sample text feature vector and the sample table feature vector to obtain a sample fusion feature vector; A classification unit is used to input the sample fusion feature vector into the classification network of the initial data classification model to obtain the sample classification result corresponding to the sample table data; The training unit is used to determine the classification loss value based on the sample classification result and the sample classification label, update the initial model parameters of the initial data classification model based on the classification loss value to obtain the adjusted target model parameters, and generate the trained data classification model based on the target model parameters.

14. An electronic device comprising: Processor and memory; The memory stores a computer program adapted to be loaded by the processor and to execute the steps of the method as described in any one of claims 1 to 11.

15. A storage medium storing a computer program that, when executed by a processor, implements the steps of the method as claimed in any one of claims 1 to 11.

16. A computer program product comprising: A computer program, when executed by a processor of an electronic device, causes the processor to perform the steps of the method as described in any one of claims 1 to 11.

Images