Data table classification method and device

Abstract

An embodiment of the present invention provides a data table classification method and device, which can be used in the field of artificial intelligence technology. The method includes: training a bidirectional encoder based on a constructed sample data table, constructing a first classification model based on the table name and a second classification model based on the data in the table; using the first classification model and the second classification model, classifying and distinguishing the data table to be classified to obtain a classification result, which can save human resources and improve the accuracy and efficiency of data table classification.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, in particular to the field of artificial intelligence technology, and more particularly to a data table classification method and device. Background Art

[0002] With the development of informatization across various industries, an increasing amount of data is stored in databases in the form of structured tables. Due to the rapid pace of industry development, the wide range of business sources, and the complexity of business lines, numerous data tables within databases are stored disorganized. In related technologies, data managers manually categorize data tables in databases. This manual categorization relies on subjective experience, allowing them to make only a rough assessment of table types based on table names or their business significance, resulting in low accuracy. Furthermore, as the number of data tables continues to increase, manual judgment consumes significant human resources, resulting in low classification efficiency. Summary of the Invention

[0003] One object of the present invention is to provide a data table classification method that can save human resources and improve the accuracy and efficiency of data table classification. Another object of the present invention is to provide a data table classification device. Another object of the present invention is to provide a computer-readable medium. Yet another object of the present invention is to provide a computer device.

[0004] In order to achieve the above objectives, the present invention discloses a data table classification method, comprising:

[0005] Based on the constructed sample data table, the bidirectional encoder is trained to construct a first classification model based on the table name and a second classification model based on the data in the table;

[0006] The data table to be classified is classified and judged by the first classification model and the second classification model to obtain a classification result.

[0007] Preferably, before training the bidirectional encoder according to the constructed sample data table and constructing the first classification model based on the table name and the second classification model based on the data in the table, the method further includes:

[0008] The acquired data tables in the library are marked according to the preset marking categories to construct a sample data table.

[0009] Preferably, the sample data table includes a sample table name and data in the sample table;

[0010] Based on the constructed sample data table, the bidirectional encoder is trained to build a first classification model based on the table name and a second classification model based on the data in the table, including:

[0011] According to the sample table names, the bidirectional encoder is trained to build the first classification model based on the table name;

[0012] According to the data in the sample table, the bidirectional encoder is trained to build a second classification model based on the data in the table.

[0013] Preferably, the bidirectional encoder is trained based on the sample table names to construct a first classification model based on the table names, including:

[0014] Perform encoding preprocessing on the sample table name to obtain the table name encoding;

[0015] Adding the set fully connected layer to the bidirectional encoder to obtain a first updated encoder;

[0016] According to the preset first hyperparameter and the table name encoding, the first update encoder is trained to obtain a first classification model.

[0017] Preferably, the bidirectional encoder is trained based on the data in the sample table to construct a second classification model based on the data in the table, including:

[0018] Perform data preprocessing on the data in the sample table to obtain data coding;

[0019] Adding the set fully connected layer group to the bidirectional encoder to obtain a second updated encoder, wherein the fully connected layer group includes a plurality of fully connected layers;

[0020] According to the preset second hyperparameters and data encoding, the second update encoder is trained to obtain a second classification model.

[0021] Preferably, data preprocessing is performed on the data in the sample table to obtain data coding, including:

[0022] Filter out the data in the sample table whose field type is character type from the data in the sample table;

[0023] The data in the filtered sample table is pre-coded to obtain data coding.

[0024] Preferably, before encoding preprocessing the data in the screened sample table to obtain the data encoding, the method further includes:

[0025] Calculate the amount of data in the sample table that has been filtered out;

[0026] If the amount of data is greater than the set threshold, a specified number of sample table data will be randomly sampled.

[0027] Preferably, the first classification model and the second classification model are used to classify the data table to be classified to obtain a classification result, including:

[0028] Classify the data table to be classified using the first classification model to obtain a first classification result;

[0029] The data table to be classified is classified by the second classification model to obtain the second classification result;

[0030] The first classification result and the second classification result are discriminated to obtain a classification result.

[0031] Preferably, the first classification result and the second classification result are discriminated to obtain a classification result, including:

[0032] If the first classification result is the same as the second classification result, the first classification result is used as the classification result of the data table to be classified;

[0033] If the first classification result is different from the second classification result, a secondary verification message is generated and displayed.

[0034] The present invention also discloses a data table classification device, comprising:

[0035] A construction unit, configured to train a bidirectional encoder based on the constructed sample data table, and to construct a first classification model based on the table name and a second classification model based on the data in the table;

[0036] The classification unit is used to classify the data table to be classified through the first classification model and the second classification model to obtain the classification result

[0037] The present invention also discloses a computer-readable medium on which a computer program is stored. When the program is executed by a processor, the method described above is implemented.

[0038] The present invention also discloses a computer device, including a memory and a processor, wherein the memory is used to store information including program instructions, the processor is used to control the execution of program instructions, and the processor implements the above method when executing the program.

[0039] The present invention also discloses a computer program product, comprising a computer program / instruction, which implements the above method when executed by a processor.

[0040] The present invention trains a bidirectional encoder based on a constructed sample data table, constructs a first classification model based on the table name and a second classification model based on the data in the table; the data table to be classified is classified and judged by the first classification model and the second classification model to obtain a classification result, which can save human resources and improve the accuracy and efficiency of data table classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 A flowchart of a data table classification method provided by an embodiment of the present invention;

[0043] Figure 2 A flowchart of another data table classification method provided by an embodiment of the present invention;

[0044] Figure 3 A schematic diagram of a fully connected layer design provided by an embodiment of the present invention;

[0045] Figure 4 A schematic diagram of a fully connected layer group design provided by an embodiment of the present invention;

[0046] Figure 5 A schematic structural diagram of a data table classification device provided by an embodiment of the present invention;

[0047] Figure 6 A schematic structural diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0049] It should be noted that the data table classification method and device disclosed in this application can be used in the field of artificial intelligence technology, and can also be used in any field other than the field of artificial intelligence technology. The application field of the data table classification method and device disclosed in this application is not limited.

[0050] To facilitate understanding of the technical solution provided by this application, the relevant contents of the technical solution of this application are first explained below. In most industries, due to the large number of business categories, there are a large number of database tables. As the business continues to develop, the number of database tables will continue to increase in the future. Therefore, using technical means to classify and organize a large number of database tables can lay a good data foundation for subsequent business applications.

[0051] The following uses a data table classification device as an example to illustrate the implementation process of the data table classification method provided by the embodiment of the present invention. It is understandable that the execution subject of the data table classification method provided by the embodiment of the present invention includes but is not limited to the data table classification device.

[0052] Figure 1 A flow chart of a data table classification method provided by an embodiment of the present invention, such as Figure 1 As shown, the method includes:

[0053] Step 101: Train a bidirectional encoder based on the constructed sample data table to construct a first classification model based on the table name and a second classification model based on the data in the table.

[0054] Step 102: Classify and discriminate the data table to be classified using the first classification model and the second classification model to obtain a classification result.

[0055] In the technical solution provided by the embodiment of the present invention, a bidirectional encoder is trained based on the constructed sample data table to construct a first classification model based on the table name and a second classification model based on the data in the table; the data table to be classified is classified and judged through the first classification model and the second classification model to obtain a classification result, which can save human resources and improve the accuracy and efficiency of data table classification.

[0056] Figure 2 A flowchart of another data table classification method provided by an embodiment of the present invention is as follows: Figure 2 As shown, the method includes:

[0057] Step 201: Mark the acquired data table in the library according to the preset marking category to construct a sample data table.

[0058] In the embodiment of the present invention, each step is performed by a data table classification device.

[0059] In this embodiment of the present invention, the tag categories are pre-set based on the data tables in the library. The data tables in the library are data tables that have been classified by data management personnel based on the table names. The tag categories include but are not limited to risk management, retail, corporate, asset management, internet finance, financial markets, credit, channel services, partners, operational support, customer management and marketing services. Each tag category corresponds to at least one category label. For example, the tag categories and corresponding category labels are shown in Table 1:

[0060] Table 1

[0061] Serial number Tag Category Category Label 1 Risk Management Risk... 2 retail Retail... 3 Corporate Pub... 4 Asset Management Asset... 5 Internet Finance Online、Fin... 6 Financial Markets Mon, Market... 7 Credit Loan... 8 Channel Services Channel... 9 Partners Partner... 10 Operational Support Oper、Support... 11 Customer management and marketing services Cust...

[0062] Among them, the category tags corresponding to risk management include but are not limited to Risk; the category tags corresponding to retail include but are not limited to Retail; the category tags corresponding to corporate banking include but are not limited to Pub; the category tags corresponding to asset management include but are not limited to Asset; the category tags corresponding to internet finance include but are not limited to Online and Fin; the category tags corresponding to financial markets include but are not limited to Mon and Market; the category tags corresponding to credit include but are not limited to Loan; the category tags corresponding to channel services include but are not limited to Channel; the category tags corresponding to partners include but are not limited to Partner; the category tags corresponding to operations support include but are not limited to Oper and Support; the category tags corresponding to customer management and marketing services include but are not limited to Cust.

[0063] Step 202: Based on the constructed sample data table, a bidirectional encoder (Bert) is trained to construct a first classification model based on the table name and a second classification model based on the data in the table.

[0064] In the embodiment of the present invention, the sample data table includes a sample table name and data in the sample table.

[0065] In the embodiment of the present invention, the bidirectional encoder (Bert) is a pre-trained model that all developers can directly inherit without having to train from scratch. Therefore, only a small number of sample data tables are required for fine-tuning operations. The number of samples in the embodiment of the present invention is sufficient to meet the needs of model fine-tuning.

[0066] In the embodiment of the present invention, step 202 specifically includes:

[0067] Step 2021: Train the bidirectional encoder based on the sample table name to build a first classification model based on the table name.

[0068] In this embodiment of the present invention, the sample table names are pre-encoded to obtain a table name encoding. Specifically, the table name of each sample data table is read, and the table name content is encoded and converted into the encoding format required by the BERT model, which conforms to the BERT input standard. The start ([CLS]) and separation ([SEP]) markers indicate the start and separation of text content. It is worth noting that this embodiment of the present invention does not limit the specific encoding method.

[0069] In this embodiment of the present invention, a fully connected layer is added to the bidirectional encoder to obtain a first updated encoder. The number of fully connected layers is one, and the number of neurons output by the fully connected layer is set based on the number of preset label categories. As an optional solution, the number of neurons output by the fully connected layer is set to 11.

[0070] Figure 3A schematic diagram of a fully connected layer design provided by an embodiment of the present invention is shown in FIG. Figure 3 As shown in FIG, a fully connected layer with 11 output neurons (n=11) is added after the Bert output layer.

[0071] In this embodiment of the present invention, a first update encoder is trained based on the table name encoding according to a pre-set first hyperparameter to obtain a first classification model. As an optional solution, the first hyperparameter is set as follows: batch size is set to 16, epoch is set to 3, and learning rate is set to 2e-5. Specifically, the internal parameters of the BERT model are set according to the first hyperparameter, and the table name encoding is divided into a training set, a validation set, and a test set. The BERT model is trained using the training set, and the BERT model is validated and tested using the validation set and the test set. The model parameters are further fine-tuned according to actual needs.

[0072] Step 2022: Train the bidirectional encoder based on the data in the sample table to build a second classification model based on the data in the table.

[0073] In an embodiment of the present invention, the amount of data in the sample table that has been screened is statistically analyzed; if the amount of data is greater than a set quantity threshold, a specified amount of data in the sample table is randomly sampled. It is worth noting that the quantity threshold can be set according to actual conditions, and the embodiment of the present invention does not limit this. As an optional solution, for a sample table with a data amount greater than a set quantity threshold, 100,000 pieces of data are randomly sampled as training sample data, 50,000 pieces of data are used as verification sample data, and 30,000 pieces of data are used as test sample data. Random sampling of data in a sample table with a large amount of data can reduce hardware pressure and improve training efficiency.

[0074] In an embodiment of the present invention, data preprocessing is performed on the data in the sample table to obtain data encoding. Specifically, the data in the sample table whose field type is character type is filtered out from the data in the sample table; the filtered data in the sample table is encoded and preprocessed to obtain data encoding. Specifically, the data in the sample table of each sample data table is read, and the data in the sample table is encoded and converted into the encoding format required by the Bert model, which meets the input standard of Bert, and the start and separation of the text content are indicated by marking the start ([CLS]) and separation ([SEP]). It is worth noting that the embodiment of the present invention does not limit the specific encoding method.

[0075] In an embodiment of the present invention, the data in the sample table includes multiple field types, among which the sample table data of field types such as numerical and time types cannot be used to judge the category of the data table. Therefore, filtering out the character-type sample table data for data encoding can reduce hardware pressure and improve training efficiency.

[0076] In an embodiment of the present invention, a set fully connected layer group is added to a bidirectional encoder to obtain a second updated encoder, where the fully connected layer group includes multiple fully connected layers. The fully connected layer group includes three fully connected layers. As an optional solution, the number of neurons output by the three fully connected layers is set to 128, 64, and 11, respectively.

[0077] Figure 4 A schematic diagram of a fully connected layer group design provided by an embodiment of the present invention is shown in FIG. Figure 4 As shown in FIG, three fully connected layers with output neurons of 128 (n=128), 64 (n=64) and 11 (n=11) are added after the Bert output layer.

[0078] In an embodiment of the present invention, a second update encoder is trained based on data encoding according to a pre-set second hyperparameter to obtain a second classification model. As an optional solution, the second hyperparameter is set as follows: batch size is set to 32, epoch is set to 4, and learning rate is set to 2e-5. Specifically, the internal parameters of the Bert model are set according to the second hyperparameter; the Bert model is trained through data encoding of the training sample data; the Bert model is verified and tested through data encoding of the validation sample data and the test sample data, and the model parameters are further fine-tuned according to actual needs.

[0079] Step 203: Classify and judge the data table to be classified using the first classification model and the second classification model to obtain a classification result.

[0080] In the embodiment of the present invention, step 203 specifically includes:

[0081] Step 2031: Classify the data table to be classified using the first classification model to obtain a first classification result.

[0082] Specifically, the data table to be classified is input into the first classification model, and the first classification result is output.

[0083] Step 2032: Classify the data table to be classified using the second classification model to obtain a second classification result.

[0084] Specifically, the data table to be classified is input into the second classification model, and the second classification result is output.

[0085] Step 2033: discriminate the first classification result and the second classification result to obtain a classification result.

[0086] Specifically, if the first classification result is the same as the second classification result, it indicates that the first classification result and the second classification result are accurate, and the first classification result will be used as the classification result of the data table to be classified; if the first classification result is different from the second classification result, it indicates that the first classification result and the second classification result are inaccurate, and manual intervention for secondary verification is required to generate and display a secondary verification message.

[0087] It is worth noting that, since the first classification result and the second classification result are the same, the second classification result can also be used as the classification result of the data table to be classified.

[0088] Furthermore, if the first classification result is different from the second classification result, a secondary verification message is generated and displayed. The secondary verification message can be displayed on the computer interface or pushed to the terminal interface of the data management personnel to notify the data management personnel to manually intervene for secondary verification. By manually checking the data table, the machine classification results can be adjusted to improve the classification accuracy.

[0089] The embodiments of the present invention can effectively and intelligently manage a large number of data tables, improving the hierarchical nature of the business system. For data management personnel, this allows for faster and more efficient data exploration; for business personnel, this allows for a clearer understanding of the entire data system; and it can improve the manageability of data tables and increase the efficiency of data table classification.

[0090] In the technical solution of the data table classification method provided by an embodiment of the present invention, a bidirectional encoder is trained based on the constructed sample data table to construct a first classification model based on the table name and a second classification model based on the data in the table; the data table to be classified is classified and judged by the first classification model and the second classification model to obtain a classification result, which can save human resources and improve the accuracy and efficiency of data table classification.

[0091] Figure 5 A schematic diagram of the structure of a data table classification device provided by an embodiment of the present invention, the device is used to perform the above-mentioned data table classification method, such as Figure 5 As shown, the device includes: a construction unit 11 and a classification unit 12.

[0092] The construction unit 11 is used to train the bidirectional encoder according to the constructed sample data table, and to construct a first classification model based on the table name and a second classification model based on the data in the table.

[0093] The classification unit 12 is used to classify and judge the data table to be classified by using the first classification model and the second classification model to obtain a classification result.

[0094] In the embodiment of the present invention, the device further includes: a marking unit 13 .

[0095] The marking unit 13 is used to mark the acquired in-library data table according to a preset marking category to construct a sample data table.

[0096] In an embodiment of the present invention, the sample data table includes a sample table name and data in the sample table; the construction unit 11 is specifically used to train the bidirectional encoder according to the sample table name and construct a first classification model based on the table name; and train the bidirectional encoder according to the data in the sample table and construct a second classification model based on the data in the table.

[0097] In an embodiment of the present invention, the construction unit 11 is specifically used to perform encoding preprocessing on the sample table name to obtain the table name encoding; add the set fully connected layer to the bidirectional encoder to obtain a first updated encoder; according to the pre-set first hyperparameter and the table name encoding, the first updated encoder is trained to obtain a first classification model.

[0098] In an embodiment of the present invention, the construction unit 11 is specifically used to perform data preprocessing on the data in the sample table to obtain data encoding; add the set fully connected layer group to the bidirectional encoder to obtain a second updated encoder, and the fully connected layer group includes multiple fully connected layers; according to the pre-set second hyperparameter and the data encoding, the second updated encoder is trained to obtain a second classification model.

[0099] In the embodiment of the present invention, the construction unit 11 is specifically configured to filter out sample table data whose field type is character type from the sample table data; and perform encoding preprocessing on the filtered sample table data to obtain data encoding.

[0100] In the embodiment of the present invention, the device further includes: a statistical unit 14 and a sampling unit 15 .

[0101] The statistical unit 14 is used to count the amount of data in the filtered sample table.

[0102] The sampling unit 15 is used to randomly sample a specified number of data in the sample table if the data volume is greater than a set quantity threshold.

[0103] In an embodiment of the present invention, the classification unit 12 is specifically used to classify the data table to be classified through a first classification model to obtain a first classification result; classify the data table to be classified through a second classification model to obtain a second classification result; and discriminate the first classification result and the second classification result to obtain a classification result.

[0104] In the embodiment of the present invention, the classification unit 12 is specifically configured to use the first classification result as the classification result of the data table to be classified if the first classification result is the same as the second classification result; and generate and display a secondary verification message if the first classification result is different from the second classification result.

[0105] In the solution of the embodiment of the present invention, a bidirectional encoder is trained based on the constructed sample data table to construct a first classification model based on the table name and a second classification model based on the data in the table; the data table to be classified is classified and judged through the first classification model and the second classification model to obtain the classification result, which can save human resources and improve the accuracy and efficiency of data table classification.

[0106] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer device. Specifically, the computer device may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0107] An embodiment of the present invention provides a computer device including a memory and a processor, wherein the memory is used to store information including program instructions, and the processor is used to control the execution of the program instructions. When the program instructions are loaded and executed by the processor, the steps of the embodiment of the above-mentioned data table classification method are implemented. For a specific description, please refer to the embodiment of the above-mentioned data table classification method.

[0108] Reference below Figure 6 , which shows a structural diagram of a computer device 600 suitable for implementing an embodiment of the present application.

[0109] like Figure 6 As shown, the computer device 600 includes a central processing unit (CPU) 601, which can perform various appropriate tasks and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage portion 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the computer device 600 are also stored in the RAM 603. The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0110] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, and the like; an output section 607 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 608 including devices such as a hard disk; and a communication section 609 including a network interface card such as a LAN card or a modem. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. Removable media 611, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 610 as needed, so that computer programs read therefrom can be installed in the storage section 608 as needed.

[0111] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program including program code for executing the methods illustrated in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication portion 609 and / or installed from removable media 611.

[0112] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0113] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0114] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0115] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0116] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0117] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0118] The acquisition, storage, use, and processing of data in this application's technical solution comply with relevant national laws and regulations.

[0119] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0120] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0121] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0122] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A data table classification method, characterized in that: The method comprises: According to the constructed sample data table, a bidirectional encoder is trained to construct a first classification model based on the table name and a second classification model based on the data in the table; wherein the sample data table includes the sample table name and the sample table data; Using the first classification model and the second classification model, the data table to be classified is classified and judged to obtain a classification result; The method of training a bidirectional encoder based on the constructed sample data table to construct a first classification model based on the table name and a second classification model based on the data in the table includes: Training the bidirectional encoder according to the sample table name to construct a first classification model based on the table name; Training the bidirectional encoder according to the data in the sample table to construct a second classification model based on the data in the table; The step of training the bidirectional encoder according to the sample table name to construct a first classification model based on the table name includes: Performing encoding preprocessing on the sample table name to obtain a table name code; Adding the set fully connected layer to the bidirectional encoder to obtain a first updated encoder; According to the preset first hyperparameter and the table name encoding, the first update encoder is trained to obtain the first classification model.

2. The data table classification method according to claim 1, characterized in that: Before training the bidirectional encoder according to the constructed sample data table and constructing the first classification model based on the table name and the second classification model based on the data in the table, the method further includes: The acquired data tables in the library are marked according to the preset marking categories to construct a sample data table.

3. The data table classification method according to claim 1, characterized in that: The step of training the bidirectional encoder according to the data in the sample table to construct a second classification model based on the data in the table includes: Performing data preprocessing on the data in the sample table to obtain data coding; Adding a set fully connected layer group to the bidirectional encoder to obtain a second updated encoder, wherein the fully connected layer group includes a plurality of fully connected layers; According to the preset second hyperparameters and the data encoding, the second update encoder is trained to obtain the second classification model.

4. The data table classification method according to claim 3, characterized in that: The preprocessing of the data in the sample table to obtain data coding includes: Filtering the sample table data whose field type is character type from the sample table data; The data in the filtered sample table is pre-coded to obtain data coding.

5. The data table classification method according to claim 4, characterized in that: Before performing encoding preprocessing on the data in the screened sample table to obtain the data encoding, the method further includes: Calculate the amount of data in the sample table that has been filtered out; If the data volume is greater than the set threshold, a specified number of data in the sample table is randomly sampled.

6. The data table classification method according to claim 1, characterized in that: The method of performing classification discrimination on the data table to be classified by using the first classification model and the second classification model to obtain a classification result includes: Classifying the data table to be classified using the first classification model to obtain a first classification result; Classifying the to-be-classified data table using the second classification model to obtain a second classification result; The first classification result and the second classification result are discriminated to obtain a classification result.

7. The data table classification method according to claim 6, characterized in that: The discriminating the first classification result and the second classification result to obtain a classification result includes: If the first classification result is the same as the second classification result, the first classification result is used as the classification result of the table of data to be classified; If the first classification result is different from the second classification result, a secondary verification message is generated and displayed.

8. A data table classification device, characterized in that: The device comprises: A construction unit, configured to train a bidirectional encoder based on the constructed sample data table to construct a first classification model based on the table name and a second classification model based on the data in the table; wherein the sample data table includes the sample table name and the sample table data; A classification unit, configured to classify and discriminate the data table to be classified using the first classification model and the second classification model to obtain a classification result; Among them, the construction unit is specifically used to train the bidirectional encoder according to the sample table name to construct a first classification model based on the table name; train the bidirectional encoder according to the data in the sample table to construct a second classification model based on the data in the table; wherein, the training of the bidirectional encoder according to the sample table name to construct the first classification model based on the table name includes: encoding preprocessing the sample table name to obtain the table name encoding; adding the set fully connected layer to the bidirectional encoder to obtain a first updated encoder; according to the pre-set first hyperparameter, the first updated encoder is trained according to the table name encoding to obtain the first classification model.

9. A computer-readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the data table classification method according to any one of claims 1 to 7 is implemented.

10. A computer device comprising a memory and a processor, wherein the memory is used to store information including program instructions, and the processor is used to control the execution of the program instructions, wherein: When the program instructions are loaded and executed by a processor, the data table classification method according to any one of claims 1 to 7 is implemented.

11. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the data table classification method according to any one of claims 1 to 7 is implemented.

Images