Data processing method and device of database, electronic equipment

Abstract

The application provides a data processing method and device of a database, electronic equipment and a computer readable storage medium; relates to database technology and artificial intelligence technology; the method comprises the following steps: querying a data table of a missing business field in the database, and acquiring feature information of the data table; acquiring a feature vector corresponding to the feature information, mapping the feature vector into probabilities of a plurality of candidate business fields; determining a business field corresponding to a probability meeting a probability value condition as a business field to which the data table belongs; and writing the business field to which the data table belongs in the data table of the database. Through the application, intelligent data management of the database can be realized, so that the efficiency of data management is improved.

Description

Technical Field

[0001] This application relates to database technology, and more particularly to a database data processing method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Artificial intelligence (AI) is a comprehensive technology in computer science that studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making. AI is widely used in internet companies, for example, in data management.

[0003] However, in the data management of internet companies, as business grows and user activity increases, a large amount of valuable data accumulates in data tables. Related technologies rely on database administrators to organize and maintain these databases, leading to unnecessary manpower consumption and low efficiency.

[0004] There is a lack of effective solutions for automating database management based on artificial intelligence technology. Summary of the Invention

[0005] This application provides a database data processing method, apparatus, electronic device, and computer-readable storage medium, which can realize intelligent data management of the database, thereby improving data management efficiency.

[0006] The technical solution of this application embodiment is implemented as follows:

[0007] This application provides a database data processing method, including:

[0008] Query the database for missing business domain data tables and obtain the characteristic information of the data tables;

[0009] Obtain the feature vector corresponding to the feature information, and map the feature vector to the probability of multiple candidate business domains;

[0010] The business domain corresponding to the probability that satisfies the probability value condition is determined as the business domain to which the data table belongs;

[0011] In the data table of the database, the business domain to which the data table belongs is written.

[0012] This application provides a database data processing apparatus, including:

[0013] The query module is used to query data tables in the database that lack data related to specific business areas.

[0014] The acquisition module is used to acquire feature information of the data table and to acquire feature vectors corresponding to the feature information, and to map the feature vectors to the probabilities of multiple candidate business domains.

[0015] The determination module is used to determine the business domain corresponding to the probability that satisfies the probability value condition as the business domain to which the data table belongs;

[0016] The write module is used to write the business domain to which the data table belongs in the data table of the database.

[0017] In the above scheme, the acquisition module is further used for:

[0018] Determine the index value corresponding to each word in the feature information;

[0019] Based on the index value, a word vector corresponding to each word is obtained by conversion;

[0020] The word vectors corresponding to each word are summed and averaged to obtain the average word vector.

[0021] The average word vector is used as the feature vector corresponding to the feature information.

[0022] In the above scheme, the acquisition module is further used for:

[0023] The table name and table description information of the data table included in the feature information are processed by word segmentation to obtain multiple words;

[0024] Determine the index value of each word in the index table;

[0025] The index table includes different words and their corresponding index values.

[0026] In the above scheme, the acquisition module is further used for:

[0027] The corresponding one-hot vector is obtained by performing a transformation based on the index value of each word.

[0028] Multiply the one-hot vector corresponding to each word by the weight matrix to obtain the word vector corresponding to each word.

[0029] In the above scheme, the acquisition module is further used for:

[0030] The feature vector is encoded to obtain the encoding result;

[0031] The encoding process is activated to obtain the probabilities of multiple candidate business domains.

[0032] In the above scheme, the determining module is further used for:

[0033] The business domain with the highest probability is taken as the business domain to which the data table belongs; or

[0034] The business areas corresponding to probabilities exceeding the probability threshold are sorted in descending order of probability, and the top-ranked business areas are selected as the business areas to which the data table belongs.

[0035] In the above scheme, the query module is further used for:

[0036] Multiple candidate data tables are selected from the logs of the database, and the candidate data tables meet at least one of the following conditions: the usage frequency is lower than the frequency threshold, and the last usage time is before a preset time.

[0037] The missing business domain data table is determined from the multiple candidate data tables.

[0038] In the above scheme, the query module is further used for:

[0039] Identify non-overlapping data tables among the multiple nodes corresponding to the distributed database;

[0040] Iterate through the non-overlapping data tables stored in each node to determine the data table for the missing business domain.

[0041] In the above scheme, the data processing device for the database further includes a replacement module, used for:

[0042] The database tables are periodically traversed to identify those tables where data changes have occurred, and the types of data changes include at least one of the following: adding data, deleting data, or modifying data.

[0043] Obtain the feature information of the data table that has undergone data changes, and determine new business areas based on the feature information of the data table that has undergone data changes;

[0044] Replace the business domain of the data table whose data has changed with the new business domain.

[0045] In the above scheme, the determining module is further used for:

[0046] Identify multiple sets of data tables in multiple data tables with similarity exceeding a similarity threshold in the missing business domain, wherein each set of data tables includes at least two data tables;

[0047] Select one data table from each group of data tables as a representative data table, obtain the feature information of each representative data table, and determine the business domain to which the representative data table belongs based on the feature information;

[0048] The business domain to which the representative data table belongs is taken as the business domain to which other data tables in the same group belong.

[0049] In the above scheme, the data processing device of the database further includes a retrieval module, used for:

[0050] In response to a search request, the database is filtered based on the business domain to which the data tables belong, to obtain data tables within a first range;

[0051] Based on the table description information of the data tables in the first range, the data tables in the second range are obtained by filtering.

[0052] Based on the table names within the second range, the data tables within the target range are filtered and returned as search results.

[0053] In the above scheme, the business domain is determined through a machine learning model, and the data processing device of the database further includes a training module for:

[0054] The machine learning model performs the following processing: obtaining feature information of data table samples, and determining the prediction business domain to which the data table samples belong based on the feature information of the data table samples;

[0055] The error in determining the predicted business domain based on the data table sample and the labeled business domain based on the data table sample;

[0056] The error is backpropagated in the machine learning model to update the parameters of the machine learning model.

[0057] This application provides an electronic device, including:

[0058] Memory, used to store executable instructions;

[0059] The processor, when executing executable instructions stored in the memory, implements the database data processing method provided in the embodiments of this application.

[0060] This application provides a computer-readable storage medium storing executable instructions, which, when executed by a processor, implement the database data processing method provided in this application.

[0061] The embodiments of this application have the following beneficial effects:

[0062] By providing feature information about data tables with missing business domains, the system predicts the business domains of the data tables and automatically fills them in. This achieves automatic completion of the business domains of the data tables, which significantly saves database maintenance costs and improves data management efficiency for businesses that need to maintain massive amounts of data tables. Attached Figure Description

[0063] Figure 1A This is a schematic diagram of the architecture of the data processing system 10 provided in an embodiment of this application;

[0064] Figure 1B This is a schematic diagram of the architecture of the data processing system 10 provided in an embodiment of this application;

[0065] Figure 1C This is a schematic diagram of the architecture of the data processing system 10 provided in an embodiment of this application;

[0066] Figure 2 This is a schematic diagram of the structure of server 200-1 provided in an embodiment of this application;

[0067] Figure 3A This is a schematic flowchart of the database data processing method provided in the embodiments of this application;

[0068] Figure 3B This is a schematic flowchart of the database data processing method provided in the embodiments of this application;

[0069] Figure 4 This is a schematic diagram of the business domain of the Internet service provided in the embodiments of this application;

[0070] Figure 5A This is a schematic diagram of the e-commerce problem table provided in the embodiments of this application;

[0071] Figure 5B This is a schematic diagram of the e-commerce problem table provided in the embodiments of this application;

[0072] Figure 6 This is a schematic diagram of the structure of the text classification model provided in the embodiments of this application;

[0073] Figure 7 This is a schematic diagram of the interaction flow of the database data processing method provided in the embodiments of this application;

[0074] Figure 8 This is a schematic diagram illustrating the training and prediction of the text classification model provided in the embodiments of this application. Detailed Implementation

[0075] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0076] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0077] In the following description, the terms "first / second / third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0078] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0079] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0080] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0081] 1) Metadata: Data used to describe data, mainly information used to describe data attributes, and can support functions such as indicating storage location, historical data, resource lookup and file records.

[0082] 2) Data Table: Also known as a table, it is a collection of two-dimensional arrays used to represent and store relationships between data objects. A table is the basic structure that makes up a tablespace, consisting of intervals. It comprises vertical columns and horizontal rows. For example, in a table named "authors" containing author information, each column contains a specific type of information for all authors, such as "surname," while each row contains all the information for a specific author: surname, first name, address, etc.

[0083] 3) Distributed Databases: In distributed databases, data tables are stored on different physical nodes, managed by different database management systems, running on different machines, supported by different operating systems, and connected by different communication networks. The physical nodes used to store data tables include two types: master nodes and slave nodes. Each master node has at least one slave node that is synchronized with it. All data tables in a distributed database are distributed across various master nodes. Slave nodes are used to improve the reliability and availability of the database, preventing the entire database from crashing due to a master node failure.

[0084] 4) Structured Query Language (SQL): It is a special-purpose programming language, a database query and programming language used to access, query, update and manage relational database systems.

[0085] Databases store data in tables. In the data management of internet companies, as business grows and user activity increases, a large amount of valuable data accumulates in these tables. Business domains, as part of metadata, categorize data from a business perspective, making them easier for database administrators to use. However, if some valuable data tables lack specific business domains, they become difficult for database administrators to manage and maintain, significantly reducing their usability.

[0086] In related technologies, there are two main methods for completing the business domain in metadata: 1. Manual completion, which involves finding the database administrator of all data tables and having the database administrator complete all missing information; 2. Systematic restriction, which restricts the database administrator to fill in the business domain to which the data table belongs when creating a new task, otherwise the data table cannot be saved.

[0087] Method 1 has two drawbacks: First, considering personnel changes, it's uncertain whether each data table can be assigned to a specific database administrator, leading to some tables not having their corresponding business domains filled in. Second, manually filling in these domains requires a significant amount of manpower. In specific scenarios, some database administrators might need to fill in hundreds or even thousands of business domains, which would directly impact their normal work and even the normal operation of a particular business. Method 2's drawback is that it only guarantees newly added data tables have complete business domains; however, many existing data tables still lack business domains, severely impacting their usability.

[0088] This application provides a data processing method for a database, enabling intelligent data management of the database and thereby improving the efficiency of data management.

[0089] The database data processing method provided in this application embodiment can be implemented by various electronic devices. For example, it can be implemented by a terminal or a server alone, or by a server and a terminal working together. For example, the terminal can independently perform the database data processing method described below, or the terminal can send a data processing request to the server, and the server can execute the database data processing method according to the received data processing request.

[0090] The electronic devices provided in this application can be various types of terminal devices or servers. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, and this application does not impose any restrictions on this.

[0091] Taking servers as an example, such as server clusters deployed in the cloud, AI as a Service (AIaaS) is offered to users. The AIaaS platform breaks down several common AI services and provides them as independent or packaged services in the cloud. This service model is similar to an AI-themed marketplace, where all users can access and use one or more AI services provided by the AIaaS platform through application programming interfaces.

[0092] For example, one type of artificial intelligence cloud service can be a data processing service, whereby a cloud server encapsulates the data processing program provided in this application embodiment. In response to a database administrator's trigger operation for processing a data table, the database administrator's terminal sends a data processing request to the cloud server. The cloud server then calls the encapsulated data processing program to determine the data table in the database that lacks a business domain, identifies the corresponding business domain, writes the business domain into the data table, and returns the business domain to the database administrator's terminal.

[0093] In some embodiments, the data processing method of the database provided in this application embodiment is described as an example of a server implementing the method alone. The server periodically traverses the data tables in the database to determine the data tables with missing business domains and obtains the feature information of the data tables with missing business domains; based on the feature information, it determines the business domain corresponding to the data table; and replaces the business domain with a new business domain and writes it into the corresponding data table.

[0094] In some embodiments, an exemplary data processing system is described using an example of a server and a terminal collaboratively implementing a database data processing method provided in the embodiments of this application. See also Figure 1A , Figure 1A This is a schematic diagram of the architecture of the data processing system 10 provided in this application embodiment. Terminal 400-1 connects to servers 200-1 and 200-2 via network 300. Network 300 can be a wide area network (WAN), a local area network (LAN), or a combination of both. Server 200-1 can be a database server, and the business domain to which the data table belongs can be determined based on the database server. For example, server 200-1 receives a data processing request from terminal 400-1 of the database administrator via network 300, determines the business domain corresponding to the data table based on the data processing request, writes the determined business domain into the data table, and returns the writing status of the business domain to terminal 400-1 via network 300. Figure 1A Server 200-2 is a dedicated server deployed in the database server deployment to determine the business domain to which a data table belongs. In some possible instances, Server 200-2 can also determine the business domain to which a data table belongs. For example... Figure 1A As shown, server 200-1 forwards the data processing request received from terminal 400-1 to server 200-2 so that server 200-2 can determine the business domain. Then, server 200-1 writes the determined business domain into the data table and returns the writing status of the business domain to terminal 400-1.

[0095] In other embodiments, the data processing system may be as follows: Figure 1B As shown, the internet user's terminal 400-2 connects to servers 200-1 and 200-3 via network 300. Server 200-3 is the backend server for the internet application. Server 200-3 receives business requests from the internet application client in terminal 400-2, such as a question-answering request for a specific business area, and sends a query request to server 200-1 to query the corresponding business area's data table in server 200-1's database. If a missing business area's data table is found during the query process, the missing business area in the data table is first filled in using the database data processing method provided in this application embodiment. Then, the data table corresponding to the specific business area (such as the question-answering table) is determined, and the data processing result (data table) is returned to the internet user's terminal 400-2.

[0096] For example, in a question-and-answer client application, after a user logs in, terminal 400-2 receives the user's question-and-answer action and displays multiple business areas for the user to choose from. Upon receiving the user's selection of a business area, it sends a query request to server 200-1 for the data table corresponding to that business area. In response to the query request, server 200-1 iterates through the database tables corresponding to the question-and-answer client to retrieve the data table for the user's selected business area. Simultaneously, for any data tables lacking business areas discovered during database iteration, it populates those tables. Server 200-1 selects multiple questions from the retrieved data table and sends the selected questions to terminal 400-2. Upon receiving the questions, the terminal displays the question-and-answer page.

[0097] This application's embodiments can also be implemented using blockchain technology; see [link to relevant documentation]. Figure 1C The servers and terminals mentioned above can all join the blockchain network 300 and become nodes within it. The type of blockchain network 300 is flexible and diverse; it can be any type of public chain, private chain, or consortium chain. Taking a public chain as an example, any electronic device of any business entity, such as a server, can access the blockchain network 300 without authorization to act as a consensus node. For example, server 200-3 is mapped to consensus node 300-0 in the blockchain network 300, server 200-1 is mapped to consensus node 300-1, and server 200-2 is mapped to consensus node 300-2.

[0098] Taking blockchain network 300 as a consortium blockchain as an example, servers 200-1, 200-2, and 200-3 can become nodes after obtaining authorization and connect to blockchain network 300. When a node receives a determined business domain, it can determine whether to write the business domain into a data table by executing a smart contract. When a node determines that the business domain can be written into the data table, it will sign a digital signature (i.e., endorsement) for the business domain. When a business domain has enough endorsements, such as exceeding the number of node endorsements, the business domain will be written into the data tables maintained by all servers. The database containing the data tables can run on blockchain network 300 or server 200-1, or it can be deployed independently of blockchain network 300 and server 200-1. It can be seen that by having multiple nodes confirm the business domain through consensus, the reliability and accuracy of the business domain written into the data table are further improved.

[0099] If a node discovers other data tables with similar business domains to the table description information in the data table, it can directly write the business domains of the other data tables into the data table as the business domains of the data table.

[0100] The electronic device implementing the embodiments of this application is as follows Figure 1A Taking server 200-1 as an example, the structure of the electronic device provided in the embodiments of this application is illustrated. See also Figure 2 , Figure 2 This is a schematic diagram of the structure of server 200-1 provided in an embodiment of this application. Figure 2 The server 200-1 shown includes at least one processor 210, memory 240, and at least one network interface 220. The various components in server 200-1 are coupled together via a bus system 230. It is understood that the bus system 230 is used to implement communication between these components. In addition to a data bus, the bus system 230 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 230.

[0101] Processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0102] The memory 240 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 240 may optionally include one or more storage devices physically located away from the processor 210.

[0103] The memory 240 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 240 described in this application embodiment is intended to include any suitable type of memory.

[0104] In some embodiments, memory 240 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0105] Operating system 241 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0106] The network communication module 242 is used to reach other computing devices via one or more (wired or wireless) network interfaces 220, such as Bluetooth, WiFi, and Universal Serial Bus (USB).

[0107] In some embodiments, the data processing apparatus for the database provided in this application can be implemented in software. Figure 2 A data processing device 243, which can be software in the form of programs and plug-ins, is shown. It includes the following software modules: a query module 2431, an acquisition module 2432, a determination module 2433, a writing module 2434, a replacement module 2435, a retrieval module 2436, and a training module 2437. These modules are logically connected and can therefore be arbitrarily combined or further divided according to their implemented functions. The functions of each module will be described below.

[0108] The following describes the data processing method for the database using a server as the executing entity. Specifically, the server can implement this method by running the various computer programs described above. Of course, based on the understanding of the following text, it is not difficult to see that the database data processing method provided in the embodiments of this application can also be implemented collaboratively by the terminal and the server.

[0109] See Figure 3A , Figure 3A This is a flowchart illustrating the data processing method for the database provided in this application embodiment, which will be combined with... Figure 3A The steps shown are explained.

[0110] In step 101, the database is queried for data tables in the missing business domain, and the characteristic information of the data tables is obtained.

[0111] In some embodiments, the database includes relational databases, non-relational databases, and key-value databases. Relational databases may be MySQL, Sybase, etc.; non-relational databases may be BigTable, Cassandra, etc.; and key-value databases may be Apache Cassandra, Dynamo, etc. The characteristic information of a data table may include the table name and table description.

[0112] Each data table has a corresponding table name, table description, and business domain. The table name is a unique identifier for users accessing the table; it is unique within the database. The business domain identifies the type of business the content in the data table addresses. There are various types of business domains, such as... Figure 4 As shown, Figure 4This is a schematic diagram illustrating the business areas of internet services provided in this application embodiment. Internet services can be divided into two main categories: internet infrastructure services and internet application services. The internet infrastructure services can be further subdivided into access services, domain name server (DNS) services, internet data center (IDC) services, content delivery network (CDN) services, and e-commerce. The internet application services can be subdivided into online entertainment, online content, online communication, information retrieval, data mining, and emerging applications. Table description information may include field ID, field name, data type, length, precision, whether it is null, default value, whether it is auto-incrementing, whether it is a primary key, and column description, etc.

[0113] Taking the database tables corresponding to the question-answering client as an example to illustrate the table description information, these tables include the question table, friend table, and tacit understanding value table, etc. Figure 5A As shown, table description information can be presented in the data table. Figure 5A The table name can be "E-commerce Problem Table", where the first column is the field name, the second column is the data type, the third column is the length, the fourth column is the precision, and the fifth column (the data in this column is defaulted) indicates whether it can be null. In some embodiments, an SQL query can be used to query the business domains corresponding to each table in the database to determine the tables for missing business domains.

[0114] In other embodiments, considering that infrequently accessed data tables in the database may be due to low inherent value or missing business domains, and that some data tables, despite lacking business domains, can still be quickly accessed based on their table names and / or descriptions, their access frequency may still be high. Therefore, when querying data tables in the database that lack business domains—that is, determining which tables need to have their business domains filled in—to improve efficiency, the focus can be on those less frequently accessed data tables. This allows for the identification of potentially high-value data tables, and the completion of the missing business domains for these tables.

[0115] In some possible examples, based on input / output (IO) information recorded in the database logs, multiple candidate data tables are filtered from the database to identify those with usage frequencies below a frequency threshold and / or whose last usage time is before a preset time. From these candidate data tables, the data table representing the missing business area is determined. Usage frequency refers to how frequently the data table is used, including access (i.e., lookups) by database administrators or internet users, and modification operations such as adding, deleting, and modifying data by database administrators. The last usage time can be the last time a database administrator or internet user accessed the data table, or the last time a database administrator modified the data table. The preset time can be a precise point in time (e.g., October 1, 2020) or a period of time prior to the current time (e.g., three months prior to the current time). The current time refers to the time when the missing business area of ​​the data table was determined.

[0116] For example, if the frequency threshold is set to 10, and filtering is based on logs, determining that table H's usage frequency is 3, we can conclude that table H has a low usage frequency and identify it as a candidate table. As another example, if the preset timeframe is 3 months prior to the current time, and table G was last accessed 4 months ago, exceeding the preset 3-month timeframe, we know that table G has not been used recently and should be included as a candidate table. After identifying the candidate tables, SQL queries can be used to determine the tables for missing business domains.

[0117] As can be seen, the embodiments of this application identify the missing business domain data table from the less frequently used data table, thereby greatly reducing the number of data tables that need to be traversed and queryed, and improving query efficiency.

[0118] In some embodiments, when the database is a distributed database, the non-overlapping data tables in the corresponding nodes of the distributed database are determined, that is, the data tables in each master node are determined. Here, the master node can be a server storing the data tables. Then, the data tables stored in each master node are traversed to determine the data tables for missing business domains in each master node. These data tables are then aggregated, and the characteristic information of each data table in the aggregated data table is obtained. It is evident that when the data is a distributed database, not only can the storage pressure on the server be reduced, but also the stored non-overlapping data tables can be queried simultaneously on different nodes to determine the data tables for missing business domains, saving query time and improving query efficiency.

[0119] In step 102, the feature vector corresponding to the feature information is obtained, and the feature vector is mapped to the probability of multiple candidate business domains.

[0120] In some embodiments, obtaining the feature vector corresponding to the feature information can be achieved using methods such as... Figure 3B Steps 1021 to 1024 shown are implemented.

[0121] In step 1021, the index value corresponding to each word in the feature information is determined.

[0122] In some embodiments, such as Figure 6 As shown, Figure 6 This is a schematic diagram of the text classification model provided in this application embodiment. The text classification model includes an input layer, an embedding layer, a fully connected layer, and a softmax layer. In the input layer of the text classification model, the table names and descriptions of the data tables, which are part of the feature information, can be segmented into multiple words. For example, in the data table corresponding to the question-answering client, the table name of one data table lacking a business domain is "E-commerce Question Table," and the table description is "The Development History of E-commerce." In this case, the feature information in the input layer is as follows: Figure 6 The example "e-commerce...history" shows multiple words obtained after feature information processing, such as "electronic" and "business". Then, the index value corresponding to each word is determined in an index table, which includes different words and their corresponding index values. When the feature information is in Chinese, the index table stores each Chinese character / word and its corresponding index value; for example, the index value corresponding to "electronic" is 1100, and the index value corresponding to "business" is 0100.

[0123] In some possible examples, the feature information may also include English words. In this case, the corresponding English index table can be consulted, which stores the index value corresponding to each letter. For example, in the index table, the letter 'a' corresponds to 0 (its corresponding binary code is 0...00, a total of 27 bits), 'b' corresponds to 1 (its corresponding binary code is 0...01, a total of 27 bits), ... 'z' corresponds to 25, and the space corresponds to 26. For any word, the binary code corresponding to each letter can be determined by consulting the index table. By concatenating the binary codes corresponding to each letter, the index value corresponding to the word can be determined.

[0124] In step 1022, the word vector corresponding to each word is obtained by converting based on the index value.

[0125] In some embodiments, the index value corresponding to each word is converted into a one-hot vector in the embedding layer of the text classification model, thus obtaining the corresponding one-hot vector. Then, the one-hot vector corresponding to each word is multiplied by the weight matrix of the embedding layer to obtain the word vector corresponding to each word. In this one-hot vector, only one element is 1, and the rest are all 0. The embedding layer is used to extract feature vectors, that is, to extract the word vectors corresponding to words.

[0126] In step 1023, the word vectors corresponding to each word are summed and averaged to obtain the average word vector.

[0127] For example, if the word vector corresponding to "electronic" is [0, 0, 1] and the word vector corresponding to "business" is [1, 0, 1], the word vectors corresponding to these two words are summed and averaged to obtain the average word vector [0.5, 0, 1].

[0128] In step 1024, the average word vector is used as the feature vector corresponding to the feature information.

[0129] In some embodiments, mapping a feature vector to the probabilities of multiple candidate business domains can be achieved as follows: The feature vector is encoded to obtain the encoding result; the encoded result is then activated using a softmax function to obtain the probabilities corresponding to the multiple candidate business domains. The encoding process includes downsampling, convolution, and linear transformation. Encoding the feature vector involves sequentially performing downsampling, convolution, and linear transformation. The linear transformation can be implemented using fully connected layers in a text classification model.

[0130] In step 103, the business domain corresponding to the probability that satisfies the probability value condition is determined as the business domain to which the data table belongs.

[0131] In some embodiments, the probability selection criteria may include: maximum probability; exceeding a probability threshold. Accordingly, the business domain with the highest probability can be designated as the business domain to which the data table belongs; or one or more business domains corresponding to probabilities exceeding the probability threshold can be designated as the business domain to which the data table belongs. For example, when multiple business domains (such as e-commerce, online entertainment, and online content) corresponding to probabilities exceeding the probability threshold are designated as the business domain to which the data table belongs, the business domains can be stored in the form of "e-commerce-online entertainment-online content".

[0132] The above conditions can be used in combination. For example, in some possible cases, when the probability value condition is the highest probability, after determining the business area with the highest probability, it can be further determined whether it is greater than the probability threshold. If it is greater, it is taken as the business area to which the data table belongs, and then it is written into the data table; if it is not greater, it means that the determined business area is inaccurate, and it is abandoned to write it into the data table.

[0133] As can be seen, the embodiments of this application can not only efficiently determine the business domain to which a data table belongs, but also, when multiple business domains are determined based on the characteristic information of the data table, they can all be included as the business domain to which the data table belongs. Thus, when performing tasks such as retrieval, searches can be conducted based on business domains, expanding the search scope and covering as many possible data tables as possible to reduce omissions in the search results.

[0134] In step 104, the business domain to which the data table belongs is written into the data table of the database.

[0135] For example, if the above text is determined Figure 5A If the business domain of the data table in the image is "e-commerce", then it can be used as follows: Figure 5B As shown, add a column to the data table to write the business domain into the data table.

[0136] In some embodiments, a human-computer interaction confirmation can be triggered on the management side before writing the business domain to the data table. The server sends a write confirmation request to the terminal, which displays the table name, table description, and the confirmed business domain, along with "Confirm" and "Reject" buttons. If the database administrator clicks the "Confirm" button, the business domain will be written to the data table; if the database administrator clicks the "Reject" button, the business domain will not be written to the data table. Therefore, introducing a manual confirmation step before writing the business domain to the data table can further improve the accuracy of the business domain.

[0137] In some possible examples, considering that writing data to a table for every missing business domain would increase the workload of the database administrator and be inefficient, database administrator confirmation can be set to be required only in the following two situations.

[0138] The first scenario involves data tables with missing business domains, either the first or initial (e.g., 10 or 20). After identifying the corresponding business domains, confirmation from the database administrator is required before these domains can be written into the data tables. In this way, these data tables with missing business domains can be used as samples for machine learning models to learn from, thereby training the machine learning model to predict the business domain to which the data tables with missing business domains belong.

[0139] The second approach involves confirming only tables located at specific positions in the database (e.g., using table numbers to represent table locations). These specific positions could be areas with a high density of tables lacking a business domain (e.g., where, on average, one in five tables lacks a business domain), or areas with a high density of tables with above-average access volume. This allows for focused confirmation of tables prone to missing business domains and frequently used tables, improving confirmation efficiency. Furthermore, the confirmed tables can be used as samples for machine learning models, enabling incremental training and improving predictive accuracy.

[0140] In some embodiments, the business domain is determined by a machine learning model. The machine learning model can be the text classification model mentioned above, or a fully connected network model such as FastText, TextCNN, BERT, RoBERTa, or ELECTRA, or an attention network model, a recurrent neural network model, or a convolutional neural network model.

[0141] As an example, the training process of the above machine learning model is as follows: The machine learning model acquires feature information of the data table samples; based on this feature information, the prediction business domain to which the data table samples belong is determined; the error between the prediction business domain and the labeled business domain of the data table samples is determined; the error is backpropagated in the machine learning model to update its parameters. During training, the parameters (such as weights and biases) in the machine learning model can be updated layer by layer using gradient descent. Gradient descent methods can include full gradient descent, stochastic gradient descent, mini-batch gradient descent, and momentum gradient descent, etc.

[0142] In some possible examples, the machine learning model is initially trained on a general training set. During the manual verification process, both correct and incorrect data tables in the business domain are obtained. Data from the incorrect data tables are used as negative samples, and data from the correct data tables are used as positive samples, forming a dedicated training set. This dedicated training set is then used to incrementally train the machine learning model periodically or in real-time until the error rate in subsequent manual verification stages falls below a threshold or the number of errors within a window time falls below a preset number. This improves the accuracy of the machine learning model's predictions. Real-time incremental training means training the model after each manual verification. If the number of samples is insufficient during incremental training (real-time or periodic training) (i.e., below the minimum number of samples per training round), the dedicated training set can be supplemented by samples collected from the database and / or samples from the general training set. As the number of samples collected during the manual verification process increases, the number of samples used in the general training set can be gradually reduced.

[0143] In some embodiments, the data processing system 10 supports a write rollback function for business domains. That is, after the server automatically writes a business domain into the data table, if the database administrator (such as an operations and maintenance personnel) finds that the business domain is unreasonable, the write can be revoked and an updated business domain can be written, thereby ensuring the traceability of the data table.

[0144] In some embodiments, the database tables are periodically traversed to identify tables where data changes have occurred. The types of data changes include at least one of the following: adding data, deleting data, and modifying data. Feature information of the tables where data changes have occurred is obtained, and a new business domain is determined based on this feature information. The business domain to which the tables where data changes have occurred is replaced with the new business domain. The periodic traversal can be performed once every month or once every week; this embodiment does not impose any limitation on this.

[0145] As can be seen, when a user accesses a data table and modifies it, causing changes to the data in the data table, the embodiments of this application can promptly adjust the business domain to which the data table belongs based on the characteristic information of the changes in the data table, ensuring the accuracy of the business domain.

[0146] In some embodiments, considering the large number of tables in the database, determining the business domain for each table with a missing business domain could be extremely time-consuming. Therefore, it is possible to first identify multiple similar tables with missing business domains in the database. For these similar tables, the business domains do not need to be calculated repeatedly; instead, the business domain of one table in the similar tables can be used as the business domain for the other similar tables. The implementation process is as follows: Identify multiple groups of tables with similarity exceeding a similarity threshold among the tables with missing business domains. Each group of tables includes at least two tables. Randomly select one table from each group as a representative table. Obtain the feature information of each representative table and determine its business domain based on the feature information. Use the business domain of the representative table as the business domain for the other tables in the same group.

[0147] As can be seen, for multiple similar data tables with missing business domains, the data processing method of the database provided in this application can be used to determine the business domain to which one of the data tables belongs, and this business domain can be used as the business domain to which other similar data tables belong, thereby greatly improving the efficiency of business domain completion.

[0148] In some embodiments, after determining the business domain to which a data table in the database belongs, a search can be performed based on information such as the business domain of the data table. The search process is described below. In response to a search request carrying keywords, the server filters data tables in the database based on the business domain to which they belong, obtaining data tables within a first range; filters data tables within the first range based on their table description information, obtaining data tables within a second range; and filters data tables within the second range based on their table names, returning the resulting data tables within the target range as search results.

[0149] In some possible examples, during the process of filtering data tables layer by layer, if the number of data tables within the obtained range is less than or equal to the quantity threshold, the filtering stops, and the data tables within that range are returned as search results. For example, if the quantity threshold is 50, and the number of data tables in the first range is 45, the data tables in the first range are returned to the terminal as search results.

[0150] As can be seen, after determining the business domain corresponding to the data table, the retrieval strategy can be optimized based on the business domain to improve the accuracy of the retrieval.

[0151] As can be seen, the embodiments of this application predict the business domain of a data table by providing feature information of the data table with missing business domains, and automatically fill in the business domain in the data table, thereby realizing automatic completion of the business domain of the data table. For businesses that need to maintain massive data tables, this significantly saves database maintenance costs and improves data management efficiency.

[0152] Below, we assume that an internet user's terminal is running a quiz client. The user can participate in interactive activities such as answering or creating questions in various business areas through this client, and the user can choose the business area for answering / creating questions. For example, when a user selects to answer questions related to the e-commerce business area, the terminal responds to the user's request by sending a quiz request to the backend server of the internet application to retrieve e-commerce questions. This will be combined with... Figure 7 The steps in the interactive flowchart shown are an exemplary application in a question-answering scenario of an internet application.

[0153] In step 201, the terminal responds to the e-commerce-related question-answering request by sending a question-answering request to the backend server.

[0154] The question request includes the business domain of the question that needs to be answered; here, we take the e-commerce domain as an example.

[0155] In step 202, the backend server responds to the question-answering request by sending a query request to the database server.

[0156] The query request is used to retrieve questions in the e-commerce domain from the database of the database server.

[0157] In step 203, the database server traverses the database to obtain data tables whose business domain is e-commerce, excluding data tables with missing business domains.

[0158] In some embodiments, the database corresponding to the question-answering client contains multiple types of data tables for each business domain, and there can be multiple tables of each type. For example, when the business domain is e-commerce, the corresponding data tables may include a question table, a friend table, and a compatibility value table. The question table stores multiple questions for the e-commerce domain, the friend table stores information about other friends associated with a user (such as usernames), and the compatibility value table indicates the compatibility value between the user and each friend. Because some business domain data tables exist in the database (possibly because the business domain field was omitted during the data table's storage process), the data tables for the e-commerce business domain are first selected from the data tables other than those with missing business domains.

[0159] In step 204, the database server determines the corresponding business domain based on the table name and table description information of the data table for the missing business domain.

[0160] In some embodiments, step 204 can be achieved through... Figure 6 The text classification model shown is an implementation of the text classification model. The text classification model can also be replaced with fully connected network models such as FastText, TextCNN, BERT, RoBERTa, or ELECTRA, and can also be replaced with attention network models, recurrent neural network models, or convolutional neural network models.

[0161] like Figure 8 As shown, Figure 8 This is a schematic diagram illustrating the training and prediction of the text classification model provided in this application embodiment. During the training of the text classification model, the table name and description information of the data table are used as sample data, and the business domain of the data table is used as the label (i.e., the business domain is labeled) to train the model. The trained model parameters are then saved. The text classification model can be trained using stochastic gradient descent to achieve optimal or locally optimal parameters. In predicting the business domain to which the data table belongs, the model parameters are first loaded, and then the table name and description information are used as input to the trained text classification model. The text classification model then outputs the business domain to which the data table belongs.

[0162] The following section will explain in detail the process of predicting the business domain of the data table, taking into account the structure of the text classification model.

[0163] The text classification model consists of an input layer, an embedding layer, a fully connected layer, and a softmax layer. In the input layer, table names or descriptions are segmented into words, and their corresponding index values ​​are determined. During word segmentation, sentences in the text are divided into words or characters. In determining the index values, a corresponding index value needs to be found for each segmented word or character. For example, in the input text, the first... The word is After indexing, a unique integer number is obtained. , where I represents the correspondence between integer numbers (i.e., index values) in the index table and words in the text.

[0164] In the embedding layer, the sentence vector corresponding to the text is obtained from the embedding layer using the index value of each word. Assume the matrix of the embedding layer is... Where V is the total number of words, and D is the dimension of each vector. This is to obtain the word vector corresponding to the i-th word in the text. First, the index value of the i-th word is converted into a one-hot vector, resulting in V one-hot vectors. In each one-hot vector, all elements except the element at the i-th position are 0. Then, the one-hot vector matrix corresponding to the i-th word is multiplied by the weight matrix E corresponding to the embedding layer to obtain the word vector corresponding to the i-th word. .

[0165] After determining the word vector corresponding to each word, the word vectors are summed and averaged to obtain the sentence vector corresponding to the text. Assuming the text contains L words, the sentence vector s can be represented as: .

[0166] In the fully connected layer, the sentence vector is transformed using the following formula (1).

[0167] (1)

[0168] Where W is the weight parameter of the fully connected layer, b is the bias parameter of the fully connected layer, and f is the activation function. is the output of the fully connected layer, and 'a' is an A-dimensional vector where A is the total number of candidate business domains.

[0169] In the Softmax layer, the probability of the output data table belonging to a candidate business domain is given. The probability calculation is shown in formula (2).

[0170] (2)

[0171] in, Let i represent the i-th element in a, where i can take values ​​from 0 to A.

[0172] Finally, the business domain with the highest probability is selected as the business domain to which the data table belongs and output. This method can determine the business domain corresponding to all data tables in the database that are missing business domains.

[0173] In step 205, the database server retrieves the data table for the business domain of e-commerce from the data table for the missing business domain.

[0174] After determining the business domain to which each missing business domain data table belongs and writing it into the corresponding data table, you can filter the data tables in the database based on the business domain of each data table to find the data tables whose business domain is e-commerce.

[0175] In step 206, the database server filters out the problem table from the data tables whose business domain is e-commerce, based on the table name and / or table description information.

[0176] Because each business domain corresponds to multiple types of tables, it is also necessary to filter out the data tables of type "problem table" from the data tables of business domain "e-commerce". The problem table can be identified based on the table name (e.g., ***problem table) or the table description information (e.g., the question mark "?" in the table description).

[0177] In step 207, the database server randomly generates multiple questions based on the questions in each question table.

[0178] After retrieving the question table for the e-commerce domain from the entire database, the database server extracts the questions from the question table, summarizes them, and randomly selects multiple (such as a specified number) questions as the data processing results.

[0179] In step 208, the database server sends multiple questions in the field of e-commerce to the terminal.

[0180] The database server sends the data processing results to the terminal, which are multiple questions (and their corresponding answers) in the field of e-commerce.

[0181] In step 209, the terminal displays multiple questions.

[0182] In step 210, the terminal obtains the user's answers to multiple questions and gives the corresponding scores.

[0183] The terminal determines whether the user's selected / entered answer is correct based on the answer to the question, and gives the corresponding score according to the scoring criteria.

[0184] As can be seen, this embodiment of the application determines the business domain to which the data table belongs based on the data table name / description information in the data table. This allows for the further identification of all question tables belonging to a specific business domain (such as e-commerce) in the database, and the return of questions from those question tables to the user. This increases the number of valid questions in the database, providing users with more diverse questions and enhancing their enthusiasm for answering. Furthermore, the business domain is automatically determined through a machine learning model, eliminating the need for human intervention, reducing manpower consumption, and ensuring the integrity of the domain information in the data table.

[0185] The following description continues to illustrate the exemplary structure of the database data processing device 243 provided in the embodiments of this application as a software module. In some embodiments, such as... Figure 2 As shown, the software modules in the data processing device 243 storing the database in the memory 240 may include: a query module 2431, an acquisition module 2432, a determination module 2433, and a writing module 2434.

[0186] The query module 2431 is used to query data tables in the database that lack business domains; the acquisition module 2432 is used to acquire feature information of the data table and to acquire feature vectors corresponding to the feature information, and to map the feature vectors to the probabilities of multiple candidate business domains; the determination module 2433 is used to determine the business domain to which the data table belongs as the probability corresponding to the probability that meets the probability value conditions; and the writing module 2434 is used to write the business domain to which the data table belongs into the data table in the database.

[0187] In some embodiments, the acquisition module 2432 is further configured to determine the index value corresponding to each word in the feature information; perform conversion based on the index value to obtain the word vector corresponding to each word; perform summation and averaging on the word vectors corresponding to each word to obtain the average word vector; and use the average word vector as the feature vector corresponding to the feature information.

[0188] In some embodiments, the acquisition module 2432 is further configured to perform word segmentation on the table name and table description information of the data table included in the feature information to obtain multiple words; and determine the index value corresponding to each word in the index table; wherein the index table includes different words and their corresponding index values.

[0189] In some embodiments, the acquisition module 2432 is further configured to perform conversion processing based on the index value corresponding to each word to obtain the corresponding one-hot vector; and multiply the one-hot vector corresponding to each word with the weight matrix to obtain the word vector corresponding to each word.

[0190] In some embodiments, the acquisition module 2432 is further configured to encode the feature vector to obtain the encoding result; and to perform activation processing on the encoding to obtain the probability of the corresponding multiple candidate business domains.

[0191] In some embodiments, the determining module 2433 is further configured to select the business area with the highest probability as the business area to which the data table belongs; or to sort the business areas corresponding to probabilities exceeding the probability threshold in descending order of probability, and select the top-ranked business areas as the business areas to which the data table belongs.

[0192] In some embodiments, the query module 2431 is further configured to filter multiple candidate data tables from the logs of the database, wherein the candidate data tables meet at least one of the following conditions: the usage frequency is lower than a frequency threshold, and the last usage time is before a preset time; and to determine the data table for the missing business area from the multiple candidate data tables.

[0193] In some embodiments, the query module 2431 is further configured to determine non-overlapping data tables in multiple nodes corresponding to the distributed database; and traverse the non-overlapping data tables stored in each node to determine the data table for the missing business domain.

[0194] In some embodiments, the database processing apparatus further includes a replacement module 2435, which is used to periodically traverse the data tables in the database to determine the data tables that have undergone data changes. The types of data changes include at least one of the following: adding data, deleting data, and modifying data; obtaining feature information of the data tables that have undergone data changes, and determining a new business domain based on the feature information of the data tables that have undergone data changes; and replacing the business domain to which the data tables that have undergone data changes belong with the new business domain.

[0195] In some embodiments, the determining module 2433 is further configured to determine multiple groups of data tables in multiple data tables of missing business domains whose similarity exceeds a similarity threshold, wherein each group of data tables includes at least two data tables; select one data table in each group of data tables as a representative data table, obtain feature information of each representative data table, and determine the business domain to which the representative data table belongs based on the feature information; and use the business domain to which the representative data table belongs as the business domain to which other data tables in the same group of data tables belong.

[0196] In some embodiments, the database processing apparatus further includes a retrieval module 2436, which, in response to a retrieval request, filters data tables within a first range based on the business domain to which the data tables in the database belong; filters data tables within a second range based on the table description information of the data tables within the first range; and filters data tables within a second range based on the table names of the data tables within the second range, returning the data tables within the target range as retrieval results.

[0197] In some embodiments, the business domain is determined by a machine learning model, and the data processing apparatus for the database further includes a training module 2437 for performing the following processes through the machine learning model: obtaining feature information of a data table sample; determining the predicted business domain to which the data table sample belongs based on the feature information of the data table sample; determining the error between the predicted business domain of the data table sample and the labeled business domain of the data table sample; and backpropagating the error in the machine learning model to update the parameters of the machine learning model.

[0198] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the database data processing method described above in this application.

[0199] This application provides a computer-readable storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to perform a database data processing method provided in this application, for example... Figure 3A The data processing method for the database is shown.

[0200] In some embodiments, the storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0201] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0202] As an example, executable instructions may, but do not necessarily, correspond to files in the file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0203] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0204] In summary, this application's embodiments first identify data tables in the database that lack business domains; then, they determine the business domains to which these data tables belong using a text classification model; finally, the business domains are written into the data tables, achieving automatic completion of business domains in the data tables, reducing manpower consumption, and improving data management efficiency. For multiple similar data tables with missing business domains, the database data processing method provided in this application's embodiments can determine the business domain of one of the data tables and use that business domain as the business domain of other similar data tables, thereby greatly improving the efficiency of business domain completion. When a data table changes, the business domain to which the data table belongs can be adjusted in a timely manner based on the changing feature information, ensuring the accuracy of the business domain. Furthermore, after determining the business domain corresponding to the data table, the retrieval strategy can be optimized based on the business domain to improve the accuracy of the retrieval.

[0205] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A data processing method for a database, characterized in that, include: Query the database for missing business domain data tables and obtain the characteristic information of the data tables; Determine the index value corresponding to each word in the feature information; Based on the index value, a word vector corresponding to each word is obtained by conversion; The word vectors corresponding to each word are summed and averaged to obtain the average word vector. The average word vector is used as the feature vector corresponding to the feature information, and the feature vector is mapped to the probability of multiple candidate business domains. The business domain corresponding to the probability that satisfies the probability value condition is determined as the business domain to which the data table belongs; In the data table of the database, the business domain to which the data table belongs is written.

2. The method according to claim 1, characterized in that, Determining the index value corresponding to each word in the feature information includes: The table name and table description information of the data table included in the feature information are processed by word segmentation to obtain multiple words; Determine the index value of each word in the index table; The index table includes different words and their corresponding index values.

3. The method according to claim 1, characterized in that, The conversion based on the index value to obtain the word vector corresponding to each word includes: The corresponding one-hot vector is obtained by performing a transformation based on the index value of each word. Multiply the one-hot vector corresponding to each word by the weight matrix to obtain the word vector corresponding to each word.

4. The method according to claim 1, characterized in that, The step of mapping the feature vector to the probabilities corresponding to multiple candidate business domains includes: The feature vector is encoded to obtain the encoding result; The encoding process is activated to obtain the probabilities of multiple candidate business domains.

5. The method according to claim 1, characterized in that, The step of determining the business domain corresponding to the probability that satisfies the probability value condition as the business domain to which the data table belongs includes: The business domain with the highest probability is taken as the business domain to which the data table belongs; or The business areas corresponding to probabilities exceeding the probability threshold are sorted in descending order of probability, and the top-ranked business areas are selected as the business areas to which the data table belongs.

6. The method according to claim 1, characterized in that, The query database includes tables containing missing business domain data, including: Multiple candidate data tables are selected from the logs of the database, and the candidate data tables meet at least one of the following conditions: the usage frequency is lower than the frequency threshold, and the last usage time is before a preset time. The missing business domain data table is determined from the multiple candidate data tables.

7. The method according to claim 1, characterized in that, When the database is a distributed database, the query for missing data tables in the business domain includes: Identify non-overlapping data tables among the multiple nodes corresponding to the distributed database; Iterate through the non-overlapping data tables stored in each node to determine the data table for the missing business domain.

8. The method according to claim 1, characterized in that, The method further includes: The database tables are periodically traversed to identify those tables where data changes have occurred, and the types of data changes include at least one of the following: adding data, deleting data, or modifying data. Obtain the feature information of the data table that has undergone data changes, and determine new business areas based on the feature information of the data table that has undergone data changes; Replace the business domain of the data table whose data has changed with the new business domain.

9. The method according to claim 1, characterized in that, After querying the database for missing data tables related to the business domain, the method further includes: Identify multiple sets of data tables in multiple data tables with similarity exceeding a similarity threshold in the missing business domain, wherein each set of data tables includes at least two data tables; Select one data table from each group of data tables as a representative data table, obtain the feature information of each representative data table, and determine the business domain to which the representative data table belongs based on the feature information; The business domain to which the representative data table belongs is taken as the business domain to which other data tables in the same group belong.

10. The method according to claim 1, characterized in that, After writing the business domain to which the data table belongs in the data table of the database, the method further includes: In response to a search request, the database is filtered based on the business domain to which the data tables belong, to obtain data tables within a first range; Based on the table description information of the data tables in the first range, the data tables in the second range are obtained by filtering. Based on the table names within the second range, the data tables within the target range are filtered and returned as search results.

11. The method according to claim 1, characterized in that, The business domain is determined through a machine learning model. Before querying the database for a data table lacking the business domain, the method further includes: The machine learning model performs the following processing: obtaining feature information of data table samples, and determining the prediction business domain to which the data table samples belong based on the feature information of the data table samples; The error in determining the predicted business domain based on the data table sample and the labeled business domain based on the data table sample; The error is backpropagated in the machine learning model to update the parameters of the machine learning model.

12. A data processing apparatus for a database, characterized in that, include: The query module is used to query data tables in the database that lack data related to specific business areas. The acquisition module is used to acquire feature information of the data table and to determine the index value corresponding to each word in the feature information; Based on the index value, a word vector corresponding to each word is obtained; the word vectors corresponding to each word are summed and averaged to obtain an average word vector; the average word vector is used as the feature vector corresponding to the feature information, and the feature vector is mapped to the probability of multiple candidate business domains; The determination module is used to determine the business domain corresponding to the probability that satisfies the probability value condition as the business domain to which the data table belongs; The write module is used to write the business domain to which the data table belongs in the data table of the database.

13. The apparatus as claimed in claim 12, characterized in that, The acquisition module is also used for: The table name and table description information of the data table included in the feature information are processed by word segmentation to obtain multiple words; the index value corresponding to each word in the index table is determined; wherein, the index table includes different words and their corresponding index values.

14. The apparatus as claimed in claim 12, characterized in that, The acquisition module is also used for: The index value corresponding to each word is transformed to obtain the corresponding one-hot vector; the one-hot vector corresponding to each word is multiplied by the weight matrix to obtain the word vector corresponding to each word.

15. The apparatus as claimed in claim 12, characterized in that, The acquisition module is also used for: The feature vector is encoded to obtain the encoding result; the encoding is then activated to obtain the probabilities of multiple candidate business domains.

16. The apparatus as claimed in claim 12, characterized in that, The determining module is further configured to: The business domain with the highest probability is taken as the business domain to which the data table belongs; or the business domains corresponding to probabilities exceeding the probability threshold are sorted in descending order of probability, and the top-ranked business domains are selected as the business domain to which the data table belongs.

17. The apparatus as claimed in claim 12, characterized in that, The query module is also used for: Multiple candidate data tables are selected from the logs of the database, and the candidate data tables meet at least one of the following conditions: the usage frequency is lower than the frequency threshold, and the last usage time is before a preset time; from the multiple candidate data tables, a data table for the missing business area is determined.

18. An electronic device, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the data processing method of the database according to any one of claims 1 to 11.

19. A computer-readable storage medium, characterized in that, It stores executable instructions for causing a processor to perform a data processing method for a database as described in any one of claims 1 to 11.

20. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the data processing method for the database according to any one of claims 1 to 11.

Images