Data table classification method, device, medium and electronic equipment

Abstract

The present application provides a data table classification method, device, medium, and electronic device, relating to the field of data table classification, including: obtaining each operation statement of the data table to be classified within a target time window to obtain a first feature vector YT; obtaining each key foreign key information of the data table to be classified to obtain a second feature vector ET; obtaining content feature information of the data table to be classified to obtain a third feature vector; inputting the third feature vector into the weight distribution layer of the data table classification model to obtain a first weight corresponding to YT and a second weight corresponding to ET; obtaining a target feature vector based on YT, the first weight, ET, and the second weight; and inputting the target feature vector into the classification layer of the data table classification model to obtain a classification result corresponding to the data table to be classified. The present application has stronger portability and adaptability, and the classification results obtained are more accurate and objective.

Description

Technical Field

[0001] The present application relates to the technical field of data table classification, and in particular to a data table classification method, device, medium and electronic equipment. Background Art

[0002] In the era of big data, databases store a vast number of complex data tables. Efficiently and accurately classifying these tables is crucial for improving data management and retrieval efficiency. Currently, the industry generally uses a method based on standardized file names, storage paths, and other criteria to classify database tables. This method enables fast and accurate classification of tables that conform to standard naming conventions, significantly improving data management efficiency.

[0003] However, with the increasing diversification of data sources and data generation methods, a large number of data tables with non-standard table and field names are prevalent in databases. Due to the lack of unified and standardized naming rules, these data tables are difficult to effectively classify using existing classification methods based on standard naming or paths. This leads to inefficient data management and significantly increases the difficulty of data retrieval, significantly affecting the overall effectiveness of data processing and analysis. Research on new data table classification technologies is urgently needed to address this problem. Summary of the Invention

[0004] In response to the above technical problems, the present application provides a data table classification method, device, medium and electronic device, which at least partially solve the problems existing in the prior art.

[0005] In a first aspect of the present application, a data table classification method is provided, the method comprising:

[0006] Obtain each operation statement of the data table to be classified within the target time window and input it into the text encoding layer of the data table classification model to obtain the first feature vector YT; where YT represents the chronological operation features of the data table to be classified within the target time window; the end time of the target time window is the current time;

[0007] Obtain each key foreign key information of the data table to be classified, and obtain a second feature vector ET based on the graph feature extraction layer of the data table classification model; wherein ET represents the data reference feature of the data table to be classified; the key foreign key information is the foreign key information between the data table to be classified and the key data table; the key data table and the data table to be classified are in the same database; and the key data table can determine the data table category according to the preset classification rules; while the data table to be classified cannot determine the data table category according to the preset classification rules;

[0008] Obtaining content feature information of the data table to be classified, and obtaining a third feature vector based on the graph feature extraction layer of the data table classification model; wherein the third feature vector represents the data content feature of the data table to be classified;

[0009] Input the third eigenvector into the weight distribution layer of the data table classification model to obtain the first weight corresponding to YT and the second weight corresponding to ET;

[0010] According to YT, the first weight, ET, and the second weight, the target feature vector is obtained;

[0011] The target feature vector is input into the classification layer of the data table classification model to obtain the classification result corresponding to the data table to be classified.

[0012] In a second aspect of the present application, a data table classification device is provided, the device comprising:

[0013] A first vector acquisition unit is configured to acquire each operation statement of the data table to be classified within a target time window and input the operation statement into a text encoding layer of the data table classification model to obtain a first feature vector YT; wherein YT represents the operation characteristics of the data table to be classified in chronological order within the target time window; the end time of the target time window is the current time;

[0014] A second vector acquisition unit is configured to acquire each key foreign key information of the data table to be classified, and obtain a second feature vector ET based on a graph feature extraction layer of the data table classification model; wherein ET represents a data reference feature of the data table to be classified; the key foreign key information is the foreign key information between the data table to be classified and the key data table; the key data table and the data table to be classified are in the same database; and the key data table can determine the data table category according to a preset classification rule; while the data table to be classified cannot determine the data table category according to the preset classification rule;

[0015] a third vector acquisition unit, configured to acquire content feature information of the data table to be classified, and obtain a third feature vector based on the graph feature extraction layer of the data table classification model; wherein the third feature vector represents the data content feature of the data table to be classified;

[0016] a weight acquisition unit, configured to input the third feature vector into a weight allocation layer of a data table classification model to obtain a first weight corresponding to YT and a second weight corresponding to ET;

[0017] A target vector acquisition unit, configured to obtain a target feature vector according to YT, a first weight, ET, and a second weight;

[0018] The classification unit is used to input the target feature vector into the classification layer of the data table classification model to obtain the classification result corresponding to the data table to be classified.

[0019] In a third aspect of the present application, a non-transitory computer-readable storage medium is provided, in which at least one instruction or at least one program is stored, and the at least one instruction or at least one program is loaded and executed by a processor to implement the aforementioned data table classification method.

[0020] In a fourth aspect of the present application, an electronic device is provided, comprising a processor and the above-mentioned non-transitory computer-readable storage medium.

[0021] This application has at least the following beneficial effects:

[0022] The data table classification method provided by the present application first obtains each operation statement of the data table to be classified within the target time window, and inputs it into the text encoding layer of the data table classification model to obtain a first feature vector; here, the operation features of the data table to be classified in chronological order within the target time window are obtained; secondly, each key foreign key information of the data table to be classified is obtained, and according to the graph feature extraction layer of the data table classification model, a second feature vector is obtained; here, the reference relationship features between the data table to be classified and other key data tables with clear classification are obtained; then, the content feature information of the data table to be classified is obtained, and according to the graph feature extraction layer of the data table classification model, a third feature vector is obtained; here, the third feature vector represents the features of the internal data of the data table to be classified; then, the first feature vector is obtained according to the third feature vector. The first weight corresponding to the quantity, and the second weight corresponding to the second eigenvector are obtained; here, since the first eigenvector and the second eigenvector are the influence of different external factors of the data table to be classified on the data table to be classified; and the third eigenvector analyzes the data features inside the data table to be classified, if the third eigenvector represents the more chaotic and non-standard data, then at this time, the first eigenvector is more important for determining the final category, and the corresponding first weight is larger; on the contrary, if the third eigenvector represents the data more standardly, then the possibility of the data being classified as non-original data is greater, at this time, the second eigenvector is more important for determining the final category, and the corresponding second weight is larger; thus, after assigning weights to the first eigenvector and the second eigenvector, the final target eigenvector is obtained, and the final classification result is obtained. The present application not only takes into account the connection between the data table to be classified and the external factors, including the data reference relationship characteristics and the operation characteristics of the data table to be classified, but also assigns corresponding weights to the data reference relationship characteristics and the operation characteristics of the data table to be classified based on the characteristics of the internal data of the data table to be classified, so that different target eigenvectors are constructed for data tables with different internal data characteristics; it has stronger migration and adaptability, and the classification results obtained are more accurate and more objective. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0024] Figure 1 A flow chart of the data table classification method provided in an embodiment of the present application;

[0025] Figure 2 This is a structural block diagram of the data table classification device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0028] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on this application, it should be understood by those skilled in the art that an aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement an apparatus and / or practice a method. In addition, other structures and / or functionalities other than one or more of the aspects described herein can be used to implement this apparatus and / or practice this method.

[0029] Please refer to Figure 1As shown, an embodiment of the present application provides a data table classification method, the method comprising:

[0030] Step S100, obtain each operation statement of the data table to be classified within the target time window, and input it into the text encoding layer of the data table classification model to obtain a first feature vector YT; wherein YT represents the operation characteristics of the data table to be classified in chronological order within the target time window; the end time of the target time window is the current time.

[0031] Specifically, the target time window can be of any length. Preferably, it can be the time from the entry of the data table to be classified to the current time. The longer the target time window, the richer the operational features that can be obtained. Operational features can be extracted from SQL statements, where operations can include queries, processing, or calculations. Because the data in different data tables may differ, the operations performed on different data tables may also differ. The operational features within the target time window are sorted chronologically to obtain a first feature vector with time series characteristics.

[0032] The text encoding layer of the data table classification model is used to encode the obtained operation statements and convert the operation statements into the final first feature vector with time series characteristics.

[0033] Step S200, obtain each key foreign key information of the data table to be classified, and obtain a second feature vector ET based on the graph feature extraction layer of the data table classification model; wherein ET represents the data reference feature of the data table to be classified; the key foreign key information is the foreign key information between the data table to be classified and the key data table; the key data table and the data table to be classified are in the same database; and the key data table can determine the data table category according to the preset classification rules; the data table to be classified cannot determine the data table category according to the preset classification rules.

[0034] Specifically, a foreign key is a matching relationship between a field value in a data table and a field value in another data table. In this embodiment, each key foreign key information of the data table to be classified represents information that the data table to be classified references other key data tables, or information that the data table to be classified is referenced by other key data tables. Here, a key data table is a data table whose data table category is determined according to a preset classification rule and is located in the same database as the data table to be classified. It should be noted that the preset classification rule can be a method for classifying standard-named data tables based on standard information such as standard data table names and / or storage paths. That is, the preset classification rule cannot classify data tables with non-standard names such as the data table to be classified.

[0035] Step S300 , obtaining content feature information of the data table to be classified, and obtaining a third feature vector based on the graph feature extraction layer of the data table classification model; wherein the third feature vector represents the data content feature of the data table to be classified.

[0036] Specifically, content feature information can include the ratio of unique field values, fluctuations in field values ​​within each field, the total number of fields, and the correlation matrix between fields. The features of the correlation matrix are extracted from the graph coding layer and combined with other feature information to generate a third feature vector.

[0037] Step S400: Input the third eigenvector into the weight distribution layer of the data table classification model to obtain a first weight corresponding to YT and a second weight corresponding to ET.

[0038] Specifically, the weight distribution layer of the data table classification model is used to obtain the first weight and the second weight based on the input third eigenvector. In one embodiment, if the third eigenvector represents the data more chaotically and non-standardly, then at this time, the possibility that the data is classified as original data is greater, and the first eigenvector is relatively more important for determining the final category, so the corresponding first weight is larger; conversely, if the third eigenvector represents the data more standardly, then the possibility that the data is classified as non-original data is greater, and at this time, the second eigenvector may be relatively more important for determining the final category, so the corresponding second weight is larger.

[0039] Step S500: Obtain a target feature vector according to YT, the first weight, ET, and the second weight.

[0040] Step S600: input the target feature vector into the classification layer of the data table classification model to obtain the classification result corresponding to the data table to be classified.

[0041] Thus, after assigning weights to the first eigenvector and the second eigenvector, the final target eigenvector is obtained, and the final classification result is acquired.

[0042] Here, the classification result can be any one of original data, cleaned data, aggregated data, and application indicator data.

[0043] Raw data is ODS data, including fields directly synchronized from the business system (such as user_id_raw and order_date_source). Typical SQL operations for this data include full insert, simple cleansing, and simple data addition. It has low complexity and no cross-table joins. Cleaned data is DWD data (such as user_id_cleaned and order_date_utc). Typical SQL operations for this data include field type conversion, data desensitization, and lightweight deduplication. It has medium complexity, multi-table joins, and clear and standardized steps. Aggregated and summarized data is DWS data (such as daily_sales_total and monthly_active_users). Typical SQL operations for this data include aggregation, window functions, and cross-fact table join calculations. It features key indicator pre-calculation, time dimension aggregation, and serving multiple business lines. Application indicator data is ADS data (such as retention_rate_7d and conversion_funnel). Typical SQL operations for this data include complex business formulas, data binning, and directly exporting results to reports or API interfaces. It has the characteristics of business term naming, very few underlying associations and final indicator output.

[0044] It's understandable that the above data table categories are processed layer by layer. That is, raw data can be processed to obtain cleaned data, which can be further processed to obtain aggregated data, and further processed to obtain application indicator data. However, these four types of data can exist simultaneously in the same database.

[0045] This application not only takes into account the connection between the data table to be classified and external factors, including data reference relationship characteristics and operation characteristics of the data table to be classified, but also assigns corresponding weights to the data reference relationship characteristics and the operation characteristics of the data table to be classified based on the characteristics of the internal data of the data table to be classified, so that data tables with different internal data characteristics construct different target feature vectors; it has stronger portability and adaptability, and the classification results obtained are more accurate and objective.

[0046] In an exemplary embodiment of the present application, step S200 includes:

[0047] Step S210 , obtaining each operation statement of the data table to be classified within the target time window to obtain an operation statement list; wherein each operation statement has a corresponding execution time.

[0048] Step S220 , input the operation statement list into the text encoding layer of the data table classification model, and extract the key operation features in each operation statement; wherein the key operation features include: operation keyword features, field name features and conditional expression features.

[0049] Specifically, operation keyword features include SELECT, INSERT, UPDATE, and DELETE. These keywords are used as features. A keyword dictionary can be created, with each keyword corresponding to a unique index. Field name features include table names and field names. A dictionary is also created for these, and unique indexes are assigned. Conditional expression features include conditions in the WHERE clause. The condition type (e.g., equal to, greater than, less than) and the columns involved can be used as features.

[0050] Step S230: construct a first feature vector YT=(YT1, YT2, ..., YT i ,...,YT n ); i = 1, 2, ..., n; where n is the number of operation statements; YT i YT is the list of eigenvalues ​​corresponding to the operation statement at position i in the order of execution time; i =(YT i，1 , YT i，2 ,...,YT i，j ,...,YT i，m , G i ); j = 1, 2, ..., m; m is the total number of key operating features; YT i，j is the characteristic value of the jth key operation feature of the operation statement ranked in the order of execution time; G i is the normalized time corresponding to the operation statement at position i in the order of execution time; G i Meet the following characteristics: G i =(T i -T min ) / (T min -T max ); where T i is the execution time of the operation statement at position i in the order of execution time; T min T is the earliest execution time among the execution times corresponding to all operation statements in the operation statement list; max The latest execution time among the execution times corresponding to all operation statements in the operation statement list.

[0051] Specifically, a feature bit is reserved for the normalized time to indicate the execution order of the SQL statement, namely G i .

[0052] For other operations, a flag bit is assigned to each keyword. If the operation statement contains a keyword, the corresponding flag bit is set to 1; otherwise, it is set to 0. Similarly, flag bits are assigned to each table name and field name based on all table names and field names. If the operation statement involves a table name or column name, the corresponding flag bit is set to 1; otherwise, it is set to 0. The same applies to conditional expressions. If the operation statement involves a conditional expression, the corresponding flag bit is set to 1; otherwise, it is set to 0.

[0053] The feature vectors of all SQL statements are sorted according to the normalized time values, ensuring they are arranged in chronological order. The sorted feature vectors are concatenated to form a long feature vector, which represents the timing information for all operation statements. Time series encoding clearly displays the execution order of each operation statement. By analyzing these chronologically ordered features, we can clearly understand the entire data processing flow, identifying which operations were executed first and which were executed later, and helping us understand how data is processed and transformed step by step. It can also help identify dependencies between operation statements. If the execution result of one statement serves as the input of another, this order and dependency will be reflected in the time series features.

[0054] In an exemplary embodiment of the present application, step S200 includes:

[0055] Step S210: Obtain each data table in the database where the data table to be classified is located except the data table to be classified, so as to obtain a data table identification list SB = (SB1, SB2, ..., SB x ,...,SB y ); x=1, 2, ..., y; where y is the number of data tables in the database where the data table to be classified is located, excluding the data table to be classified.

[0056] Step S220: classify each data table in SB according to the preset classification rules to obtain a classified data table identifier list SBY = (SBY1, SBY2, ..., SBY a ,...,SBY b ); a=1, 2, ..., b; where b is the number of classified data tables; SBY a is the ath classified data table; b≤y.

[0057] Specifically, the preset classification rule can be a method for classifying standard-named data tables based on standard information such as standard data table names and / or storage paths. That is, the preset classification rule cannot classify data tables with non-standard names, such as the data table to be classified. In other words, a classified data table is also a data table within the database containing the data table to be classified whose data table type can be determined using the preset classification rule.

[0058] Step S230, traverse SBY according to each key foreign key information of the data table to be classified to obtain a key data table identifier list GSBY = (GSBY1, GSBY2, ..., GSBY c ,...,GSBY d ); c = 1, 2, ..., d; where d is the number of key data tables; GSBY c is the cth key data table; d≤b; the key data table is a classified data table that has at least one reference relationship with the data table to be classified.

[0059] Specifically, d key data tables that have a reference or referenced relationship with the data table to be classified are obtained in SBY.

[0060] Step S240, obtaining a directed graph based on each key foreign key information and GSBY of the data table to be classified; wherein the directed graph contains nodes and directed edges; the nodes include the data table node to be classified and the key data table node; the directed edges represent the reference relationship between the data table node to be classified and each key data table node; the directed edges are unidirectional arrows; the arrows point to the referencing party; each of the nodes also contains the type information of the corresponding data table to be classified or the key data table.

[0061] Specifically, the directed graph radiates outward with the node of the data table to be classified as the center, and each directed edge represents the reference or referenced relationship between each connected key data table node and the data table to be classified. Furthermore, the directed edge is a unidirectional arrow; the arrow points to the referencing party; as an example: the arrow points to the data table to be classified and back to the key data table A, and the reference relationship between the data table to be classified and A is: a field value in the data table to be classified references a field value in the key data table A.

[0062] Step S250 : Input the directed graph into the graph feature extraction layer of the data table classification model to obtain a second feature vector ET.

[0063] Specifically, the directed graph is treated as an image containing a large amount of information, and graph features are extracted from it to obtain a second feature vector ET. As an example, the graph feature may be graph density or the like.

[0064] The second eigenvector obtained in this embodiment fully utilizes the already classified data tables that have foreign key relationships with the data table to be classified to construct a directed graph. This allows the data table to be identified as to which category it has a reference relationship with, as well as the direction of the reference relationship, i.e., the degree of closeness. This serves as an important reference for ultimately determining the category to which the data table to be classified belongs.

[0065] In an exemplary embodiment of the present application, the data table to be classified includes several fields; the fields are divided into numerical fields and non-numerical fields; each field has several field values. Step S300 includes:

[0066] Step S310 , obtaining the unique value ratio corresponding to each field in the data table to be classified, so as to obtain the average unique value ratio WP corresponding to the data table to be classified.

[0067] Specifically, the unique value ratio of a field refers to the non-duplicate ratio of all field values ​​corresponding to that field in the data table. For example, if field A has 100 corresponding field values ​​in a data table, and all 100 field values ​​are different, then the unique value ratio of field A is 100%. The average unique value ratio of the data table to be classified is the average unique value ratio of all fields in the data table to be classified. A larger WP indicates a higher degree of data dispersion, with each value being more unique and less duplicated. Conversely, a smaller WP indicates a lower degree of data dispersion and more duplicated data.

[0068] Step S320, obtain the numerical fluctuation ratio ZD=SC / f corresponding to the data table to be classified; wherein SC is the number of fluctuating numerical fields in the data table to be classified; f is the total number of numerical fields in the data table to be classified; the fluctuating numerical fields are fields whose corresponding standard deviations are greater than the preset standard deviation threshold.

[0069] Specifically, a fluctuating numerical field may also be a field whose corresponding variance is greater than a preset variance threshold. A larger ZD indicates that the differences between the data values ​​in the data table to be classified are larger, the distribution is wider, and there are more fields with potentially extreme values. Conversely, a smaller ZD indicates that the differences between the data values ​​in the data table to be classified are larger, the distribution is wider, and there are fewer fields with potentially extreme values. Furthermore, the later the data is processed, the smaller the standard deviation of the field becomes after standardization and other processes. Therefore, in one embodiment, the larger the ZD, the more likely the data table is in its early stages, that is, the original data table. Conversely, the smaller the likelihood.

[0070] Step S330: Obtain the correlation matrix XG corresponding to the data table to be classified; wherein XG meets the following conditions:

[0071] ;

[0072] Where, e=1, 2, ..., f; XG e,f It is the correlation coefficient between the e-th numeric field contained in the data table to be classified and the f-th numeric field contained in the data table to be classified.

[0073] Here, XG e,f =(cov(XG e ,XG f )) / (σ XGe ×σ XGf ); Among them, cov(XG e ,XG f ) is the covariance between the e-th numeric field contained in the data table to be classified and the f-th numeric field contained in the data table to be classified; σ XGe is the standard deviation of the e-th numeric field contained in the data table to be classified; σ XGf The standard deviation of the f-th numeric field in the data table to be classified.

[0074] In one embodiment, the larger the XG, the higher the correlation between fields, that is, the more likely the data table to be classified is original data. This is because in the original data, due to the lack of deduplication, clarification, standardization, aggregation, etc., there may be highly correlated fields. Conversely, the smaller the XG, the lower the correlation between fields, that is, the less likely the data table to be classified is original data. It is also more likely that the data was processed later.

[0075] Step S340 , inputting XG into the graph feature extraction layer of the data table classification model to obtain image features TXG corresponding to XG.

[0076] Step S350: Obtain a third feature vector ST=(WP, ZD, z, TXG) according to WP, ZD, and TXG; wherein z is the total number of fields in the data table to be classified.

[0077] Specifically, the above features and the total number of fields in the data table to be classified are finally taken together as the third feature vector. Here, the third feature vector can represent the discrete features of the data contained in the data table to be classified, the field features, and the association features between the fields.

[0078] In an exemplary embodiment of the present application, the target feature vector MT meets the following conditions:

[0079] MT=(α×YT, β×ET);

[0080] Among them, α is the first weight; β is the second weight.

[0081] Specifically, the obtained first weight and second weight are used to adjust the proportion of the first eigenvector and the second eigenvector in the target eigenvector, so that the obtained target eigenvector can better reflect the characteristics of the data table to be classified. The target eigenvectors corresponding to the data tables to be classified with different characteristics are different, so that the final classification result has stronger mobility and adaptability, and is more accurate.

[0082] It should be noted that the above steps are all implemented by modules in the data table classification model.

[0083] Please refer to Figure 2 As shown, an embodiment of the present application provides a data table classification device 100, the device comprising:

[0084] The first vector acquisition unit 110 is used to obtain each operation statement of the data table to be classified within the target time window and input it into the text encoding layer of the data table classification model to obtain a first feature vector YT; wherein YT represents the operation characteristics of the data table to be classified in chronological order within the target time window; the end time of the target time window is the current time.

[0085] The second vector acquisition unit 120 is used to obtain each key foreign key information of the data table to be classified, and obtain a second feature vector ET based on the graph feature extraction layer of the data table classification model; wherein ET represents the data reference feature of the data table to be classified; the key foreign key information is the foreign key information between the data table to be classified and the key data table; the key data table and the data table to be classified are in the same database; and the key data table can determine the data table category according to the preset classification rules; the data table to be classified cannot determine the data table category according to the preset classification rules.

[0086] The third vector acquisition unit 130 is used to acquire content feature information of the data table to be classified, and obtain a third feature vector according to the graph feature extraction layer of the data table classification model; wherein the third feature vector represents the data content feature of the data table to be classified.

[0087] The weight acquisition unit 140 is used to input the third feature vector into the weight allocation layer of the data table classification model to obtain a first weight corresponding to YT and a second weight corresponding to ET.

[0088] The target vector acquisition unit 150 is used to obtain a target feature vector according to YT, the first weight, ET and the second weight.

[0089] The classification unit 160 is used to input the target feature vector into the classification layer of the data table classification model to obtain the classification result corresponding to the data table to be classified.

[0090] An embodiment of the present application further provides a computer program product, which includes program code. When the program product is run on an electronic device, the program code is used to enable the electronic device to execute the steps of the method according to various exemplary embodiments of the present application described above in this specification.

[0091] Furthermore, although the steps of the method of the present application are described in a particular order in the accompanying drawings, this does not require or imply that the steps must be performed in this particular order, or that all steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0092] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the embodiments of the present application.

[0093] In an exemplary embodiment of the present application, an electronic device capable of implementing the above method is also provided.

[0094] Those skilled in the art will appreciate that various aspects of the present application can be implemented as systems, methods, or program products. Therefore, various aspects of the present application can be specifically implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation that combines hardware and software aspects, which may be collectively referred to herein as a "circuit," "module," or "system."

[0095] The electronic device according to this embodiment of the present application is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0096] The electronic device is implemented as a general-purpose computing device. Components of the electronic device may include, but are not limited to, the at least one processor, the at least one memory, and a bus connecting different system components (including the memory and the processor).

[0097] The storage stores program codes, which can be executed by the processor, so that the processor executes the steps described in the above “Exemplary Method” section of this specification according to various exemplary embodiments of the present application.

[0098] The memory may include readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory, and may further include read only memory (ROM).

[0099] The storage may also include a program / utility having a set (at least one) of program modules, such program modules including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0100] The bus may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures.

[0101] The electronic device may also communicate with one or more external devices (e.g., a keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device, and / or any device that enables the electronic device to communicate with one or more other computing devices (e.g., a router, modem, etc.). This communication may occur via an input / output (I / O) interface. Furthermore, the electronic device may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter. As shown in the figure, the network adapter communicates with other modules of the electronic device via a bus. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0102] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present application.

[0103] In exemplary embodiments of the present application, a computer-readable storage medium is also provided, on which is stored a program product capable of implementing the aforementioned methods of this specification. In some possible implementations, various aspects of the present application may also be implemented in the form of a program product, which includes program code. When the program product is executed on a terminal device, the program code is used to cause the terminal device to execute the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present application.

[0104] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0105] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0106] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0107] The program code used to perform the operations of the present application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0108] Furthermore, the above-mentioned figures are merely illustrative of the processes included in the methods according to exemplary embodiments of the present application and are not intended to be limiting. It is readily understood that the processes illustrated in the above-mentioned figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0109] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiment of the application, the features and functions of two or more modules or units described above can be concretized in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0110] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A data table classification method, characterized in that: The method comprises: Obtain each operation statement of the data table to be classified within the target time window and input it into the text encoding layer of the data table classification model to obtain the first feature vector YT; where YT represents the chronological operation features of the data table to be classified within the target time window; the end time of the target time window is the current time; Obtain each key foreign key information of the data table to be classified, and obtain a second feature vector ET based on the graph feature extraction layer of the data table classification model; wherein ET represents the data reference feature of the data table to be classified; the key foreign key information is the foreign key information between the data table to be classified and the key data table; the key data table and the data table to be classified are in the same database; and the key data table can determine the data table category according to the preset classification rules; while the data table to be classified cannot determine the data table category according to the preset classification rules; Obtaining content feature information of the data table to be classified, and obtaining a third feature vector based on the graph feature extraction layer of the data table classification model; wherein the third feature vector represents the data content feature of the data table to be classified; Input the third eigenvector into the weight distribution layer of the data table classification model to obtain the first weight corresponding to YT and the second weight corresponding to ET; According to YT, the first weight, ET, and the second weight, the target feature vector is obtained; The target feature vector is input into the classification layer of the data table classification model to obtain the classification result corresponding to the data table to be classified.

2. The data table classification method according to claim 1, characterized in that: Obtain each operation statement of the data table to be classified within the target time window and input it into the text encoding layer of the data table classification model to obtain the first feature vector, including: Obtain each operation statement of the data table to be classified within the target time window to obtain an operation statement list; wherein each operation statement has a corresponding execution time; Input the operation statement list into the text encoding layer of the data table classification model to extract the key operation features in each operation statement; the key operation features include: operation keyword features, field name features, and conditional expression features; Construct the first feature vector YT=(YT1, YT2, ..., YT i ,...,YT n ); i = 1, 2, ..., n; where n is the number of operation statements; YT i YT is the list of eigenvalues ​​corresponding to the operation statement at position i in the order of execution time; i =(YT i，1 , YT i，2 ,...,YT i，j ,...,YT i，m , G i ); j = 1, 2, ..., m; m is the total number of key operating features; YT i，j is the characteristic value of the jth key operation feature of the operation statement ranked in the order of execution time; G i is the normalized time corresponding to the operation statement at position i in the order of execution time; G i Meet the following characteristics: G i =(T i -T min ) / (T min -T max ); where T i is the execution time of the operation statement at position i in the order of execution time; T min T is the earliest execution time among the execution times corresponding to all operation statements in the operation statement list; max The latest execution time among the execution times corresponding to all operation statements in the operation statement list.

3. The data table classification method according to claim 1, characterized in that: Obtain each key foreign key information of the data table to be classified, and extract the second feature vector according to the graph feature extraction layer of the data table classification model, including: Obtain each data table in the database where the data table to be classified is located except the data table to be classified, so as to obtain a data table identification list SB=(SB1, SB2, ..., SB x ,...,SB y ); x=1, 2, ..., y; where y is the number of data tables in the database where the data table to be classified is located, excluding the data table to be classified; Classify each data table in SB according to the preset classification rules to obtain the classified data table identification list SBY=(SBY1, SBY2, ..., SBY a ,...,SBY b ); a=1, 2, ..., b; where b is the number of classified data tables; SBY a is the a-th classified data table; b≤y; Traverse SBY according to each key foreign key information of the data table to be classified to obtain the key data table identification list GSBY=(GSBY1, GSBY2, ..., GSBY c ,...,GSBY d ); c = 1, 2, ..., d; where d is the number of key data tables; GSBY c is the cth key data table; d≤b; the key data table is a classified data table that has at least one reference relationship with the data table to be classified; A directed graph is obtained based on each key foreign key information and GSBY of the data table to be classified; wherein the directed graph includes nodes and directed edges; the nodes include nodes of the data table to be classified and nodes of the key data table; the directed edges represent reference relationships between the nodes of the data table to be classified and each node of the key data table; the directed edges are unidirectional arrows; the arrows point to the referencing party; each of the nodes also includes type information of the corresponding data table to be classified or key data table; The directed graph is input into the graph feature extraction layer of the data table classification model to obtain a second feature vector ET.

4. The data table classification method according to claim 1, characterized in that: The data table to be classified includes several fields; the fields are divided into numerical fields and non-numerical fields; each field has several field values.

5. The data table classification method according to claim 4, characterized in that: Obtain content feature information of the data table to be classified, and extract the third feature vector based on the graph feature extraction layer of the data table classification model, including: Obtain the unique value ratio corresponding to each field in the data table to be classified to obtain the average unique value ratio WP corresponding to the data table to be classified; Get the numerical fluctuation ratio ZD=SC / f corresponding to the data table to be classified; where SC is the number of fluctuating numerical fields in the data table to be classified; f is the total number of numerical fields in the data table to be classified; and a fluctuating numerical field is a field whose corresponding standard deviation is greater than the preset standard deviation threshold. Obtain the correlation matrix XG corresponding to the data table to be classified; where XG meets the following conditions: ; Where, e=1, 2, ..., f; XG e,f is the correlation coefficient between the e-th numeric field in the data table to be classified and the f-th numeric field in the data table to be classified; Input XG into the graph feature extraction layer of the data table classification model to obtain the image feature TXG corresponding to XG; According to WP, ZD and TXG, the third eigenvector ST=(WP, ZD, z, TXG) is obtained; wherein z is the total number of fields in the data table to be classified.

6. The data table classification method according to claim 1, characterized in that: The target feature vector MT meets the following conditions: MT=(α×YT, β×ET); Among them, α is the first weight; β is the second weight.

7. The data table classification method according to claim 1, characterized in that: The classification result is any one of original data, cleaned data, aggregated data and application indicator data.

8. A data table classification device, characterized in that: The device comprises: A first vector acquisition unit is configured to acquire each operation statement of the data table to be classified within a target time window and input the operation statement into a text encoding layer of the data table classification model to obtain a first feature vector YT; wherein YT represents the operation characteristics of the data table to be classified in chronological order within the target time window; the end time of the target time window is the current time; A second vector acquisition unit is configured to acquire each key foreign key information of the data table to be classified, and obtain a second feature vector ET based on a graph feature extraction layer of the data table classification model; wherein ET represents a data reference feature of the data table to be classified; the key foreign key information is the foreign key information between the data table to be classified and the key data table; the key data table and the data table to be classified are in the same database; and the key data table can determine the data table category according to a preset classification rule; while the data table to be classified cannot determine the data table category according to the preset classification rule; a third vector acquisition unit, configured to acquire content feature information of the data table to be classified, and obtain a third feature vector based on the graph feature extraction layer of the data table classification model; wherein the third feature vector represents the data content feature of the data table to be classified; a weight acquisition unit, configured to input the third feature vector into a weight allocation layer of a data table classification model to obtain a first weight corresponding to YT and a second weight corresponding to ET; A target vector acquisition unit, configured to obtain a target feature vector according to YT, a first weight, ET, and a second weight; The classification unit is used to input the target feature vector into the classification layer of the data table classification model to obtain the classification result corresponding to the data table to be classified.

9. A non-transitory computer-readable storage medium, characterized in that The storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: The device comprises a processor and the non-transitory computer-readable storage medium as claimed in claim 9.

Images