A method for tracing page-table relationships

Abstract

The present application relates to the technical field of database, disclose a kind of page and table relationship's traceability method, comprising the following steps: 101, after login business system, the html field after rendering of analysis page and the json data of response return are obtained;102, delete repeated html field and obtain html to be matched field;103, delete the non-object structure in json data and single line json, the response data in json data is clustered, and a json cluster result set is generated for the response data in each cluster cluster;104, for each json cluster result set generates a to-be-matched field sequence;105, by maximum common subsequence algorithm for each to-be-matched field sequence matches database table;The present application can be based on the access of business personnel to page to trace database table and page, and business personnel can understand the content of database table and the logic relationship with page based on the result of traceability, facilitate communication with developer.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database, more particularly, it relates to a page and table relationship tracing method. BACKGROUND

[0002] When the business personnel performs auditing, table checking and other operations, they often need to know which specific back-end data table in the data center the queried page corresponds to, and even need to know the table relationship in the data center. However, the table exists in the database, and the business personnel usually has information barriers for the system development and the table building logic of the back-end database table. When the business personnel proposes the requirement of optimizing the database table in the data center to the developer according to the business demand, the business personnel does not know the content and the logical relationship of the database table, and the developer is difficult to understand the business demand, resulting in poor communication efficiency. SUMMARY

[0003] The present application provides a page and table relationship tracing method, which solves the technical problem that the business personnel does not know the content and the logical relationship of the database table in the related art, the developer is difficult to understand the business demand, and the communication efficiency is poor.

[0004] The present application provides a page and table relationship tracing method, which includes the following steps:

[0005] 101, after logging into the business system, obtaining the html field after rendering the analysis page and the json data returned by the response;

[0006]

[0007] 102, deleting the repeated html fields to obtain the html matching fields;

[0008] 103, deleting the non-object structure and single-row json in the json data, clustering the response data in the json data, and generating a json clustering result set for the response data in each cluster;

[0009] 104, generating a matching field sequence for each json clustering result set, and the method of generating a matching field sequence for each json clustering result set includes: matching an html matching field for each response data in the json clustering result set, and arranging the html matching fields matched by all the response data in the json clustering result set in sequence to generate the matching field sequence;

[0010] The method of matching a database table for each matching field sequence includes:

[0011] ​201, generating field sequences for each database table, saving field sequences of all database tables with fitness greater than a set first fitness threshold in an alternative library;

[0012] 202, iteratively performing the following steps: initializing j = 1, selecting the jth database table, and generating a new database table field sequence by splicing the field sequence of the jth database table with the field sequences of the (j+1)th to Nth database tables calculating the fitness of the new database table field sequence with the to-be-matched field sequence putting the new database table field sequence into the alternative library if the fitness is greater than the set first fitness threshold;

[0013] 203, if i < N, adding 1 to j, and then returning to step 202, if i = N, then terminating the steps;

[0014] sorting the field sequences of the database tables in the alternative library according to the fitness from large to small, and then selecting the first M database tables corresponding to the field sequences with the largest fitness to establish a traceability relationship with the page.

[0015] Further, step 102 first extracts features from the html fields, obtains a mutually exclusive feature set by analyzing the feature vectors, and then constructs a feature matrix as the input of an unsupervised clustering algorithm. The clustering result includes two categories, namely non-page fields and page fields. The html fields belonging to the page field category are extracted as html to-be-matched fields.

[0016] Further, the method for matching a response data with an html to-be-matched field comprises:

[0017] calculating the similarity between the response data and all html to-be-matched fields, and matching the response data with the html to-be-matched field with the largest similarity.

[0018] Further, the calculation formula of the fitness result of the field sequence of the database table in the alternative library is as follows:

[0019]

[0020] The accuracy represents the similarity between the field sequence of the database table and the to-be-matched field sequence, and the tableNum represents the number of individual field sequences of the database table combined with the field sequence.

[0021] Further, the similarity between the field sequence of the database table and the to-be-matched field sequence uses the number of the same fields in the sequence as the value.

[0022] Further, the following steps are included: ​​

[0023] Step 301, a plurality of pages accessed by a service personnel in a set time period are extracted to generate a page set, a database table of page provenance in the page set is obtained, and then a node graph is generated, the nodes in the node graph are one-to-one mapped with the field names of the database table of page provenance, and there is a connection between the field names of the same database table;

[0024] Step 302, input the generated node graph into a graph neural network, and the graph neural network outputs whether the field name is a field name that the service personnel needs to understand;

[0025] Step 303, delete the database table of page provenance that does not contain the field name that the service personnel needs to understand, and return the remaining database table to the service personnel.

[0026] Further, the access time of the two adjacent pages in the plurality of continuously accessed pages is less than a set first time.

[0027] Further, the full connection layer of the graph neural network outputs binary classification, which respectively corresponds to whether the field name is a field name that the service personnel needs to understand; and the label is specified by the service personnel during training.

[0028] Further, the calculation formula of the hidden layer of the graph neural network is as follows:

[0029] And respectively represent the initial node vectors of nodes i and j, W and b respectively represent the weight parameter and the bias parameter, σ represents the activation function, V i represents the node vector of node i output by the first hidden layer, T i represents a set of nodes directly connected to the i-th node, and T represents the total number of nodes in T i , wherein τ represents a scaling factor.

[0030] Further, the initial node vector of the node in the node graph is obtained by encoding the field name corresponding to the node.

[0031] The beneficial effects of the present application are that the present application can trace the database table and the page based on the access of the service personnel to the page, the service personnel can understand the content of the database table and the logical relationship with the page based on the tracing result, and the communication with the developer is facilitated. BRIEF DESCRIPTION OF DRAWINGS

[0032] Fig. 1 is a flowchart of a page and table relationship tracing method of the present application;

[0033] Fig. 2 is a flowchart of a method for matching a database table by a to-be-matched field sequence of the present application; ​

[0034] Fig. 3 is a flowchart of a further processing method of the present application. DETAILED DESCRIPTION

[0035] The subject matter described herein will now be discussed with reference to example implementations. It should be understood that discussions of these implementations are merely provided to enable those skilled in the art to better understand subject matter in the context of the present specification and are not intended to limit the scope of subject matter to implementations described. Various elements of the implementations discussed can be used in combinations with each other in ways not specifically referred to in this description, and can be omitted, replaced, or added to as desired. Additionally, features described in relation to some examples can be combined in other examples.

[0036] As shown in FIG. 1, a page and table relationship traceability method includes the following steps: Figs. 1-3

[0037] 101, after logging into a business system, obtaining html fields after rendering of an analysis page and response returned json data;

[0038] 102, deleting duplicate html fields to obtain html fields to be matched;

[0039] In an embodiment of the present application, step 102 first extracts features from the html fields, obtains a mutually exclusive feature set (the feature set includes the following features: pure English type, English and dash symbol type, English and number type, English and number and dash symbol type, mixed type of English and number and dash symbol random combination, 0 starting number type, and non-0 starting number type and self-incrementing number type, and dom tree node label) by analyzing the feature vector, and then constructs a feature matrix as an input of an unsupervised clustering algorithm. The clustering result includes two categories, which are non-page fields and page fields, respectively. The html fields belonging to the page field category are extracted as the html fields to be matched.

[0040] 103, deleting non-object structures and single-line json in the json data, clustering the response data in the json data, and generating a json clustering result set for the response data in each clustering cluster;

[0041] 104, generating a matching field sequence for each json clustering result set, a method for generating a matching field sequence for each json clustering result set includes:

[0042] matching an html field to be matched for each response data in the json clustering result set, and arranging the html fields to be matched matched by all response data in the json clustering result set in order to generate a matching field sequence;

[0043] ​The method for matching the response data with an html field to be matched comprises:

[0044] The similarity of the response data with all the html fields to be matched is calculated, and the html field to be matched with the largest similarity is matched with the response data.

[0045] 105, matching each field sequence to be matched with the database table by using the maximum common subsequence algorithm;

[0046] The method for matching each field sequence to be matched with the database table comprises:

[0047] 201, generating a field sequence for each database table, and saving the field sequences of all the database tables with the fitness greater than a set first fitness threshold in a candidate library;

[0048] 202, iteratively performing the following steps: initializing j=1, selecting the jth database table, and splicing the field sequence of the jth database table with the field sequences of the (j+1)th to Nth database tables to generate a new field sequence of the database table calculating the similarity with the field sequence to be matched, and putting the database table with the fitness greater than the set first fitness threshold into the candidate library;

[0049] 203, if i

[0050] The calculation formula of the fitness result of the field sequence of the database table in the candidate library is as follows

[0051]

[0052] The accuracy represents the similarity of the field sequence of the database table with the field sequence to be matched, and the tableNum represents the number of the field sequence of the single database table in the field sequence combination of the database table. The similarity of the field sequence of the database table with the field sequence to be matched can use the number of the same fields in the sequences as the value.

[0053] The field sequences of the database tables in the candidate library are sorted according to the fitness from large to small, and then the database tables corresponding to the first M field sequences of the database tables with the largest fitness are selected to establish the traceability relationship with the page.

[0054] In an embodiment of the present application, the business system is an audit system, and the business personnel are auditors.

[0055] ​​​The method mainly considers the logical relationship of background data call for tracing, and the consideration of the actual understanding demand of the business personnel for the database table is less. If the M value is too small, the database table that the business personnel needs to understand will be filtered out. If the M value is too large, the number of returned database tables will be large, which increases the burden of the business personnel. In fact, it is difficult to find a suitable M value.

[0056] In an embodiment of the application, a page and table relationship tracing method further comprises the following steps:

[0057] Step 301: Extracting a plurality of pages continuously accessed by the business personnel in a set time period to generate a page set, obtaining database tables traced by the pages in the page set, and then generating a node graph, wherein the nodes in the node graph are one-to-one mapped with field names of the database tables traced by the pages, and there is a connection between the field names of the same database table.

[0058] The access time of the two adjacent pages in the plurality of continuously accessed pages is less than a set first time.

[0059] Step 302: Inputting the generated node graph into a graph neural network, and the graph neural network outputs whether the field name is a field name that the business personnel needs to understand.

[0060] The full connection layer of the graph neural network outputs binary classification, which respectively corresponds to whether the field name is a field name that the business personnel needs to understand. The label is specified by the business personnel during training.

[0061] The calculation formula of the hidden layer of the graph neural network is as follows:

[0062] And respectively represent the initial node vectors of nodes i and j, W and b respectively represent the weight parameter and the bias parameter, and sigma represents the activation function. i V represents the node vector of node i output by the first hidden layer, and T i represents a set of nodes directly connected to the i-th node, and T represents the total number of nodes in T i . wherein tau represents a scaling factor, and the default value is 0.1.

[0063] Step 303: Deleting the database tables traced by the pages which do not contain the field name that the business personnel needs to understand, and returning the remaining database tables to the business personnel.

[0064] In an embodiment of the application, the initial node vector of the node in the node graph is obtained by encoding the field name corresponding to the node. Specifically, it can be word vector encoding.

[0065] The above describes the embodiments of the present embodiment, but the present embodiment is not limited to the above-described specific embodiments, and the above-described specific embodiments are only illustrative but not restrictive, and those skilled in the art can make many forms under the inspiration of the present embodiment, which all belong to the protection of the present embodiment.

Claims

1. A method of tracing page-to-table relationships, the method comprising: The method comprises the following steps: 101, after logging into the business system, obtaining the html field after rendering the analysis page and the returned json data; 102, deleting repeated html fields to obtain html matching fields; 103, deleting non-object structures and single-line json in the json data, clustering the response data in the json data, and generating a json cluster result set for the response data in each cluster; 104, generating a matching field sequence for each json cluster result set, a method for generating a matching field sequence for each json cluster result set comprising: matching each response data in the json cluster result set with an html matching field, and arranging the html matching fields matched by all response data in the json cluster result set in order to generate a matching field sequence; 105, matching each matching field sequence with a database table through a maximum common subsequence algorithm; A method for matching a matching field sequence with a database table comprises: 201, generating a field sequence for each database table, and saving the field sequences of all database tables with a fitness greater than a first fitness threshold in a candidate library; 202, iteratively perform the following steps: initialize j = 1, select the jth database table, concatenate the field sequence of the jth database table with the field sequence of the (j+1)th to Nth database tables to generate a new field sequence of a database table put into the candidate pool if the fitness is greater than a set first fitness threshold​​​ 203, if i The database tables in the candidate library are sorted according to the fitness from large to small, and then the database tables corresponding to the first M database table field sequences with the largest fitness are selected to establish a traceability relationship with the page.

2. The method of claim 1, wherein, Step 102 first extracts features from the html field, obtains mutually exclusive feature sets through analysis of the feature vectors, and then constructs a feature matrix as the input of an unsupervised clustering algorithm. The clustering result includes two categories, namely non-page fields and page fields. The html fields belonging to the page field category are extracted as html matching fields.

3. The method of claim 1, wherein, A method for matching a response data with an html matching field comprises: Calculate the similarity of the response data and all html matching fields, and match the response data with the html matching field with the largest similarity.

4. The method of claim 1, wherein, The calculation formula of the fitness result of the field sequence of the database table in the candidate library is as follows: The accuracy represents the similarity between the field sequence of the database table and the matching field sequence, and the tableNum represents the number of single database table field sequences in the field sequence combination of the database table.

5. The method of claim 4, wherein, The similarity between the field sequence of the database table and the matching field sequence uses the number of the same fields in the sequence as the value.

6. The method of claim 1, wherein, Further comprising the following steps: Step 301, extracting multiple pages accessed by the business personnel in a set time period to generate a page set, obtaining the database table traced by the page in the page set, and then generating a node graph. The nodes in the node graph are one-to-one mapped with the field names of the database table traced by the page, and there is a connection between the field names of the same database table; Step 302, inputting the generated node graph into a graph neural network, and the graph neural network outputs whether the field name is a field name that the business personnel needs to understand. Step 303, deleting the database table of the page trace which does not contain the field name that the business personnel need to know, and returning the remaining database table to the business personnel.

7. The method of claim 6, wherein, The access time of two adjacent pages in the plurality of pages accessed continuously is less than a set first time.

8. The method of claim 6, wherein, The full connection layer of the graph neural network outputs a binary classification, corresponding to whether the field name is one that the business personnel need to know or not; the label is specified by the business personnel during training.

9. The method of claim 6, wherein, The calculation formula of the hidden layer of the graph neural network is as follows: and denote the initial node vectors of nodes i and j, respectively, W and b denote the weight parameters and bias parameters, respectively, σ denotes an activation function, V i denotes the node vector of node i of the output of the first hidden layer, T i denotes the set of nodes directly connected to the i-th node, T denotes the total number of nodes in T i where τ denotes a scaling factor.​ 10. The method of claim 6, wherein, The initial node vector of the node in the node graph is obtained by encoding the field name corresponding to the node.

Images