Data relationship extraction method and related device

Abstract

The invention discloses a data relation extraction method and a related device, and relates to the technical field of databases, the data relation extraction method comprises the following steps: dividing sampling values of fields into an equivalent sampling table and an approximate sampling table according to association modes of the fields contained in a target data table, and for each field contained in the target data table, dividing the sampling values of the fields into an equivalent sampling table and an approximate sampling table; and determining the number of the same sampling values of each field in the corresponding sampling table and the field, and calculating the similarity between the target field and each field in the target data table according to the number of the same sampling values, so as to determine the associated field of the target field based on the similarity. According to the method, the similarity is calculated according to the field sampling values, then the data relation between the fields is extracted on the basis of the similarity, and due to the fact that in an actual scene, all the used fields of the database put into use have the sampling values, the mode of calculating the similarity through the sampling values can enable the extracted data relation to be more complete.

Description

Technical Field

[0001] The present application relates to the field of database technology, and in particular to a data relationship extraction method and related devices. Background Art

[0002] With the rapid development of the digital age, data is growing explosively. How to efficiently manage the increasing and diverse data has become a key issue that needs to be addressed. Driven by this urgent and significant demand, data governance has emerged. It uses scientific methods to comprehensively optimize data acquisition, processing and supervision to achieve effective data management.

[0003] Data relationship extraction and management are important parts of data governance. At present, the method for extracting data relationships between fields in a database is to parse the SQL (Structured Query Language) statements used in data processing, query and other processes in the database, analyze the data relationships contained in the SQL statements, and obtain the data relationships between the fields in each data table.

[0004] The above SQL statement parsing method can automatically and quickly obtain data relations, but in some scenarios, the SQL statement may not be obtained or the data relations in the SQL statement are incomplete, which results in incomplete data relations extracted by the SQL statement parsing method. Summary of the invention

[0005] In view of the above problems, the present application provides a data relationship extraction method and related devices to achieve the purpose of extracting complete data relationships in the target data table. The specific scheme is as follows:

[0006] The first aspect of the present application provides a data relationship extraction method, comprising:

[0007] According to the association mode of each field contained in the target data table, the sampling values ​​of each field are divided into an equal value sampling table and an approximate sampling table, wherein the sampling value of any field is obtained by removing duplicates and sorting all field values ​​of the field, and then sampling a preset number of field values ​​that are ranked first and ranked last, respectively, and the target data table includes at least one data table that needs to be subjected to data relationship extraction;

[0008] For each field included in the target data table, determine the number of identical sampling values ​​that each field in the corresponding sampling table has with the field, wherein if the association mode of the field is equal value association, the corresponding sampling table is the equal value sampling table, and the identical sampling values ​​are equal value sampling values; if the association mode of the field is approximate association, the corresponding sampling table is the approximate sampling table, and the identical sampling values ​​are approximately equal sampling values;

[0009] The similarity between the target field and each field in the target data table is calculated according to the number of the same sampling values, so as to determine the associated field of the target field based on the similarity, wherein the target field refers to the field in the target data table for which data relationship extraction is to be performed.

[0010] In a possible implementation, determining the number of identical sampling values ​​of each field in the corresponding sampling table and the field includes:

[0011] If the association mode of the field is the equal value association, the corresponding sampling table is used as an inverted table, and a left join mode is used to determine the number of identical sampling values ​​that each field in the corresponding sampling table has with the field.

[0012] In a possible implementation, determining the number of identical sampling values ​​of each field in the corresponding sampling table and the field includes:

[0013] If the association mode of the field is the approximate association, each sampling value of the field is sequentially used as a first sampling value for approximate calculation processing, wherein the process of performing the approximate calculation processing on the first sampling value includes: using a similarity calculation function to calculate the approximation between the first sampling value and each sampling value in the corresponding sampling table, and recording the field name and data table name to which the sampling value in the corresponding sampling table whose approximation is greater than a similarity threshold belongs in the approximate calculation table;

[0014] The approximate calculation table obtained after performing the approximate calculation process on all sample values ​​of the field is used as the target approximate calculation table;

[0015] The number of fields in the target approximate calculation table that have the same name as the data table name is counted as the number of identical sampling values ​​that each field in the corresponding sampling table has with the field.

[0016] In a possible implementation, if the target data table is an Oracle database table, the similarity calculation function is an edit distance calculation function.

[0017] In a possible implementation, calculating the similarity between the target field and each field in the target data table according to the number of the same sample values ​​includes:

[0018] Using each field in the target data table as a field to be compared in turn;

[0019] Determine the number of sample values ​​of the field to be compared as a first number;

[0020] Determine the number of sample values ​​of the target field as a second number;

[0021] Acquire the number of the same sample values ​​of the field to be compared and the target field as a third number;

[0022] Determine the similarity between the field to be compared and the target field according to the first number, the second number and the third number;

[0023] To obtain the similarity between each field in the target data table and the target field.

[0024] In a possible implementation, determining the similarity between the field to be compared and the target field according to the first number, the second number, and the third number includes:

[0025] If the first number is less than or equal to the second number, dividing the third number by the first number to obtain the similarity between the field to be compared and the target field;

[0026] If the first number is greater than the second number, the third number is divided by the second number to obtain the similarity between the field to be compared and the target field.

[0027] A second aspect of the present application provides a data relationship extraction device, comprising:

[0028] A sampling value classification module, used for classifying the sampling values ​​of each field into an equal value sampling table and an approximate sampling table according to the association mode of each field contained in the target data table, wherein the sampling value of any field is obtained by removing duplicates and sorting all field values ​​of the field, and then sampling a preset number of field values ​​that are ranked first and ranked last, respectively, and the target data table includes at least one data table that needs to be subjected to data relationship extraction;

[0029] A sampling value comparison module is used to determine, for each field included in the target data table, the number of identical sampling values ​​that each field in the corresponding sampling table has with the field, wherein if the association mode of the field is equal value association, the corresponding sampling table is the equal value sampling table, and the identical sampling values ​​are equal value sampling values; if the association mode of the field is approximate association, the corresponding sampling table is the approximate sampling table, and the identical sampling values ​​are approximately equal sampling values;

[0030] An associated field extraction module is used to calculate the similarity between the target field and each field in the target data table according to the number of the same sampling values, so as to determine the associated field of the target field based on the similarity, wherein the target field refers to the field in the target data table for which data relationship extraction is to be performed.

[0031] A third aspect of the present application provides a computer program product, comprising computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the data relationship extraction method of the first aspect or any implementation of the first aspect.

[0032] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0033] The memory is used to store computer programs;

[0034] The processor is used to execute the computer program so that the electronic device can implement the data relationship extraction method of the first aspect or any implementation manner of the first aspect.

[0035] In a fifth aspect, the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the data relationship extraction method in the first aspect or any implementation method of the first aspect.

[0036] By means of the above technical scheme, the data relationship extraction method provided by the present application divides the sampling values ​​of each field into an equal value sampling table and an approximate sampling table according to the association mode of each field contained in the target data table. For each field contained in the target data table, the number of identical sampling values ​​of each field in the corresponding sampling table and the field is determined. The similarity between the target field and each field in the target data table is calculated based on the number of identical sampling values, so as to determine the associated field of the target field based on the similarity. It can be seen that the present application calculates the similarity based on the field sampling values, and then extracts the data relationship between the fields based on the similarity. Since all fields have sampling values, the extracted data relationship can be made more complete by calculating the similarity through the sampling values. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0038] Figure 1 A schematic diagram of a system architecture provided for this application;

[0039] Figure 2 A schematic diagram of an optional hardware structure of the terminal 100 provided in this application;

[0040] Figure 3 A schematic diagram of the structure of a server 200 provided in this application;

[0041] Figure 4 A flowchart of a data relationship extraction method provided in this application;

[0042] Figure 5 A schematic diagram of the structure of the data sampling table provided for this application;

[0043] Figure 6 A schematic diagram of the structure of the association method table provided for this application;

[0044] Figure 7 A schematic diagram of the structure of a data relationship extraction device provided in this application;

[0045] Figure 8 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0046] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0047] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0048] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0049] See also Figure 1 , Figure 1 A schematic diagram of a system architecture is shown. The system may include a terminal 100 and a server 200. The server 200 may include one or more servers ( Figure 1 In the example, a server is included, and the server 200 can provide the method provided in the embodiment of the present application for one or more terminals.

[0050] Among them, an application can be installed on the terminal 100, and the above application and web page can provide an interface. The terminal 100 can receive relevant parameters entered by the user on the interface and send the above parameters to the server 200. The server 200 can obtain processing results based on the received parameters and return the processing results to the terminal 100.

[0051] It should be understood that in some optional implementations, the terminal 100 can also complete the action of obtaining the processing result based on the received parameters by itself without the cooperation of the server, and the embodiments of the present application are not limited to this.

[0052] Next describe Figure 1 The product form of the mid-terminal 100;

[0053] The terminal 100 in the embodiment of the present application can be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., and the embodiment of the present application does not impose any restrictions on this.

[0054] Figure 2 An optional hardware structure diagram of the terminal 100 is shown.

[0055] refer to Figure 2 As shown, the terminal 100 may include a radio frequency unit 110, a memory 120, an input unit 130, a display unit 140, a camera 150 (optional), an audio circuit 160 (optional), a speaker 161 (optional), a microphone 162 (optional), an earphone jack 163 (optional), a processor 170, an external interface 180, a power supply 190 and other components. Those skilled in the art will appreciate that Figure 2 These are merely examples of terminals or multi-function devices and do not constitute limitations on the terminals or multi-function devices, which may include more or fewer components than those shown in the figures, or combinations of certain components, or different components.

[0056] The input unit 130 can be used to receive input digital or character information, and generate key signal input related to the user settings and function control of the portable multifunctional device. Specifically, the input unit 130 may include a touch screen 131 (optional) and / or other input devices 132. The touch screen 131 can collect the user's touch operations on or near it (such as the user's operation on or near the touch screen using any suitable object such as fingers, joints, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch screen can detect the user's touch action on the touch screen, convert the touch action into a touch signal and send it to the processor 170, and can receive and execute the command sent by the processor 170; the touch signal at least includes the touch point coordinate information. The touch screen 131 can provide an input interface and an output interface between the terminal 100 and the user. In addition, the touch screen can be implemented using multiple types such as resistive, capacitive, infrared and surface acoustic wave. In addition to the touch screen 131, the input unit 130 can also include other input devices. Specifically, other input devices 132 may include, but are not limited to, one or more of a physical keyboard, function keys (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, and the like.

[0057] Among them, the input device 132 can receive input data and the like.

[0058] The display unit 140 may be used to display information input by a user or provided to a user, various menus of the terminal 100, an interactive interface, file display, and / or playback of any multimedia file.

[0059] The memory 120 can be used to store instructions and data. The memory 120 can mainly include an instruction storage area and a data storage area. The data storage area can store various data, such as multimedia files, texts, etc.; the instruction storage area can store software units such as operating systems, applications, instructions required for at least one function, or their subsets and extensions. It can also include a non-volatile random access memory; provide the processor 170 with hardware, software and data resources including management of computing and processing equipment, and support control software and applications. It is also used for the storage of multimedia files, and the storage of running programs and applications.

[0060] The processor 170 is the control center of the terminal 100. It uses various interfaces and lines to connect various parts of the entire terminal 100. By running or executing instructions stored in the memory 120 and calling data stored in the memory 120, it executes various functions of the terminal 100 and processes data, thereby controlling the terminal device as a whole. Optionally, the processor 170 may include one or more processing units; preferably, the processor 170 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 170. In some embodiments, the processor and the memory may be implemented on a single chip, and in some embodiments, they may also be implemented separately on separate chips. The processor 170 may also be used to generate corresponding operation control signals, send them to corresponding components of the computing and processing device, read and process data in the software, especially read and process data and programs in the memory 120, so that each functional module therein performs corresponding functions, thereby controlling the corresponding components to act according to the requirements of the instructions.

[0061] Among them, the memory 120 can be used to store software codes related to the data relationship extraction method, the processor 170 can execute the steps of the data relationship extraction method, and can also schedule other units (such as the above-mentioned input unit 130 and display unit 140) to achieve corresponding functions.

[0062] The radio frequency unit 110 (optional) can be used for receiving and sending information or receiving and sending signals during a call, for example, after receiving the downlink information of the base station, it is sent to the processor 170 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the radio frequency unit 110 can also communicate with network devices and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile Communication (Global System of Mobile Communication, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.

[0063] In this embodiment of the present application, the RF unit 110 can send data to the server 200 and receive processing results sent by the server 200.

[0064] It should be understood that the radio frequency unit 110 is optional and can be replaced by other communication interfaces, such as a network port.

[0065] The terminal 100 also includes a power supply 190 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 170 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption.

[0066] The terminal 100 further includes an external interface 180 , which may be a standard Micro USB interface or a multi-pin connector, and may be used to connect the terminal 100 to communicate with other devices, or to connect a charger to charge the terminal 100 .

[0067] Although not shown, the terminal 100 may also include a flashlight, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with different functions, etc., which are not described in detail here. Some or all of the methods described below may be applied in the following embodiments. Figure 2 In the terminal 100 shown.

[0068] Next describe Figure 1 The product form of the server 200;

[0069] Figure 3 A structural diagram of a server 200 is provided, such as Figure 3 As shown, the server 200 includes a bus 201, a processor 202, a communication interface 203, and a memory 204. The processor 202, the memory 204, and the communication interface 203 communicate with each other via the bus 201.

[0070] The bus 201 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0071] The processor 202 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0072] The memory 204 may include a volatile memory, such as a random access memory (RAM). The memory 204 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0073] The memory 204 may be used to store software codes related to the data relationship extraction method, and the processor 202 may execute the steps of the data relationship extraction method of the chip, and may also schedule other units to implement corresponding functions.

[0074] It should be understood that the above-mentioned terminal 100 and server 200 can be centralized or distributed devices, and the processors in the above-mentioned terminal 100 and server 200 (such as processor 170 and processor 202) can be hardware circuits (such as application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), general-purpose processors, DSPs, microprocessors or microcontrollers, etc.), or a combination of these hardware circuits. For example, the processor can be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.

[0075] The present application provides a data relationship extraction method. The data relationship extraction method of the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0076] Reference Figure 4 , Figure 4 A schematic diagram of a data relationship extraction method provided in an embodiment of the present application, the method may include:

[0077] Step S401: According to the association mode of each field contained in the target data table, the sampling value of each field is divided into an equal value sampling table and an approximate sampling table.

[0078] Specifically, in this embodiment, at least one data table that needs to perform data relationship extraction may be collectively referred to as a target data table. The at least one data table may be a table in the same type of database or a table in a different type of database, for example, a MySQL database table, an Oracle database table, and so on.

[0079] It is worth noting that each of the at least one data table mentioned above refers to a database table that has been put into use in an actual scenario (only fields in the database table that has been put into use in an actual scenario have field values, and the following sampling can be performed).

[0080] Since there are many field values ​​in the target data table, it will take a lot of time to extract and analyze data relationships based on all field values. In this embodiment, in order to improve efficiency, each field included in the target data table can be sampled separately.

[0081] Since sampling will reduce the amount of data, in order to avoid a decrease in the accuracy of data relationships due to a reduction in the amount of data, this embodiment may perform sampling in the following manner so that the sampled data can replace the overall data as much as possible to obtain a similar calculation result.

[0082] Optionally, the sampling process in this embodiment is: for each field contained in the target data table, all field values ​​of the field are deduplicated and sorted, and then a preset number of field values ​​that are sorted at the front and back are sampled as sampling values ​​of the field.

[0083] More specifically, taking the i-th data table table_i contained in the target data table as an example, the j-th field of table_i is recorded as col_i_j. When sampling the field col_i_j, all field values ​​of the field are first deduplicated, and then sorted in lexicographical order, and then the first x field values ​​and the last x field values ​​in the sorted result are taken, and a total of 2x field values ​​are obtained as the sampling value of the field col_i_j.

[0084] Optionally, after sampling, this embodiment may first insert 2x field values ​​into the data sampling table table_s, which contains three fields, namely, table name, field name and field value (ie, sampling value). The specific structure is as follows: Figure 5 shown.

[0085] Optionally, the above sampling process may be implemented through an SQL statement, that is, this embodiment may call a corresponding SQL statement to perform sampling for each data table included in the target data table.

[0086] Taking the Oracle database table as an example, the SQL statement corresponding to sampling and inserting data into the sampling table is:

[0087] insert into table_s (table_name,column_name,value)

[0088] select 'table_i','col_i_j',col_i_j as value

[0089] from (select distinct col_i_j ,count(*) num from table_i order bycol_i_j asc)

[0090] where ROWNUM<=1000 or ROWNUM>=num-1000;

[0091] Optionally, the above x can be set to 1000. Then, as shown in the above SQL statement, the field values ​​with row numbers ROWNUM less than or equal to 1000 and greater than or equal to num-1000 can be obtained respectively, where num represents the total number of col_i_j field values ​​after deduplication. Deduplication can avoid the situation where the first 1000 values ​​and the last 1000 values ​​overlap when the amount of data is small, resulting in duplicate data being sampled.

[0092] In this embodiment, all fields included in the target data table may be sampled in sequence according to the above sampling process, and the sampled values ​​may be inserted into table_s in sequence until all fields are sampled.

[0093] After the above sampling, data association can be performed. Data association refers to the process of batch associating the values ​​of two fields in some way to find equal values ​​between the two. Equality includes complete equality and approximate equality, and the corresponding association methods are equal value association and approximate association.

[0094] In a possible implementation, the association mode of each field contained in the target data table can be pre-configured. Optionally, the configuration principle can be: if the field values ​​need to be completely consistent, the association mode of the fields can be configured as equal-value association, such as for numeric values, dictionary types and other fields, the association mode can be configured as equal-value association; if the field values ​​need to be roughly the same and there can be some differences, the association mode of the fields can be configured as approximate association.

[0095] Optionally, in order to facilitate subsequent processing, the configured association method can be recorded in an association method table. The structure of the association method table is as follows: Figure 6 , including: table name, field name and association method. Optionally, the association method value of the field that needs equal association is 0, and the association method value of the field that needs approximate association is 1.

[0096] In this embodiment, in order to facilitate subsequent different data association operations on different fields, the sampling values ​​in the above-mentioned data sampling table table_s can be split according to the association method of the fields, and the sampling values ​​of the fields that need equal value association are stored in the equal value sampling table table_s2, and the sampling values ​​of the fields that need approximate association are stored in the approximate sampling table table_s3. Here, the table structures of table_s2 and table_s3 are the same as table_s.

[0097] Optionally, in this embodiment, indexes may be created on the table names, field names, and field values ​​of the above table_s2 and table_s3 to improve the efficiency of querying data.

[0098] Step S402: for each field included in the target data table, determine the number of identical sampling values ​​that each field in the corresponding sampling table has with the field.

[0099] As described above, each field in table_s2 and table_s3 has multiple sampling values. In this embodiment, for each field included in the target data table, the number of identical sampling values ​​that each field in the sampling table corresponding to the field has can be determined.

[0100] Among them, if the association method of the field contained in the target data table is equal-value association, then the above-mentioned corresponding sampling table is an equal-value sampling table, and the above-mentioned same sampling value refers to an equal-value sampling value; if the association method of the field contained in the target data table is approximate association, then the above-mentioned corresponding sampling table is an approximate sampling table, and the above-mentioned same sampling value refers to an approximately equal sampling value.

[0101] Taking the equally associated field b1 as an example, this embodiment can compare the sampling values ​​of each field in table_s2 (the fields here do not include b1) with the sampling values ​​of field b1 for equality. Taking field b2 in table_s2 as an example, assuming that the sampling values ​​of b1 include 100, 110, 120, 130, 140 and 150, and the sampling values ​​of b2 include 100, 120, 125, 135, 140 and 155, then the number of identical sampling values ​​of b1 and b2 is 3.

[0102] Taking the approximately associated field b3 as an example, this embodiment can approximately compare the sampling values ​​of each field in table_s3 (the fields here do not include b3) with the sampling values ​​of field b3. Taking field b4 in table_s3 as an example, assuming that the sampling values ​​of b3 include abc, def, ghj, sdf and wer, and the sampling values ​​of b4 include: abd, gty, set, opj and wes, assuming that abc and abd are approximately equal, and wer and wes are approximately equal, then the number of identical sampling values ​​​​of b3 and b4 is 2.

[0103] Step S403: Calculate the similarity between the target field and each field in the target data table according to the number of identical sampling values, so as to determine the associated field of the target field based on the similarity.

[0104] The target field refers to the field in the target data table whose data relationship is to be extracted.

[0105] As introduced above, the "number of identical sampling values" in this step includes the number of identical sampling values ​​of the target field and each field in the corresponding sampling table. Therefore, this step of "calculating the similarity between the target field and each field in the target data table" actually refers to calculating the similarity between the target field and each field in the corresponding sampling table.

[0106] That is, in this step, the similarity between the target field and each field in the corresponding sampling table can be calculated according to the number of identical sampling values ​​between the target field and each field in the corresponding sampling table.

[0107] Furthermore, based on the similarity between the target field and each field in the corresponding sampling table, the associated field of the target field is determined from the fields included in the corresponding sampling table.

[0108] Optionally, a similarity threshold may be preset, and if the similarity between the target field and a field in the corresponding sampling table is greater than the similarity threshold, the field is determined to be an associated field of the target field.

[0109] The data relationship extraction method provided by the present application divides the sampling values ​​of each field into an equal value sampling table and an approximate sampling table according to the association mode of each field contained in the target data table. For each field contained in the target data table, the number of identical sampling values ​​of each field in the corresponding sampling table and the field is determined. The similarity between the target field and each field in the target data table is calculated based on the number of identical sampling values, so as to determine the associated field of the target field based on the similarity. It can be seen that the present application calculates the similarity based on the field sampling values, and then extracts the data relationship between the fields based on the similarity. Since all fields have sampling values, the extracted data relationship can be made more complete by calculating the similarity through the sampling values.

[0110] The SQL statement parsing method provided by the prior art has different SQL statements for different data tables, so it is necessary to write a corresponding parsing program for each data table for parsing. The parsing program is difficult to adapt to other data tables, resulting in high complexity of the parsing process. The present embodiment only performs a simple similarity calculation through the sampling values ​​to extract the complete data relationship, which has lower complexity and higher versatility.

[0111] In some embodiments of the present application, the process of "determining the number of identical sampling values ​​of each field in the corresponding sampling table and the field" in the above step S402 is introduced.

[0112] In this embodiment, there are different implementation processes for fields with equal value association and fields with approximately equal value association, which are introduced one by one below.

[0113] First, if the association method of the field is equal value association, the process of "determining the number of identical sampling values ​​that each field in the corresponding sampling table has with the field" may include: using the corresponding sampling table as an inverted table and using a leftjoin method to determine the number of identical sampling values ​​that each field in the corresponding sampling table has with the field.

[0114] Taking the value-associated field col_i_j in table_i as an example, the field name of the field col_i_j is value. Optionally, this embodiment can first query the sample value of col_i_j from table_s2, save it to the sample data table sample_i_j, and then delete the data of col_i_j from table_s2.

[0115] Taking the Oracle database table as an example, the corresponding SQL statement is:

[0116] # Insert data

[0117] insert into sample_i_j (value)

[0118] select value from table_s2 where table_name=table_i and column_name=col_i_j;

[0119] # Delete data

[0120] delete from table_s2 where table_name=table_i and column_name=col_i_j;

[0121] After obtaining sample_i_j, data association can be performed by querying table_s2.

[0122] The above-mentioned equal value sampling table table_s2 can be used as an inverted table, which contains the mapping relationship between field values, field names, and table names. In this embodiment, the sampling value equal to the sampling value in sample_i_j, as well as the corresponding field name and table name can be quickly found by querying whether the sampling value in sample_i_j exists in table_s2.

[0123] In the specific implementation, in order to improve the search efficiency, you can optionally use the left join method to perform the association.

[0124] Optionally, this embodiment can save the association result of the left join into a similarity calculation table table_similarity. Here, table_similarity contains five fields, namely, table name a1, field name b1, table name a2, field name b2 and the same field number same_num. The corresponding SQL statement is as follows:

[0125] insert into table_similarity (a1,b1,a2,b2,same_num)

[0126] select 'table_i','col_i_j',table_s2.table_name,table_s2.column_name,

[0127] count(*) same_num

[0128] from sample_i_j left join table_s2 on sample_i_j.value=table_s2.value

[0129] group by table_s2.table_name,table_s2.column_name

[0130] In this embodiment, the same sampling values ​​in table_s2 and sample_i_j can be quickly obtained through left join, and then grouped by table name table_name and field name col_i_j, and counted through count to obtain the number of sampling values ​​of the field col_i_j that are the same as the sampling values ​​of each field in the corresponding sampling table.

[0131] In this embodiment, data association needs to be performed for each equal-value associated field contained in the target data table according to the above-mentioned left join method. In order to avoid problems in the data association process of subsequent fields, sample_i_j needs to be deleted after each data association. Taking the field col_i_j as an example, the SQL statement for deleting sample_i_j of col_i_j is as follows:

[0132] drop table sample_i_j;

[0133] In summary, this embodiment can quickly perform data association of fields with equal value association through the left join method, thereby improving the efficiency of data association.

[0134] Second, if the association mode of the field is approximate association, the process of "determining the number of identical sampling values ​​that each field in the corresponding sampling table has with the field" may include: if the association mode of the field is approximate association, then each sampling value of the field is sequentially taken as the first sampling value for approximate calculation processing, wherein the process of approximate calculation processing of the first sampling value includes: using a similarity calculation function to calculate the approximation between the first sampling value and each sampling value in the corresponding sampling table, and recording the field name and data table name to which the sampling values ​​in the corresponding sampling table whose approximation is greater than the approximation threshold belong in the approximate calculation table; taking the approximate calculation table obtained after approximate calculation processing is performed on all sampling values ​​of the field as the target approximate calculation table, and counting the number of identical field names and data table names in the target approximate calculation table as the number of identical sampling values ​​that each field in the corresponding sampling table has with the field.

[0135] Taking the target data table as an Oracle database table as an example, optionally, the similarity calculation function may be an edit distance calculation function UTL_MATCH.edit_distance_similarity.

[0136] Taking the approximately associated field col_i_k in table_i as an example, the field name of the field col_i_k is value. Optionally, this embodiment can first query the sample value of col_i_k from table_s3, save it to the sample data table sample_i_k, and then delete the data of col_i_k from table_s3.

[0137] Taking the Oracle database table as an example, the corresponding SQL statement is:

[0138] # Insert data

[0139] insert into sample_i_k (value)

[0140] select value from table_s3 where table_name=table_i and column_name=col_i_k;

[0141] # Delete data

[0142] delete from table_s3 where table_name=table_i and column_name=col_i_k;

[0143] After obtaining sample_i_k, data association can be performed by querying table_s3. Since col_i_k uses approximate data association, it is not possible to use the join method for association, and a similarity calculation function is required for comparison. Taking Oracle as an example, Oracle provides an edit distance calculation function UTL_MATCH.edit_distance_similarity, which can be used to calculate the similarity between strings. The calculation result is between 0 and 100, and the larger the value, the higher the similarity. In the embodiment of the present application, a similarity threshold ξ is set for judgment. If it is greater than the similarity threshold, it means that the two strings are similar, otherwise they are not similar.

[0144] Since the similarity calculation function can only calculate the approximation between one sampling value in sample_i_k (for the convenience of description, this sampling value is recorded as the first sampling value in this embodiment) and one sampling value in table_s3 at a time, it is necessary to take each sampling value in sample_i_k in turn for calculation once, and finally perform statistics.

[0145] For the convenience of statistics, in this embodiment, the calculation result of the similarity calculation function may be saved in an approximate calculation table table_distance, where table_distance includes two fields: table name table_name and field name column_name.

[0146] The SQL statement for associating and saving the result with the approximate data of the mth sample value (i.e., the first sample value) of sample_i_k is as follows:

[0147] insert into table_distance(table_name, column_name)

[0148] select table_name, column_name from (select table_name, column_name,

[0149] UTL_MATCH.edit_distance_similarity(value, 'value_col_i_k_m') assimilarity

[0150] from table_s3) as a

[0151] where similarity >=

[0152] Each sample value of sample_i_k is calculated and saved according to the above process to obtain the target approximate calculation table.

[0153] Taking the sampling values ​​1 to 3 of col_i_k in sample_i_k as an example, assuming that the sampling value 1 is approximate calculated, the following five groups of field names and data table names are obtained: ([field b3, data table a3], [field b3, data table a3], [field b4, data table a3], [field b5, data table a4], [field b6, data table a5]), the sampling value 2 is approximate calculated, the following three groups of field names and data table names are obtained: ([field b3, data table a3], [field b3, data table a3], [field b4, data table a3]), the sampling value 3 is approximate calculated, the following four groups of field names and data table names are obtained: ([field b3, data table a3], [field b4, data table a3], [field b7, data table a4], [field b6, data table a5]).

[0154] The table name and field name in the target approximate calculation table can be found in Table 1 below.

[0155] Table 1 Target approximate calculation table

[0156]

[0157] Then, taking Table 1 as an example, in this embodiment, “the number of fields in the statistical target approximate calculation table that have the same field name and data table name, as the number of identical sampling values ​​that each field in the corresponding sampling table has” includes the following: ([field b3, data table a3], 5), ([field b4, data table a3], 3), ([field b5, data table a4], 1), ([field b6, data table a5], 2), ([field b7, data table a4], 1).

[0158] Optionally, in order to be consistent with the above-mentioned equal value association method, this embodiment can insert the number of identical sampling values ​​of each field in the corresponding sampling table into table_similarity. The corresponding SQL statement is as follows:

[0159] insert into table_similarity (a1,b1,a2,b2,same_num)

[0160] select 'table_i','col_i_k',table_name,column_name,count(*) same_num

[0161] from table_distance

[0162] group by table_name,column_name

[0163] In addition, you need to clear the data in the table_distance table to facilitate storing information of other fields next time:

[0164] delete from table_distance

[0165] Similarly, delete the sample data table sample_i_k of col_i_k:

[0166] drop table sample_i_k;

[0167] At this point, the data association of the approximately associated field col_i_k and the saving of the association result are completed.

[0168] Taking the statistics of ([field b1, data table a1], 5) in Table 1 above as an example, a piece of data inserted into table_similarity is: (a1, b1, a2, b2, same_num) = (table_i, col_i_k, a1, b1, 5).

[0169] In this embodiment, the data associations between the above-mentioned equal-value associated fields and the data associations between the approximately associated fields can be summarized into the same table_similarity. Subsequently, the similarity between the fields can be calculated based on the number of identical sampling values ​​in table_similarity, and then the data relationship extraction can be performed, thereby improving efficiency.

[0170] In other embodiments of the present application, the process of "calculating the similarity between the target field and each field in the target data table according to the number of identical sampling values" in step S403 is introduced.

[0171] For ease of description, this embodiment can take each field in the target data table as a field to be compared in turn. Optionally, the process of calculating the similarity between the field to be compared and the target field may include: determining the number of sampling values ​​of the field to be compared as a first number, and determining the number of sampling values ​​of the target field as a second number; obtaining the number of identical sampling values ​​of the field to be compared and the target field from table_similarity as a third number; and determining the similarity between the field to be compared and the target field based on the first number, the second number and the third number.

[0172] The above process is used to calculate each field in the target data table, and the similarity between each field in the target data table and the target field can be obtained.

[0173] Optionally, the process of "determining the similarity between the field to be compared and the target field based on the first quantity, the second quantity and the third quantity" may include: if the first quantity is less than or equal to the second quantity, dividing the third quantity by the first quantity to obtain the similarity between the field to be compared and the target field; if the first quantity is greater than the second quantity, dividing the third quantity by the second quantity to obtain the similarity between the field to be compared and the target field.

[0174] Specifically, although the total number of sampling values ​​is specified in the previous text during sampling, there may be a situation where the number of field values ​​is less than the specified total number of sampling values. For this reason, this embodiment can compare the first number and the second number, select the smaller number as the denominator, and use the third number as the numerator. The similarity between the field to be compared and the target field can be obtained by dividing the two, which is used to represent the ratio of the number of identical or similar field values ​​between the field to be compared and the target field.

[0175] Optionally, the SQL statements involved in similarity calculation are as follows:

[0176] update table_similarity set similarity=same_num /

[0177] (min(select count(*) num from table_s where table_name = a1 andcolumn_name = b1,

[0178] select count(*) num from table_s where table_name = a2 and column_name = b2))

[0179] Optionally, the table_similarity may further include a sixth field similarity, which indicates the similarity between the field b1 and the field b2. Then, after the similarity between the field to be compared and the target field is calculated, the similarity may be recorded in the similarity field of the table_similarity.

[0180] It should be understood that the field similarity relationship is mutual, and the similarity between field b1 of table a1 and field b2 of table a2 is equal to the similarity between field b2 of table a2 and field b1 of table a1. Therefore, after this embodiment calculates the similarity between field b1 of table a1 and field b2 of table a2 through the above calculation process, it is also necessary to insert the similarity between field b2 of table a2 and field b1 of table a1 into the similarity field of table_similarity, so as to avoid repeated calculation and improve calculation efficiency.

[0181] Optionally, the SQL statement for inserting the similarity information between field b2 of table a2 and field b1 of table a1 into the table is as follows:

[0182] insert into table_similarity(a1,b1,a2,b2,same_num,similarity)

[0183] select a2,b2,a1,b1,same_num,similarity from table_similarity

[0184] After the similarity calculation is completed, similarity query can be performed when needed. A parameter λ is used as the similarity threshold. The value range of λ is 0~1. If the similarity similarity is greater than the similarity threshold, it means that the two fields are related, otherwise it means that the two fields are not related. The SQL corresponding to querying the association information of the field col_i_j in the table table_i is:

[0185] select * from table_similarity where a1='table_i' and b1='col_i_j'and similarity>=λ

[0186] It should also be noted that the above-mentioned process of calculating similarity is only an example. In addition to this, there may be other implementation methods, for example, determining the similarity between the field to be compared and the target field only based on the number of identical sampling values ​​between the field to be compared and the target field, etc. This application does not make specific limitations.

[0187] In summary, the present application provides a data relationship extraction method based on similarity calculation, which can extract the association relationship between all fields in the database table based on the data of the database table through data sampling, data association, similarity calculation and other steps. When sampling data, the method of sorting and taking the first and last parts of the data is used to reduce the amount of data, and deduplication is used to avoid duplicate data from affecting data association, thereby improving the accuracy of similarity calculation. Two methods of equal association and approximate association are adopted, respectively for the fields that need to be equally associated and the fields that need to be approximately associated, and the differences in actual data are fully considered to improve the accuracy of the calculation results. In the equal association, with the inverted index method, all the association information of a field can be obtained at one time, which improves the efficiency of equal data association. In the approximate association, the database built-in or customized function is used, combined with the approximate threshold control to obtain the approximate association result. The results of equal association and approximate association are summarized in the similarity calculation table, and then the similarity between all fields is calculated at one time through the SQL statement, which has high calculation efficiency. Finally, a similarity threshold is set to determine whether the two fields are related. The threshold can be modified according to the actual situation during the query, and has good flexibility.

[0188] This application can analyze the table data and obtain the field relationship without relying on the existing SQL statements, thus avoiding the problem of incomplete relationship extraction caused by the difficulty in obtaining or missing SQL statements. At the same time, through data sampling, cross-database field relationship analysis can be realized, and only special sampling SQL statements need to be used for each database. However, it is difficult to realize cross-database relationship analysis with the traditional method, and a special program needs to be written for each database table, which is more complicated. At the same time, sampling can also greatly reduce the amount of data and reduce the complexity of subsequent operations.

[0189] In addition, this application designs special equivalent association logic and approximate association logic, respectively for fields that require equivalent association and fields that require approximate association, and cooperates with the similarity calculation method to further improve the efficiency and accuracy of the solution.

[0190] A data relationship extraction method provided in an embodiment of the present application is introduced above, and a device for executing the above data relationship extraction method will be introduced below.

[0191] See also Figure 7 , Figure 7 This is a schematic diagram of the structure of a data relationship extraction device provided in an embodiment of the present application. Figure 7 As shown, the device may include:

[0192] The sampling value classification module 501 is used to classify the sampling values ​​of each field into an equal value sampling table and an approximate sampling table according to the association mode of each field contained in the target data table, wherein the sampling value of any field is obtained by removing duplicates and sorting all field values ​​of the field, and then sampling a preset number of field values ​​that are ranked first and ranked last, respectively, and the target data table includes at least one data table that needs to be subjected to data relationship extraction;

[0193] The sampling value comparison module 502 is used to determine, for each field included in the target data table, the number of identical sampling values ​​of each field in the corresponding sampling table and the field, wherein if the association mode of the field is equal value association, the corresponding sampling table is an equal value sampling table, and the identical sampling values ​​are equal value sampling values; if the association mode of the field is approximate association, the corresponding sampling table is an approximate sampling table, and the identical sampling values ​​are approximately equal sampling values;

[0194] The associated field extraction module 503 is used to calculate the similarity between the target field and each field in the target data table according to the number of identical sampling values, so as to determine the associated field of the target field based on the similarity, wherein the target field refers to the field in the target data table whose data relationship is to be extracted.

[0195] In a possible implementation, when determining the number of identical sampling values ​​of each field in the corresponding sampling table and the field, the sampling value comparison module may be specifically used to:

[0196] If the association mode of the field is equal value association, the corresponding sampling table is used as an inverted table, and the left join method is used to determine the number of identical sampling values ​​that each field in the corresponding sampling table has with the field.

[0197] In a possible implementation, when determining the number of identical sampling values ​​of each field in the corresponding sampling table and the field, the sampling value comparison module may be specifically used to:

[0198] If the association mode of the field is approximate association, each sampling value of the field is sequentially used as the first sampling value for approximate calculation processing, wherein the process of approximate calculation processing for the first sampling value includes: using a similarity calculation function to calculate the approximation between the first sampling value and each sampling value in the corresponding sampling table, and recording the field name and data table name to which the sampling value in the corresponding sampling table whose approximation is greater than the approximation threshold belongs in the approximate calculation table;

[0199] The approximate calculation table obtained after approximate calculation processing is performed on all sample values ​​of the field is used as the target approximate calculation table;

[0200] The statistical target is to approximately calculate the number of fields in the table that have the same name as the data table name, as the number of identical sampling values ​​that each field in the corresponding sampling table has.

[0201] In a possible implementation, if the target data table is an Oracle database table, the similarity calculation function is an edit distance calculation function.

[0202] In a possible implementation, when the above-mentioned associated field extraction module calculates the similarity between the target field and each field in the target data table according to the number of identical sample values, it can be specifically used to:

[0203] Use each field in the target data table as the field to be compared;

[0204] Determine the number of sample values ​​of the field to be compared as a first number;

[0205] Determine the number of sample values ​​of the target field as the second number;

[0206] Obtain the number of identical sample values ​​of the field to be compared and the target field as the third number;

[0207] Determine the similarity between the field to be compared and the target field according to the first quantity, the second quantity and the third quantity;

[0208] To obtain the similarity between each field in the target data table and the target field.

[0209] In a possible implementation, when the associated field extraction module determines the similarity between the field to be compared and the target field according to the first quantity, the second quantity, and the third quantity, it can be specifically used to:

[0210] If the first number is less than or equal to the second number, the third number is divided by the first number to obtain the similarity between the field to be compared and the target field;

[0211] If the first number is greater than the second number, the third number is divided by the second number to obtain the similarity between the field to be compared and the target field.

[0212] The data relationship extraction method provided in this application corresponds to the data relationship extraction device provided in the previous text. For details, please refer to the previous text introduction and will not be repeated here.

[0213] The present application also provides an electronic device in an embodiment. Figure 8 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 8The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0214] like Figure 8 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0215] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a memory card, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0216] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any one of the data relationship extraction methods provided in the embodiments of the present application.

[0217] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any data relationship extraction method provided in the embodiment of the present application.

[0218] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0219] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0220] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0221] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A data relationship extraction method, characterized in that: include: According to the association mode of each field contained in the target data table, the sampling values ​​of each field are divided into an equal value sampling table and an approximate sampling table, wherein the sampling value of any field is obtained by removing duplicates and sorting all field values ​​of the field, and then sampling a preset number of field values ​​that are ranked first and ranked last, respectively, and the target data table includes at least one data table that needs to be subjected to data relationship extraction; For each field included in the target data table, determine the number of identical sampling values ​​that each field in the corresponding sampling table has with the field, wherein if the association mode of the field is equal value association, the corresponding sampling table is the equal value sampling table, and the identical sampling values ​​are equal value sampling values; if the association mode of the field is approximate association, the corresponding sampling table is the approximate sampling table, and the identical sampling values ​​are approximately equal sampling values; The similarity between the target field and each field in the target data table is calculated according to the number of the same sampling values, so as to determine the associated field of the target field based on the similarity, wherein the target field refers to the field in the target data table for which data relationship extraction is to be performed.

2. The data relationship extraction method according to claim 1, characterized in that: The determining of the number of identical sampling values ​​of each field in the corresponding sampling table and the field includes: If the association mode of the field is the equal value association, the corresponding sampling table is used as an inverted table, and a leftjoin method is used to determine the number of identical sampling values ​​that each field in the corresponding sampling table has with the field.

3. The data relationship extraction method according to claim 1, characterized in that: The determining of the number of identical sampling values ​​of each field in the corresponding sampling table and the field includes: If the association mode of the field is the approximate association, each sampling value of the field is sequentially used as a first sampling value for approximate calculation processing, wherein the process of performing the approximate calculation processing on the first sampling value includes: using a similarity calculation function to calculate the approximation between the first sampling value and each sampling value in the corresponding sampling table, and recording the field name and data table name to which the sampling value in the corresponding sampling table whose approximation is greater than a similarity threshold belongs in the approximate calculation table; The approximate calculation table obtained after performing the approximate calculation process on all sample values ​​of the field is used as the target approximate calculation table; The number of fields in the target approximate calculation table that have the same name as the data table name is counted as the number of identical sampling values ​​that each field in the corresponding sampling table has with the field.

4. The data relationship extraction method according to claim 3, characterized in that: If the target data table is an Oracle database table, the similarity calculation function is an edit distance calculation function.

5. The data relationship extraction method according to claim 1, characterized in that: The calculating the similarity between the target field and each field in the target data table according to the number of the same sample values ​​includes: Using each field in the target data table as a field to be compared in turn; Determine the number of sample values ​​of the field to be compared as a first number; Determine the number of sample values ​​of the target field as a second number; Acquire the number of the same sample values ​​of the field to be compared and the target field as a third number; Determine the similarity between the field to be compared and the target field according to the first number, the second number and the third number; To obtain the similarity between each field in the target data table and the target field.

6. The data relationship extraction method according to claim 5, characterized in that: The determining, according to the first number, the second number, and the third number, the similarity between the field to be compared and the target field includes: If the first number is less than or equal to the second number, dividing the third number by the first number to obtain the similarity between the field to be compared and the target field; If the first number is greater than the second number, the third number is divided by the second number to obtain the similarity between the field to be compared and the target field.

7. A data relationship extraction device, characterized in that: include: A sampling value classification module, used for classifying the sampling values ​​of each field into an equal value sampling table and an approximate sampling table according to the association mode of each field contained in the target data table, wherein the sampling value of any field is obtained by removing duplicates and sorting all field values ​​of the field, and then sampling a preset number of field values ​​that are ranked first and ranked last, respectively, and the target data table includes at least one data table that needs to be subjected to data relationship extraction; A sampling value comparison module is used to determine, for each field included in the target data table, the number of identical sampling values ​​that each field in the corresponding sampling table has with the field, wherein if the association mode of the field is equal value association, the corresponding sampling table is the equal value sampling table, and the identical sampling values ​​are equal value sampling values; if the association mode of the field is approximate association, the corresponding sampling table is the approximate sampling table, and the identical sampling values ​​are approximately equal sampling values; An associated field extraction module is used to calculate the similarity between the target field and each field in the target data table according to the number of the same sampling values, so as to determine the associated field of the target field based on the similarity, wherein the target field refers to the field in the target data table for which data relationship extraction is to be performed.

8. A computer program product, characterized in that It comprises computer-readable instructions, and when the computer-readable instructions are executed on an electronic device, the electronic device implements the data relationship extraction method as claimed in any one of claims 1 to 6.

9. An electronic device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the data relationship extraction method as described in any one of claims 1 to 6.

10. A computer storage medium, characterized in that: The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the data relationship extraction method as described in any one of claims 1 to 6.

Images