Database data redundancy processing method, device, electronic device and storage medium

Abstract

The embodiment of the present invention provides a method, device, electronic device and storage medium for data redundancy processing of a database. The method for data redundancy processing of the database includes: obtaining a first summary table and a second summary table pre-stored in the database; based on preset matrix conversion strategy information, converting the first summary table into a first matrix and the second summary table into a second matrix respectively; based on the first matrix and the second matrix, determining a first similarity between the first summary table and the second summary table; when it is determined that the first similarity reaches a similarity threshold, based on a preset mapping relationship, performing a first data redundancy processing operation corresponding to the first similarity on the database; wherein the preset mapping relationship is a mapping relationship between similarity and data redundancy processing operation. The method for data redundancy processing of the database according to the embodiment of the present invention can improve the data redundancy processing effect of the database.

Description

Technical Field

[0001] The present invention belongs to the technical field of big data, and in particular relates to a method, device, electronic equipment and computer storage medium for processing data redundancy in a database. Background Art

[0002] As the market is unpredictable, big data analysis is used to respond to market changes, improve market management capabilities and efficiency, and provide an effective basis for analysis and decision-making for various departments of the company. Although there are more and more computers and clusters used for big data analysis, and their performance is getting better and better, they are still unable to meet the various analysis needs of a large number of demand departments and the rapidly changing large market analysis needs. Therefore, it is necessary to perform data redundancy detection and data redundancy processing on the database to avoid a large waste of database storage space and computing resources.

[0003] At present, the common methods for processing data redundancy in databases are: 1. Big data redundancy detection methods based on file similarity, which mainly detect redundant data by comparing the hash values ​​of stored files, are limited to the analysis and judgment of redundant data blocks from the perspective of files, and can only effectively reduce the amount of data storage. 2. Redundant pattern elimination methods in the database design process, which can only eliminate the redundancy problem of table design in relational databases based on polynomial time complexity. 3. Redundant data detection methods in three-dimensional data patches, which target redundant data in the image data of three-dimensional objects, and only solve the impact of occupying storage space and subsequent three-dimensional imaging processing of three-dimensional data. However, these methods are difficult to accurately determine redundant data, resulting in poor data redundancy processing effects in the database.

[0004] Therefore, how to improve the data redundancy processing effect of the database is a technical problem that technical personnel in this field need to solve urgently. Summary of the invention

[0005] The embodiments of the present invention provide a method, device, electronic device and computer storage medium for processing data redundancy in a database, which can improve the effect of processing data redundancy in the database.

[0006] In a first aspect, an embodiment of the present invention provides a method for processing data redundancy in a database, comprising:

[0007] Obtain a first summary table and a second summary table pre-stored in a database;

[0008] Based on the preset matrix conversion strategy information, the first summary table is converted into a first matrix, and the second summary table is converted into a second matrix;

[0009] Determining a first similarity between the first summary table and the second summary table based on the first matrix and the second matrix;

[0010] When it is determined that the first similarity reaches the similarity threshold, based on a preset mapping relationship, a first data redundancy processing operation corresponding to the first similarity is performed on the database; wherein the preset mapping relationship is a mapping relationship between similarity and data redundancy processing operation.

[0011] Optionally, based on preset matrix conversion strategy information, converting the first summary table into a first matrix and converting the second summary table into a second matrix respectively includes:

[0012] Respectively obtain the composition structure data of the first summary table and the composition structure data of the second summary table; wherein the composition structure data includes a basic table and fields;

[0013] Based on the matrix conversion strategy information, the first matrix and the second matrix are determined respectively using the component structure data of the first summary table and the component structure data of the second summary table.

[0014] Optionally, respectively obtaining the composition structure data of the first summary table and the composition structure data of the second summary table includes:

[0015] Acquire generation logic information of the summary table and process structured query language (SQL) information; wherein the summary table includes a first summary table and a second summary table;

[0016] Based on the generation logic information and the processing SQL information, the composition structure data of the first summary table and the composition structure data of the second summary table are respectively obtained.

[0017] Optionally, based on the generated logic information and the processed SQL information, respectively obtaining the composition structure data of the first summary table and the composition structure data of the second summary table includes:

[0018] Based on the generated logic information and the processed SQL information, the basic table of the first summary table and the basic table of the second summary table are respectively obtained by using the first regular expression;

[0019] Based on the generated logic information and the processed SQL information, the fields of the first summary table and the fields of the second summary table are respectively obtained using the second regular expression.

[0020] Optionally, determining a first similarity between the first summary table and the second summary table based on the first matrix and the second matrix includes:

[0021] Perform a difference operation on the first matrix and the second matrix to obtain a third matrix;

[0022] Based on the third matrix, determining a Euclidean norm value corresponding to the third matrix;

[0023] A first similarity between the first summary table and the second summary table is determined according to the Euclidean norm value.

[0024] In a second aspect, an embodiment of the present invention provides a data redundancy processing device for a database, comprising:

[0025] An acquisition module, used for acquiring a first summary table and a second summary table pre-stored in a database;

[0026] A conversion module, used for converting the first summary table into a first matrix and the second summary table into a second matrix respectively based on preset matrix conversion strategy information;

[0027] A determination module, configured to determine a first similarity between the first summary table and the second summary table based on the first matrix and the second matrix;

[0028] The execution module is used to execute a first data redundancy processing operation corresponding to the first similarity on the database based on a preset mapping relationship when it is determined that the first similarity reaches a similarity threshold; wherein the preset mapping relationship is a mapping relationship between similarity and data redundancy processing operation.

[0029] Optionally, the conversion module is used to respectively obtain the composition structure data of the first summary table and the composition structure data of the second summary table; wherein the composition structure data includes a basic table and fields; based on the matrix conversion strategy information, the composition structure data of the first summary table and the composition structure data of the second summary table are used to respectively determine the first matrix and the second matrix.

[0030] Optionally, the conversion module is used to obtain the generation logic information of the summary table and process the structured query language SQL information; wherein the summary table includes a first summary table and a second summary table; based on the generation logic information and the processing SQL information, the composition structure data of the first summary table and the composition structure data of the second summary table are respectively obtained.

[0031] Optionally, the conversion module is used to obtain the basic table of the first summary table and the basic table of the second summary table respectively based on the generated logical information and processed SQL information using the first regular expression; based on the generated logical information and processed SQL information, use the second regular expression to obtain the fields of the first summary table and the fields of the second summary table respectively.

[0032] Optionally, a determination module is used to perform a difference operation on the first matrix and the second matrix to obtain a third matrix; based on the third matrix, determine the Euclidean norm value corresponding to the third matrix; and determine the first similarity between the first summary table and the second summary table according to the Euclidean norm value.

[0033] In a third aspect, an embodiment of the present invention provides an electronic device, the electronic device comprising: a processor, and a memory storing computer program instructions;

[0034] The processor reads and executes computer program instructions to implement the data redundancy processing method of the database in the first aspect or any optional implementation of the first aspect.

[0035] In a fourth aspect, an embodiment of the present invention provides a computer storage medium having computer program instructions stored thereon, which, when executed by a processor, implements the method for processing database data redundancy in the first aspect or any optional implementation of the first aspect.

[0036] The data redundancy processing method, device, electronic device and computer storage medium of the database of the embodiment of the present invention can improve the data redundancy processing effect of the database. The data redundancy processing method of the database converts the first summary table pre-stored in the database into the first matrix and the second summary table into the second matrix based on the preset matrix conversion strategy information; based on the first matrix and the second matrix, the first similarity between the first summary table and the second summary table can be more accurately determined; therefore, when it is determined that the first similarity reaches the similarity threshold, based on the preset mapping relationship, the first data redundancy processing operation corresponding to the first similarity is performed on the database, which can improve the data redundancy processing effect of the database. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the following is a brief introduction to the drawings required for use in the embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 It is a flowchart of a method for processing data redundancy in a database provided by an embodiment of the present invention;

[0039] Figure 2 It is a flow chart of another method for processing data redundancy in a database provided by an embodiment of the present invention;

[0040] Figure 3 It is a structural schematic diagram of a data redundancy processing device for a database provided by an embodiment of the present invention;

[0041] Figure 4 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by illustrating examples of the present invention.

[0043] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0044] At present, the common methods for processing data redundancy in databases are: 1. Big data redundancy detection methods based on file similarity, which mainly detect redundant data by comparing the hash values ​​of stored files, are limited to the analysis and judgment of redundant data blocks from the perspective of files, and can only effectively reduce the amount of data storage. 2. Redundant pattern elimination methods in the database design process, which can only eliminate the redundancy problem of table design in relational databases based on polynomial time complexity. 3. Redundant data detection methods in three-dimensional data patches, which target redundant data in the image data of three-dimensional objects, and only solve the impact of occupying storage space and subsequent three-dimensional imaging processing of three-dimensional data. However, these methods are difficult to accurately determine redundant data, resulting in poor data redundancy processing effects in the database.

[0045] In order to solve the problems in the prior art, the embodiments of the present invention provide a method, device, electronic device and computer storage medium for processing data redundancy in a database. The following first introduces the method for processing data redundancy in a database provided by the embodiments of the present invention.

[0046] Figure 1 FIG. 1 is a flow chart showing a method for processing data redundancy in a database according to an embodiment of the present invention. Figure 1 As shown, the method for processing data redundancy in the database may include the following steps:

[0047] S101: Obtain a first summary table and a second summary table pre-stored in a database.

[0048] There are two main sources of tables in the database: one is the basic table formed by importing files or other databases, and the other is the table formed by processing and summarizing these basic tables, that is, the summary table. For example, in China Mobile's centralized big data platform, the amount of data is huge, and many of the summary tables processed and summarized based on the basic tables have similar contents, which will cause a lot of waste of storage space and computing resources.

[0049] The first summary table and the second summary table in step S101 may be two arbitrarily selected summary tables, but the similarity of the contents between the two summary tables cannot be directly and accurately determined, so step S102 needs to be executed.

[0050] S102: Based on preset matrix conversion strategy information, convert the first summary table into a first matrix and the second summary table into a second matrix.

[0051] In order to more accurately determine the first matrix and the second matrix, in one embodiment, based on preset matrix conversion strategy information, converting the first summary table into the first matrix and the second summary table into the second matrix respectively may include: obtaining the composition structure data of the first summary table and the composition structure data of the second summary table respectively; wherein the composition structure data includes a basic table and a field; based on the matrix conversion strategy information, using the composition structure data of the first summary table and the composition structure data of the second summary table to respectively determine the first matrix and the second matrix.

[0052] In order to more accurately obtain the composition structure data, in one embodiment, respectively obtaining the composition structure data of the first summary table and the composition structure data of the second summary table may include: obtaining the generation logic information and processing SQL information of the summary table; wherein the summary table includes the first summary table and the second summary table; based on the generation logic information and the processing SQL information, respectively obtaining the composition structure data of the first summary table and the composition structure data of the second summary table.

[0053] In order to more accurately obtain the basic tables and fields, in one embodiment, based on generating logic information and processing SQL information, the composition structure data of the first summary table and the composition structure data of the second summary table are respectively obtained, including: based on generating logic information and processing SQL information, using a first regular expression to respectively obtain the basic tables of the first summary table and the basic tables of the second summary table; based on generating logic information and processing SQL information, using a second regular expression to respectively obtain the fields of the first summary table and the fields of the second summary table.

[0054] After the first matrix and the second matrix are obtained, step S103 needs to be performed in order to determine redundant data.

[0055] S103: Determine a first similarity between the first summary table and the second summary table based on the first matrix and the second matrix.

[0056] In order to more accurately determine the first similarity between the first summary table and the second summary table, in one embodiment, based on the first matrix and the second matrix, determining the first similarity between the first summary table and the second summary table can include: performing a difference operation on the first matrix and the second matrix to obtain a third matrix; based on the third matrix, determining the Euclidean norm value corresponding to the third matrix; and determining the first similarity between the first summary table and the second summary table according to the Euclidean norm value.

[0057] S104. When it is determined that the first similarity reaches a similarity threshold, based on a preset mapping relationship, a first data redundancy processing operation corresponding to the first similarity is performed on the database; wherein the preset mapping relationship is a mapping relationship between similarity and data redundancy processing operation.

[0058] The data redundancy processing method for a database provided by an embodiment of the present invention converts a first summary table pre-stored in a database into a first matrix and a second summary table into a second matrix based on preset matrix conversion strategy information; based on the first matrix and the second matrix, a first similarity between the first summary table and the second summary table can be more accurately determined; therefore, when it is determined that the first similarity reaches a similarity threshold, based on a preset mapping relationship, a first data redundancy processing operation corresponding to the first similarity is performed on the database, which can improve the data redundancy processing effect of the database.

[0059] The following is an example to illustrate the above content, as follows:

[0060] Step 1: Find out the lineage of the summary table, that is, the basic tables and fields used to process the summary table.

[0061] According to the SQL statement for processing the summary table, the string after the from keyword is filtered out through a regular expression to obtain the table name of the basic table. The string between select and from is obtained through a regular expression to obtain the field name of the basic table, and the table name and field name are saved separately.

[0062] Step 2: Represent the basic tables and fields used in the processing summary table in matrix form.

[0063] Assume that in the database, m is the number of basic tables, and n is the maximum number of basic table fields in the database. If the fields of summary table a are derived from the jth field of the ith basic table, then a can be used. ij Indicates that aij The value of represents how many are composed of the jth field of the i-th basic table. The matrix is ​​represented as follows:

[0064]

[0065] For example, the AA summary table structure is shown in Table 1:

[0066] Table 1

[0067]

[0068]

[0069] The matrix corresponding to the AA summary table can be expressed as:

[0070]

[0071] For example, the BB summary table structure is shown in Table 2:

[0072] Table 2

[0073] Field 1 This field comes from field 3 of table 3 <![CDATA[aa 33 =1;]]> Field 2 This field comes from field 4 of table 3. <![CDATA[aa 43 =1;]]> Field 3 This field comes from fields 3 and 4 of table 2. <![CDATA[aa 32 =1;aa 42 =1]]> Field 4 This field comes from field 1 of table 2. <![CDATA[aa 12 =1;]]> Field 5 This field comes from field 3 of table 2. <![CDATA[aa 32 =1;]]> Field 6 This field comes from field 4 of table 2. <![CDATA[aa 42 =1;]]> Field 7 This field comes from field 2 of table 4 <![CDATA[aa 24 =1;]]> Field 8 This field comes from field 3 of table 4 <![CDATA[aa 34 =1;]]>

[0074] The matrix corresponding to the BB summary table can be expressed as:

[0075]

[0076] Step 3: Calculate the similarity between the two summary tables.

[0077] From step 1 and step 2, we can get the matrix representation of the two tables. We can perform a difference operation on the two matrices to get a new matrix A. We can find the L2 norm (also known as the Euclidean norm) of this new matrix, which is the formula:

[0078]

[0079] Where λ1 is A T The maximum eigenvalue of A, that is, the square root of the maximum eigenvalue of the A'A matrix. If the Euclidean norm value is smaller, it means that the two matrices are more similar, that is, the two tables are more similar. If the Euclidean norm value is 0, then the two tables are exactly the same.

[0080] Figure 2 FIG. 1 is a flow chart of another method for processing data redundancy in a database provided by an embodiment of the present invention. Figure 2As shown, first extract the blood relationship table from summary table A, for example, extract the first field of table aa, the second field of table bb, the third field of table cc, etc. from summary table A, and then convert them into a matrix form, that is, the matrix corresponding to summary table A. Correspondingly, perform the same operation on summary table B to obtain the matrix corresponding to summary table B. Then perform a difference operation on the matrix corresponding to summary table A and the matrix corresponding to summary table B to obtain a new matrix, calculate the L2 norm of the new matrix, and judge the similarity between summary table A and summary table B based on the L2 norm. The smaller the L2 norm value, the more similar summary table A and summary table B are.

[0081] The method for processing database data redundancy provided by the embodiment of the present invention has the following beneficial effects:

[0082] First, it is highly versatile and applicable to all types of large databases. Faced with numerous analysis needs, it can effectively avoid stovepipe-style duplicate development and save development and maintenance costs. Second, by reducing the redundancy of processed and summarized data through the easy-to-calculate matrix L2 norm, it can effectively reduce database storage and computing resources, thereby saving system resources. Finally, the calculation is simple, and it only calculates the matrix L2 norm value between source tables based on the blood relationship of the data, without the need to compare and calculate the heavy data content.

[0083] Figure 3 is a schematic diagram of the structure of a data redundancy processing device for a database provided by an embodiment of the present invention. Figure 3 As shown, the data redundancy processing device of the database may include:

[0084] An acquisition module 301 is used to acquire a first summary table and a second summary table pre-stored in a database;

[0085] A conversion module 302, configured to convert the first summary table into a first matrix and the second summary table into a second matrix based on preset matrix conversion strategy information;

[0086] A determination module 303, configured to determine a first similarity between the first summary table and the second summary table based on the first matrix and the second matrix;

[0087] The execution module 304 is used to execute a first data redundancy processing operation corresponding to the first similarity on the database based on a preset mapping relationship when it is determined that the first similarity reaches a similarity threshold; wherein the preset mapping relationship is a mapping relationship between similarity and data redundancy processing operation.

[0088] Optionally, in one embodiment, the conversion module 302 is used to respectively obtain the composition structure data of the first summary table and the composition structure data of the second summary table; wherein the composition structure data includes a basic table and fields; based on the matrix conversion strategy information, the composition structure data of the first summary table and the composition structure data of the second summary table are used to respectively determine the first matrix and the second matrix.

[0089] Optionally, in one embodiment, the conversion module 302 is used to obtain generation logic information and processing SQL information of the summary table; wherein the summary table includes a first summary table and a second summary table; based on the generation logic information and the processing SQL information, the composition structure data of the first summary table and the composition structure data of the second summary table are respectively obtained.

[0090] Optionally, in one embodiment, the conversion module 302 is used to obtain the basic table of the first summary table and the basic table of the second summary table respectively based on the generated logic information and the processed SQL information using a first regular expression; and to obtain the fields of the first summary table and the fields of the second summary table respectively based on the generated logic information and the processed SQL information using a second regular expression.

[0091] Optionally, in one embodiment, the determination module 303 is used to perform a difference operation on the first matrix and the second matrix to obtain a third matrix; based on the third matrix, determine the Euclidean norm value corresponding to the third matrix; and determine the first similarity between the first summary table and the second summary table according to the Euclidean norm value.

[0092] Figure 3 Each module in the device shown has the function of realizing Figure 1 The functions of each step in the process can achieve the corresponding technical effects, which will not be described in detail here for the sake of brevity.

[0093] Figure 4 A schematic structural diagram of an electronic device provided by an embodiment of the present invention is shown.

[0094] The electronic device may include a processor 401 and a memory 402 storing computer program instructions.

[0095] Specifically, the processor 401 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.

[0096] Memory 402 may include a large capacity memory for data or instructions. By way of example and not limitation, memory 402 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In one example, memory 402 may include a removable or non-removable (or fixed) medium, or memory 402 is a non-volatile solid-state memory. Memory 402 may be inside or outside the electronic device.

[0097] In one example, the memory 402 may be a read-only memory (ROM). In one example, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0098] The processor 401 reads and executes the computer program instructions stored in the memory 402 to implement Figure 1 The method for processing data redundancy in the database in the embodiment shown in the figure can achieve Figure 1 The corresponding technical effects achieved by executing the method in the example shown are not repeated here for the sake of brevity.

[0099] In one example, the electronic device may further include a communication interface 403 and a bus 410. Figure 4 As shown, the processor 401, the memory 402, and the communication interface 403 are connected via a bus 410 and communicate with each other.

[0100] The communication interface 403 is mainly used to implement the communication between the modules, devices, units and / or equipment in the embodiment of the present invention.

[0101] Bus 410 includes hardware, software or both, and the components of electronic equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (Accelerated Graphics Port, AGP) or other graphics bus, enhanced industry standard architecture (Extended Industry Standard Architecture, EISA) bus, front side bus (Front Side Bus, FSB), Hyper Transport (Hyper Transport, HT) interconnection, industry standard architecture (Industry Standard Architecture, ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 410 may include one or more buses. Although the embodiment of the present invention describes and shows a specific bus, the present invention considers any suitable bus or interconnection.

[0102] In addition, the embodiment of the present invention can be implemented by providing a computer storage medium. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, Figure 1 The data redundancy processing method of the database shown.

[0103] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0104] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), appropriate firmware, plug-in, function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (Radio Frequency, RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0105] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.

[0106] The above is only a specific implementation of the present invention. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system, module and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be covered within the protection scope of the present invention.

Claims

1. A method for processing data redundancy in a database, characterized in that: include: Obtain a first summary table and a second summary table pre-stored in a database; Based on preset matrix conversion strategy information, the first summary table is converted into a first matrix, and the second summary table is converted into a second matrix; Determining a first similarity between the first summary table and the second summary table based on the first matrix and the second matrix; When it is determined that the first similarity reaches a similarity threshold, based on a preset mapping relationship, a first data redundancy processing operation corresponding to the first similarity is performed on the database; wherein the preset mapping relationship is a mapping relationship between similarity and data redundancy processing operation; The step of converting the first summary table into a first matrix and the second summary table into a second matrix based on the preset matrix conversion strategy information includes: Respectively obtain the composition structure data of the first summary table and the composition structure data of the second summary table; wherein the composition structure data includes a basic table and fields; Based on the composition structure data of the first summary table and the composition structure data of the second summary table, the first matrix and the second matrix are determined respectively, and the matrix expressions of the first matrix and the second matrix are as follows: Among them, m represents the number of basic tables, n represents the maximum number of basic table fields in the database, and a ij Indicates that the fields of the summary table come from the jth field of the ith basic table, a ij The value of indicates the number of fields in the summary table that are composed of the jth field in the i-th basic table; The determining, based on the first matrix and the second matrix, a first similarity between the first summary table and the second summary table comprises: Performing a difference operation on the first matrix and the second matrix to obtain a third matrix; Based on the third matrix, determining a Euclidean norm value corresponding to the third matrix; The first similarity between the first summary table and the second summary table is determined according to the Euclidean norm value.

2. A data redundancy processing device for a database, characterized in that: include: An acquisition module, used for acquiring a first summary table and a second summary table pre-stored in a database; A conversion module, configured to convert the first summary table into a first matrix and the second summary table into a second matrix respectively based on preset matrix conversion strategy information; A determination module, configured to determine a first similarity between the first summary table and the second summary table based on the first matrix and the second matrix; an execution module, configured to execute, when it is determined that the first similarity reaches a similarity threshold, a first data redundancy processing operation corresponding to the first similarity for the database based on a preset mapping relationship; wherein the preset mapping relationship is a mapping relationship between similarity and data redundancy processing operation; The conversion module is specifically used to respectively obtain the composition structure data of the first summary table and the composition structure data of the second summary table; wherein the composition structure data includes a basic table and a field; based on the composition structure data of the first summary table and the composition structure data of the second summary table, respectively determine the first matrix and the second matrix, and the matrix expressions of the first matrix and the second matrix are as follows: Among them, m represents the number of basic tables, n represents the maximum number of basic table fields in the database, and a ij Indicates that the fields of the summary table come from the jth field of the ith basic table, a ij The value of indicates the number of fields in the summary table that are composed of the jth field in the i-th basic table; The determination module is specifically used to perform a difference operation on the first matrix and the second matrix to obtain a third matrix; based on the third matrix, determine the Euclidean norm value corresponding to the third matrix; and according to the Euclidean norm value, determine the first similarity between the first summary table and the second summary table.

3. An electronic device, characterized in that: The electronic device comprises: a processor, and a memory storing computer program instructions; The processor reads and executes the computer program instructions to implement the data redundancy processing method for a database as claimed in claim 1.

4. A computer storage medium, characterized in that: The computer storage medium stores computer program instructions, and when the computer program instructions are executed by the processor, the method for processing database data redundancy as claimed in claim 1 is implemented.

Images