A data item pairing method, device and storage medium

Abstract

The application relates to a data item matching method, device and storage medium, wherein the data item matching method comprises the following steps: obtaining data items to be matched from a data table to be matched; calculating text similarity between the data items to be matched; clustering the data items to be matched according to the calculated text similarity; and matching the data items to be matched according to the clustering result. Through the application, the text similarity between the data items to be matched is mined, the text similarity between the data items to be matched is utilized, the data items to be matched are clustered first, for each type of the clustered data items, only a limited number of data items need to be matched, and the matching work of all the data items can be completed, so that the data item matching work efficiency is greatly improved, and the problem of low data item matching efficiency in the prior art is solved.

Description

Technical Field

[0001] This application relates to the field of data processing, and in particular to a data item matching method, apparatus and storage medium. Background Technology

[0002] With the popularization and development of internet technology, massive amounts of data are constantly emerging from our lives. The development of big data and artificial intelligence technologies, based on distributed data storage and computing, has provided the foundation and application scenarios for the use of massive amounts of data. To enable users to more easily extract value from massive amounts of data, technologies such as data organization and processing, and data asset management have also received widespread attention. Among these, automated data governance technologies, such as data element benchmarking, have played a significant role in improving the quality and reducing the cost of data governance.

[0003] Existing data element benchmarking technologies require benchmarking a large number of data items against data elements, which typically takes a significant amount of time, resulting in low data element benchmarking efficiency.

[0004] There is currently no effective solution to the problem of low data element benchmarking efficiency in existing technologies. Summary of the Invention

[0005] This embodiment provides a data item matching method, apparatus, and storage medium to solve the problem of low data element matching efficiency in the prior art.

[0006] Firstly, this embodiment provides a data item matching method, the method comprising:

[0007] Obtain the data items to be benchmarked from the data table to be benchmarked;

[0008] Calculate the text similarity between the data items to be compared;

[0009] Based on the calculated text similarity, the data items to be compared are clustered;

[0010] Based on the clustering results, the data items to be compared are compared.

[0011] In some embodiments, calculating the text similarity between the data items to be compared includes:

[0012] The data items to be benchmarked are segmented into words to obtain a segmented list of the data items to be benchmarked.

[0013] Based on the word segmentation list, construct the content word segmentation set of the data item to be targeted;

[0014] Based on the content segmentation set, the text similarity between the data items to be compared is calculated.

[0015] In some embodiments, constructing the content segmentation set of the data item to be targeted based on the segmentation list includes:

[0016] The word segments in the word segmentation list are deduplicated, and the content word segmentation set of the data item to be targeted is constructed based on the deduplicated word segmentation list.

[0017] In some embodiments, clustering the data items to be compared based on the calculated text similarity includes:

[0018] Based on the calculated text similarity and the preset similarity threshold, the data items to be compared are clustered.

[0019] In some embodiments, the step of clustering the data items to be compared based on the calculated text similarity and a preset similarity threshold includes:

[0020] Based on the calculated text similarity and the preset similarity threshold, select data items from the data items to be compared that have a text similarity greater than the preset similarity threshold as data item relationship pairs; classify the data item relationship pairs into the same cluster set;

[0021] Update the cluster set based on the data items to be compared in the cluster set.

[0022] In some embodiments, updating the cluster set based on the data item to be matched in the cluster set includes:

[0023] The data items to be classified are classified based on the text similarity between the data items to be classified and the data items to be classified in the cluster set, so as to update the cluster set.

[0024] In some embodiments, the step of benchmarking the data items to be benchmarked based on the clustering results includes:

[0025] Select one data item from the cluster set to be compared;

[0026] Based on the benchmarking results, complete the benchmarking of all data items to be benchmarked.

[0027] In some embodiments, the data item to be compared is a text-type data item or a character-type data item in the data table to be compared.

[0028] Secondly, this embodiment provides a data item matching device, the device comprising:

[0029] The acquisition module is used to obtain the data items to be benchmarked from the data table to be benchmarked;

[0030] The calculation module is used to calculate the text similarity between the data items to be compared;

[0031] The classification module is used to cluster the data items to be compared based on the calculated text similarity.

[0032] The benchmarking module is used to benchmark the data items to be benchmarked based on the clustering results.

[0033] Thirdly, this embodiment provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the data item matching method described in the first aspect.

[0034] Compared with the prior art, the data item matching method, apparatus and storage medium provided in this embodiment greatly improves the efficiency of data item matching by mining the text similarity between the content of data items and using the text similarity between the data items to be matched to first cluster the data items to be matched. For each clustered data item, only a limited number of data items need to be matched to complete the matching of all data items.

[0035] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0036] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0037] Figure 1 This is a hardware structure block diagram of the terminal that executes the data item matching method of this embodiment;

[0038] Figure 2 This is a flowchart of the data item matching method in this embodiment;

[0039] Figure 3 This is a flowchart of a data item matching method according to a preferred embodiment of the present invention;

[0040] Figure 4 This is a structural block diagram of a data item matching device in this embodiment. Detailed Implementation

[0041] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0042] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning as understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these,” used in this application, do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to such processes, methods, products, or devices. The terms “connected,” “linked,” and “coupled,” used in this application, are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. The term “multiple” used in this application refers to two or more. The "and / or" operator describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: A alone, A and B simultaneously, and B alone. Typically, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," and "third," etc., used in this application are merely for distinguishing similar objects and do not represent a specific ordering of the objects.

[0043] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal executing the data item matching method of this embodiment. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0044] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the data item matching method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0045] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0046] This embodiment provides a data item benchmarking method. Figure 2 This is a flowchart of the data item matching method in this embodiment, such as... Figure 2 As shown, the process includes the following steps:

[0047] Step S210: Obtain the data items to be benchmarked from the data table to be benchmarked.

[0048] Specifically, the data is stored in the form of data tables, which include data items of various types. To benchmark text-type or character-type data items, it is necessary to obtain the data items to be benchmarked from the data table, which are of type text or character.

[0049] Step S220: Calculate the text similarity between the data items to be compared.

[0050] Specifically, based on the data items to be compared obtained in step S210, the text similarity between each pair of data items to be compared is calculated. The data items to be compared are text-type data items or character-type data items, and a text similarity algorithm can be used to calculate the similarity between each pair of data items to be compared.

[0051] Step S230: Cluster the data items to be compared based on the calculated text similarity.

[0052] Specifically, based on the text similarity between each pair of data items to be compared calculated in step S220, the data items to be compared are clustered. This clustering involves setting a preset similarity threshold, and then clustering the data items to be compared based on the calculated text similarity and the preset similarity threshold. In other words, the data items to be compared are classified according to the calculated text similarity and the preset similarity threshold, and classified into a finite number of cluster sets.

[0053] Step S240: Based on the clustering results, perform benchmarking on the data items to be benchmarked.

[0054] Specifically, based on the clustering results of step S230, the data items to be compared are compared. More specifically, any data item to be compared in the cluster set obtained in step S230 is compared, and after the comparison of any data item to be compared is completed, other data items in the cluster set are compared based on the comparison result of the data item to be compared.

[0055] In this embodiment, by mining the text similarity between the contents of data items, and using the text similarity between the data items to be compared, the data items to be compared are first clustered. For each clustered data item, only a limited number of data items need to be compared to complete the comparison of all data items, which greatly improves the efficiency of data element comparison.

[0056] In some embodiments, calculating the text similarity between the data items to be compared includes: performing word segmentation on the data items to be compared to obtain a word segmentation list of the data items to be compared; constructing a content word segmentation set of the data items to be compared based on the word segmentation list; and calculating the text similarity between the data items to be compared based on the content word segmentation set.

[0057] Specifically, the word segments in the word segmentation list are deduplicated, and a content word segmentation set for the data items to be compared is constructed based on the deduplicated word segmentation list. For the data items to be compared, text preprocessing such as word segmentation and stop word removal is performed to obtain a word segmentation list for each data item. This step of the word segmentation list does not need to be deduplicated, meaning that some word segments may appear more than once in the word segmentation list. For the word segmentation list obtained for each data item, deduplication is performed to construct a content word segmentation set for each data item, ensuring that each word in the set is unique. Based on the content word segmentation set of each data item to be compared, the text similarity between each pair of data items to be compared is calculated.

[0058] In some of these embodiments, clustering the data items to be compared based on the calculated text similarity includes: clustering the data items to be compared based on the calculated text similarity and a preset similarity threshold.

[0059] Specifically, based on the calculated text similarity and the preset similarity threshold, data items with text similarity greater than the preset similarity threshold are selected from the data items to be compared as data item relationship pairs, and these data item relationship pairs are classified into the same cluster set.

[0060] More specifically, based on the calculated text similarity between each pair of data items to be compared and a preset similarity threshold, data items with text similarity greater than the preset similarity threshold are selected as data item relationship pairs, and the data items corresponding to these data item relationship pairs are classified into the same cluster set. These data item relationship pairs can include two or more data items to be compared. For example, when the text similarity between any pair of three data items to be compared is greater than or equal to the preset similarity threshold, these three data items constitute a data item relationship pair, and the three data items corresponding to this data item relationship pair are classified into the same cluster set. When the text similarity between the data item A to be classified and any data item in any constructed cluster set is less than a preset similarity threshold, the similarity between the data item A to be classified and any unclassified data item to be classified is calculated. When the text similarity between the data item A to be classified and the data item B to be classified is greater than or equal to the preset similarity threshold, a new cluster set A is constructed, and the data items A and B to be classified are classified into cluster set A to perform the classification of the data items A and B to be classified.

[0061] In some of these embodiments, the cluster set is updated based on the data items to be matched in the cluster set.

[0062] Specifically, the data item to be classified is classified based on the text similarity between the data item to be classified and the data item to be classified in the cluster set, thereby updating the cluster set. For example, when the text similarity between the data item to be classified and the data item to be classified in cluster set A is greater than or equal to a preset similarity threshold, the data item to be classified is classified into cluster set A, thereby updating the cluster set.

[0063] In some of these embodiments, benchmarking the data items to be benchmarked based on the clustering results includes: selecting one data item from the cluster set to be benchmarked; and completing the benchmarking of all data items to be benchmarked based on the benchmarking results.

[0064] Specifically, based on the cluster set generated in the aforementioned embodiments, any data item to be matched in cluster set B is selected for matching to obtain the data element corresponding to that data item. Based on the matching result of any data item, the matching of all data items to be matched in cluster set B is completed, that is, all data items to be matched in cluster set B are matched to the data element corresponding to that data item, thus completing the matching of all data items to be matched in cluster set B. All cluster sets obtained by classifying all data items to be matched are then matched using the aforementioned matching method for data items to be matched in cluster set B, thus completing the matching of all data items to be matched in all cluster sets.

[0065] In this embodiment, by mining the text similarity between the contents of data items, and using the text similarity between the data items to be compared, the data items to be compared are first clustered. For each clustered data item, only a limited number of data items need to be compared to complete the comparison of all data items, which greatly improves the efficiency of data element comparison.

[0066] The present embodiment will now be described and illustrated through preferred embodiments.

[0067] Figure 3 This is a flowchart of a data item matching method according to a preferred embodiment of this invention. Figure 3 As shown, the data item benchmarking method includes the following steps:

[0068] Step S310, data preprocessing.

[0069] Specifically, before conducting the data benchmarking task, it is necessary to first catalog the data items to be benchmarked, filter out text data items or character data items, and perform text preprocessing techniques such as word segmentation and stop word removal on the content of each text data item or character data item to obtain a word segmentation list for each data item.

[0070] This is to facilitate subsequent text similarity calculations. Step S310 includes the following steps:

[0071] Step S311: Extract text data items.

[0072] Specifically, for a data element benchmarking project, it is necessary to first organize the data items of the data table to be benchmarked, filter out the data items of the character type (or the data items of the text type), and catalog and record the table name and data item name, as shown in Table 1, so as to make it easier to find similar data items of the same category after subsequent clustering.

[0073] Table 1 Data Item Catalog

[0074] Data table name Data item name Data table number Data item number ods_ryckxx_ds gmsfhm 01 01 dwd_ryckxx_ds gmsfhm 02 03

[0075] Step S312, text preprocessing.

[0076] Specifically, for the text data items that have been numbered and recorded in step S311, a word segmentation list is obtained for each data item through text preprocessing techniques such as word segmentation and stop word removal. This word segmentation list does not need to be deduplicated, meaning that some words may appear more than once in the word segmentation list.

[0077] Step S313: Construct a content segmentation set.

[0078] Specifically, for each data item's word segmentation list obtained in step S312, a deduplication operation is performed to construct a content word segmentation set for each data item, ensuring that each word segmentation in the set is unique.

[0079] Step S320: Calculate text similarity.

[0080] Specifically, after obtaining the word segmentation set of each data item according to step S310, it is necessary to calculate the similarity of each data item's word segmentation set pairwise using a text similarity algorithm. The following are three recommended text similarity algorithms: 1. Jaccard Similarity Algorithm: This algorithm calculates the similarity between two sets A and B by finding the ratio of the number of elements in their intersection to the number of elements in their union. This algorithm is relatively easy to understand, has a simple calculation process, and provides accurate results, but its computational performance is relatively poor. It is suitable for use when the set data is relatively small. 2. MiniHash Minimum Hash Algorithm: This algorithm requires constructing a 0-1 matrix of word vectors for each data item set and randomly shuffling it multiple times to obtain the minimum hash signature matrix. Then, the similarity of each data item is calculated. This algorithm is relatively complex, and its calculation results are not as accurate as the Jaccard similarity algorithm, but its computational performance is higher than the Jaccard similarity algorithm. It is more suitable for use when the set data is large than the Jaccard similarity algorithm. 3. Locality Sensitive Hash Algorithm (LSH): This algorithm divides the minimum hash signature matrix obtained by the miniHash algorithm into b rows, performs hash mapping on each row, and finally calculates the similarity. This algorithm is relatively complex and the calculation results are not as accurate as the miniHash algorithm, but it has the highest computational performance and is more suitable for use when the set of data is large.

[0081] Based on the above text similarity algorithm, we obtain the text similarity between each pair of data items. We record the data item number and similarity of the data items for which similarity calculation was performed, and obtain a data item similarity record table, as shown in Table 2.

[0082] Table 2 Data Item Similarity Record Table

[0083] Data table number 1 Data item number 1 Data Table No. 2 Data item number 2 Similarity 01 01 01 02 0.9846 01 01 01 03 0.7685

[0084] Step S330: Cluster according to similarity.

[0085] Specifically, the similarity between the data items obtained in step S320 can be set to a threshold minSimilarity (a decimal between 0 and 1). Data item relationship pairs with similarity greater than this threshold are filtered out. Based on these data item relationship pairs, the data items can be clustered. Specifically, if there is a relationship pair between data item 1 and data item 2, then data item 1 and data item 2 belong to the same set. Through these data item relationship pairs, the data item set will be continuously expanded and merged. Eventually, all the data items to be compared will be covered by several finite sets. Each data item set has the same set number until all data items are divided into one set and have a set number, as shown in Table 3.

[0086] Table 3. Data Item Set Relationship Table

[0087] Data table number Data item number Set Number 01 01 01 01 02 02

[0088] Ideally, assuming there are 10,000 data items to be matched, aggregated into 100 sets, and the matching result for each data item in each set is the same data element, then only the data element of any one data item in the 100 sets needs to be matched to complete the matching of all 10,000 data items, reducing the workload by two orders of magnitude. However, in reality, when the threshold `minSimilarity` is set too low, the model cannot completely guarantee that all data items in each set are absolutely matched to the same data element. For example, if the threshold `minSimilarity` is set too low, it may lead to a large number of data items in the same set actually matching different data elements. To avoid or reduce this situation, a small batch of datasets can be used for testing first, and the value of the threshold `minSimilarity` can be adjusted to reduce the model error rate to an acceptable range. Here, `data element` is a fundamental concept in the field of data governance, which can be understood as a standard description of the same thing defined by humans.

[0089] Step S340, data element benchmarking.

[0090] Based on the data item set relationship table obtained in step S330, after any data item in the set is benchmarked, other data items in the same set can also use the benchmarked data element. For example, if there are N data elements to be benchmarked, they are aggregated into M sets through steps S310 to S330. Only the data element of any data item in the M sets needs to be benchmarked to complete the benchmarking work of N data items, reducing the workload by N / M orders of magnitude.

[0091] In this embodiment, by mining the text similarity between the contents of data items, and using the text similarity between the data items to be compared, the data items to be compared are first clustered. For each clustered data item, only a limited number of data items need to be compared to complete the comparison of all data items, which greatly improves the efficiency of data element comparison.

[0092] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0093] This embodiment also provides a data item matching device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0094] Figure 4 This is a structural block diagram of a data item matching device according to this embodiment, such as... Figure 4 As shown, the device includes:

[0095] The acquisition module 410 is used to obtain the data items to be benchmarked from the data table to be benchmarked.

[0096] The calculation module 420 is used to calculate the text similarity between the data items to be compared.

[0097] The classification module 430 is used to cluster the data items to be compared based on the calculated text similarity.

[0098] The benchmarking module 440 is used to benchmark the data items to be benchmarked based on the clustering results.

[0099] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0100] This embodiment also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0101] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0102] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0103] S1, Obtain the data items to be benchmarked from the data table to be benchmarked;

[0104] S2, calculate the text similarity between the data items to be compared;

[0105] S3, based on the calculated text similarity, cluster the data items to be compared;

[0106] S4. Based on the clustering results, perform benchmarking on the data items to be benchmarked.

[0107] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0108] Furthermore, in conjunction with the data item matching method provided in the above embodiments, this embodiment can also provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements the steps of any of the data item matching methods in the above embodiments.

[0109] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0110] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0111] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0112] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A data item benchmarking method, characterized in that, The method includes: Obtain the data items to be benchmarked from the data table to be benchmarked; Calculate the text similarity between the data items to be compared; Based on the calculated text similarity, the data items to be compared are clustered; Based on the clustering results, the data items to be compared are compared. Calculating the text similarity between the data items to be compared includes: The data items to be benchmarked are segmented into words to obtain a segmented list of the data items to be benchmarked. Based on the word segmentation list, construct the content word segmentation set of the data item to be targeted; Based on the content segmentation set, calculate the text similarity between the data items to be compared; Based on the calculated text similarity, the data items to be compared are clustered, including: Based on the calculated text similarity and the preset similarity threshold, select data items from the data items to be compared that have a text similarity greater than the preset similarity threshold as data item relationship pairs; classify the data item relationship pairs into the same cluster set; Update the cluster set according to the data items to be matched in the cluster set; Based on the clustering results, the data items to be compared are compared, including: For each obtained cluster set, select any data item to be matched in the corresponding cluster set for matching, so as to obtain the data element corresponding to the data item to be matched in the corresponding cluster set. Match all data items to be matched in the corresponding cluster set to the data element corresponding to the data item to be matched in the corresponding cluster set, so as to complete the matching of all data items to be matched in the corresponding cluster set.

2. The data item benchmarking method according to claim 1, characterized in that, The step of constructing the content segmentation set of the data item to be targeted based on the segmentation list includes: The word segments in the word segmentation list are deduplicated, and the content word segmentation set of the data item to be targeted is constructed based on the deduplicated word segmentation list.

3. The data item benchmarking method according to claim 1 or 2, characterized in that, The step of updating the cluster set based on the data items to be compared in the cluster set includes: The data items to be classified are classified based on the text similarity between the data items to be classified and the data items to be classified in the cluster set, so as to update the cluster set.

4. The data item benchmarking method according to claim 1, characterized in that, The data items to be compared are text-type or character-type data items in the data table to be compared.

5. A data item matching device, characterized in that, The device includes: The acquisition module is used to obtain the data items to be benchmarked from the data table to be benchmarked; The calculation module is used to calculate the text similarity between the data items to be compared; The classification module is used to cluster the data items to be compared based on the calculated text similarity. The benchmarking module is used to benchmark the data items to be benchmarked based on the clustering results; The calculation module is further configured to perform word segmentation on the data items to be benchmarked, and obtain a word segmentation list of the data items to be benchmarked; construct a content word segmentation set of the data items to be benchmarked based on the word segmentation list; and calculate the text similarity between the data items to be benchmarked based on the content word segmentation set. The classification module is further configured to, based on the calculated text similarity and a preset similarity threshold, select data items from the data items to be compared that have a text similarity greater than the preset similarity threshold as data item relationship pairs; classify the data item relationship pairs into the same cluster set; and update the cluster set based on the data items to be compared in the cluster set. The benchmarking module is further configured to select any data item to be benchmarked in each obtained cluster set and benchmark it to obtain the data element corresponding to any data item to be benchmarked in the corresponding cluster set. All data items to be benchmarked in the corresponding cluster set are then benchmarked to the data element corresponding to any data item to be benchmarked in the corresponding cluster set, so as to complete the benchmarking of all data items to be benchmarked in the corresponding cluster set.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the data item matching method according to any one of claims 1 to 4.

Images