Multi-source text data intelligent matching method, device, storage medium and equipment

Abstract

The present invention discloses a method, device, storage medium and equipment for intelligent matching of multi-source text data. The method comprises: obtaining first text data corresponding to different information systems, wherein the first text data of each of the above information systems comprises text information for recording the address of the above information system; performing cluster analysis on the above first text data using a hierarchical clustering algorithm to obtain multiple address classes; extracting address feature information of each of the above address classes to obtain multiple address feature information; matching the above first text data corresponding to different above information systems based on the multiple address feature information to obtain matching results. The present invention solves the technical problems of high matching difficulty and low matching accuracy in the text data matching method in the prior art.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, storage medium and equipment for intelligent matching of multi-source text data. Background Art

[0002] Nowadays, the wave of machine learning and big data allows enterprises to conduct business based on their own existing data, but if they want to open up new business scenarios, the existing data of many companies cannot meet the needs, so there is a tendency for enterprises to cooperate and expand their own business based on the data stored by multiple parties. However, with the increasing importance of Internet data security management and data value, the willingness of enterprises to share, the privacy protection of data, and the difficulty of integrating the same information specifications between different enterprises have become obstacles to enterprise cooperation.

[0003] In the prior art, when matching users in the internal information systems of two groups or enterprises, sometimes the user name cannot uniquely identify the user, and address matching is required. In the two information systems, the user address lacks a standard address writing format, resulting in multiple address writing methods for the same location, which increases the difficulty of user address matching. Traditional address matching methods mainly focus on the matching relationship between words in address texts, and cannot accurately match addresses in different expressions of the same location.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a method, apparatus, storage medium and device for intelligent matching of multi-source text data, so as to at least solve the technical problems of high matching difficulty and low matching accuracy in the text data matching method in the prior art.

[0006] According to one aspect of an embodiment of the present invention, a method for intelligent matching of multi-source text data is provided, comprising: acquiring first text data corresponding to different information systems, wherein the first text data of each of the above-mentioned information systems includes text information for recording the address of the above-mentioned information system; performing clustering analysis on the above-mentioned first text data using a hierarchical clustering algorithm to obtain a plurality of address classes; extracting address feature information of each of the above-mentioned address classes to obtain a plurality of the above-mentioned address feature information; matching the above-mentioned first text data corresponding to different above-mentioned information systems based on the plurality of the above-mentioned address feature information to obtain a matching result.

[0007] Optionally, before the hierarchical clustering algorithm is used to perform cluster analysis on the first text data to obtain multiple address classes, the method further includes: preprocessing the first text data, wherein the preprocessing at least includes: deleting the first text data containing only numbers; deleting the first text data with a data volume lower than a preset threshold; deleting characters contained in the first text data that do not comply with preset regulations.

[0008] Optionally, before the hierarchical clustering algorithm is used to perform cluster analysis on the first text data to obtain multiple address classes, the method further includes: transmitting the first text data to a public server; performing word segmentation processing on the first text data using a word segmentation component to obtain a word list; and constructing a bag-of-words model based on the word list.

[0009] Optionally, the above-mentioned hierarchical clustering algorithm is used to perform cluster analysis on the above-mentioned first text data to obtain multiple address classes, including: randomly selecting test samples from the above-mentioned bag-of-words model for cluster analysis, obtaining and recording the number of clusters; and determining multiple address classes based on the above-mentioned cluster number.

[0010] Optionally, the above-mentioned extraction of address feature information of each of the above-mentioned address classes to obtain multiple address feature information includes: using a weighted algorithm to calculate a weight vector for each of the above-mentioned address classes to obtain multiple weight vectors; obtaining a target vector that is greater than a preset weight threshold value from the multiple weight vectors; and using the above-mentioned target vector as the address feature information corresponding to the above-mentioned address class.

[0011] Optionally, the above-mentioned first text data corresponding to different information systems are matched based on the above-mentioned multiple address feature information to obtain a matching result, including: using a hierarchical clustering algorithm to perform cluster analysis on the above-mentioned address feature information to obtain multiple feature classes; obtaining the above-mentioned first text data belonging to multiple different information systems in each of the above-mentioned feature classes to obtain second text data; performing text similarity matching processing on the above-mentioned second text data to obtain a text similarity value; and selecting the above-mentioned second text data with the highest text similarity value as the matching result.

[0012] According to another aspect of an embodiment of the present invention, a multi-source text data intelligent matching device is also provided, including: an acquisition module, used to acquire first text data corresponding to different information systems, wherein the first text data of each of the above-mentioned information systems includes text information for recording the address of the above-mentioned information system; an analysis module, used to perform clustering analysis on the above-mentioned first text data using a hierarchical clustering algorithm to obtain multiple address classes; an extraction module, used to extract address feature information of each of the above-mentioned address classes to obtain multiple address feature information; a matching module, used to match the above-mentioned first text data corresponding to different above-mentioned information systems based on the multiple address feature information to obtain a matching result.

[0013] According to another aspect of an embodiment of the present invention, a non-volatile storage medium is provided, wherein the non-volatile storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing any one of the multi-source text data intelligent matching methods.

[0014] According to another aspect of an embodiment of the present invention, a processor is further provided, and the processor is used to run a program, wherein the program is configured to execute any one of the multi-source text data intelligent matching methods when running.

[0015] According to another aspect of an embodiment of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute any one of the multi-source text data intelligent matching methods described above.

[0016] In an embodiment of the present invention, by obtaining first text data corresponding to different information systems, wherein the first text data of each of the above-mentioned information systems includes text information for recording the address of the above-mentioned information system; clustering analysis is performed on the above-mentioned first text data using a hierarchical clustering algorithm to obtain multiple address classes; address feature information of each of the above-mentioned address classes is extracted to obtain multiple address feature information; and matching results are obtained by matching the above-mentioned first text data corresponding to different above-mentioned information systems based on the multiple address feature information. The purpose of realizing text data clustering analysis and address feature extraction is achieved under the premise of ensuring data security, thereby achieving the technical effect of safely and efficiently matching text data, and further solving the technical problems of high matching difficulty and low matching accuracy in the text data matching method in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0018] Figure 1 is a flow chart of a multi-source text data intelligent matching method according to an embodiment of the present invention;

[0019] Figure 2 is a schematic diagram of an optional text data matching method according to an embodiment of the present invention;

[0020] Figure 3 is a schematic diagram of an optional clustering index graph according to an embodiment of the present invention;

[0021] Figure 4 It is a structural schematic diagram of a multi-source text data intelligent matching device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0024] Example 1

[0025] According to an embodiment of the present invention, an embodiment of a method for intelligent matching of multi-source text data is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0026] Figure 1 is a flow chart of a multi-source text data intelligent matching method according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0027] Step S102, obtaining first text data corresponding to different information systems, wherein the first text data of each of the above information systems includes text information for recording the address of the above information system;

[0028] Step S104, using a hierarchical clustering algorithm to perform cluster analysis on the first text data to obtain multiple address classes;

[0029] Step S106, extracting address feature information of each of the above address classes to obtain a plurality of the above address feature information;

[0030] Step S108: matching the first text data corresponding to different information systems based on the plurality of address feature information to obtain a matching result.

[0031] In an embodiment of the present invention, the executor of the multi-source text data intelligent matching method provided in the above steps S102 to S108 is a data matching system, which uses the above data matching system to obtain first text data corresponding to different information systems, and uses a hierarchical clustering algorithm and a federated learning method to perform cluster analysis on the above first text data to obtain multiple address classes; extract address feature information of each of the above address classes to obtain multiple address feature information; match the above first text data corresponding to different above information systems with the multiple address feature information in combination with text similarity to obtain a matching result.

[0032] It should be noted that the above multi-source text data intelligent matching method is usually applied to text data matching between two or more different information systems; the data content and data type of the above text data are not specifically limited, for example: address data. In the prior art, matching is performed based on the similarity of address texts in two information systems, but the accuracy of this method is low, and the security of data privacy cannot be guaranteed when the two parties exchange data.

[0033] In an embodiment of the present invention, a method combining text similarity and federated learning is used to extract text features and perform final address text matching based on address characteristics while ensuring data security, thereby effectively solving the problems of data privacy leakage and low address matching rate and achieving matching of user addresses of two systems.

[0034] It should be noted that the essence of the above-mentioned federated learning is a distributed machine learning, which is a combination of samples or features. In vertical federated learning, the user overlap of enterprises is high and the feature overlap is small. The modeling process requires that there is an intersection between enterprise users and that one party needs to have a label.

[0035] It should also be noted that the first text data of each of the above-mentioned information systems includes text information for recording the address of the above-mentioned information system.

[0036] As an optional embodiment, Figure 2 The schematic diagram of the text data matching method shown in the figure takes the address data matching process of the information systems of two different enterprises as an example. After receiving the address data, the above data matching system first pre-processes the data and deletes the missing values ​​and addresses that do not meet the requirements in the addresses of the two systems to ensure that subsequent processing is not interfered with; then the address texts of the two systems are transmitted to the public server in an encrypted manner, and then the word segmentation component is used for word segmentation, and then the word column and the bag-of-words model are constructed using the segmented vocabulary; the addresses of the two systems are clustered separately using the hierarchical clustering method, and features are extracted for each category, and the extracted features are clustered; addresses belonging to two different systems are found in each category, and the address data uploaded by both parties are obtained through data decryption, and then the two groups of addresses are matched for text similarity on the public server to obtain the address with the highest similarity in the two groups of addresses.

[0037] It should be noted that the feature data after feature extraction does not contain privacy data; the similarity can be measured by using the cosine similarity indicator applicable to text data.

[0038] Through the embodiments of the present invention, the address text is processed in stages, and the features are combined for cluster matching, which can effectively avoid data privacy leakage and match the addresses of two enterprises, thereby achieving the purpose of address data clustering analysis and address feature extraction under the premise of ensuring data security, thereby achieving the technical effect of matching address data safely and efficiently, and further solving the technical problems of high matching difficulty and low matching accuracy in the text data matching method in the prior art.

[0039] In an optional embodiment, before the hierarchical clustering algorithm is used to perform cluster analysis on the first text data to obtain multiple address classes, the method further includes:

[0040] Step S202, preprocessing the first text data, wherein the preprocessing at least includes: deleting the first text data containing only numbers; deleting the first text data with a data volume lower than a preset threshold; deleting characters in the first text data that do not comply with preset regulations.

[0041] In the embodiment of the present invention, after the first text data corresponding to different information systems are obtained, the first text data are firstly preprocessed.

[0042] As an optional embodiment, the data preprocessing process mainly consists of the following two sub-steps: deleting data containing only numbers and data with serious data missing, for example: data containing only XX district or XX city; deleting illegal characters contained in illegal character data, such as: "&", "@", etc.

[0043] In an optional embodiment, before the hierarchical clustering algorithm is used to perform cluster analysis on the first text data to obtain multiple address classes, the method further includes:

[0044] Step S302, transferring the first text data to a public server;

[0045] Step S304, using a word segmentation component to perform word segmentation processing on the first text data to obtain a word list;

[0046] Step S306: construct a bag-of-words model based on the above-mentioned entry list.

[0047] As an optional embodiment, the data is encrypted and transmitted to the public server, and the word segmentation component is used to segment the addresses of the two systems respectively, and then a word list is constructed; and a word bag model is constructed based on the above word list.

[0048] It should be noted that, for example, the two addresses "A City, M District" and "A City, N District" become [A City, M District] and [A City, N District] after word segmentation. The entry list composed of these two addresses is [A City, M District, N District], and the corresponding bag-of-words models are [1,1,0] and [1,0,1].

[0049] In an optional embodiment, the hierarchical clustering algorithm is used to perform cluster analysis on the first text data to obtain multiple address classes, including:

[0050] Step S402, randomly selecting test samples from the bag-of-words model to perform cluster analysis, and obtaining and recording the number of clusters;

[0051] Step S404: determining a plurality of address classes based on the above clustering number.

[0052] In the embodiment of the present invention, cluster analysis is an unsupervised machine learning algorithm. According to the characteristics of federated learning, the unsupervised clustering method allows each enterprise to use its own data to conduct autonomous learning and corresponding feature extraction without using the data of the other enterprise. In this way, the extracted features not only reflect the characteristics of the sample data to the greatest extent, but also protect the privacy of its own data accordingly.

[0053] It should be noted that, by adopting this method, it is possible to correspond relatively broad locations such as "road" and "street" to the correct "street" and "community committee", and it is also possible to correctly match the specific "No. XX".

[0054] As an optional embodiment, the average distance is selected as the clustering threshold, and the calculation formula is as follows:

[0055]

[0056] Among them, C i , C j It is the clustering cluster generated in the process of address data clustering; d(C i , C j ) is used to calculate the average distance between the two clusters; x and z represent the address samples in the two clusters respectively.

[0057] As an optional embodiment, hierarchical clustering is performed on the two system addresses respectively, and some test samples are selected for hierarchical clustering. During clustering, DBI (Davies-Bouldin Index) and DI (Dunn Index) clustering quality measurement indicators are recorded to find a suitable number of clusters.

[0058] In the embodiment of the present invention, the selected sample is 1092 data of a group system address, such as Figure 3 The schematic diagram of the clustering index diagram is shown in Figure 1. The DBI and DI indexes are indicators for measuring the quality of the clustering results. The smaller the DBI value, the better, while the larger the DI value, the better. According to the trend of the index diagram, the optimal point of clustering may appear at the step. From the figure, it can be seen that when the number of categories is about 990, the classification effect is the best. Therefore, the number of categories can be determined as 990, and the average distance of the categories is calculated to be 0.118; this distance threshold is used as the standard for the end of clustering, and the 90,950 samples in system 1 and the 1,092 samples in system 2 are clustered, and the number of clusters is 10,953 and 990 respectively.

[0059] In an optional embodiment, the address feature information of each of the address classes is extracted to obtain a plurality of address feature information, including:

[0060] Step S502, using a weighted algorithm to calculate a weight vector for each of the above address classes, to obtain a plurality of the above weight vectors;

[0061] Step S504, obtaining a target vector whose value is greater than a preset weight threshold value among the plurality of weight vectors;

[0062] Step S506: Using the target vector as address feature information corresponding to the address class.

[0063] As an optional embodiment, the weight vector of the vocabulary in each category is obtained through the weighted algorithm TF-IDF (Term Frequency-Inverse Document Frequency) model, and the target vector with a weight higher than 0.08 in each category is obtained as the feature of the category. Meaningless words such as "city" and "district" are eliminated; for example, the extracted features are (A, B, C).

[0064] It should be noted that A above represents a city, B represents a county, and C represents a town or a street (or a building).

[0065] In an optional embodiment, the first text data corresponding to different information systems are matched based on the plurality of address feature information to obtain a matching result, including:

[0066] Step S602, using a hierarchical clustering algorithm to perform cluster analysis on the above address feature information to obtain multiple feature classes;

[0067] Step S604, acquiring the first text data belonging to the plurality of different information systems in each of the feature classes to obtain second text data;

[0068] Step S606, performing text similarity matching processing on the second text data to obtain a text similarity value;

[0069] Step S608: selecting the second text data with the highest text similarity value as a matching result.

[0070] In the embodiment of the present invention, the characteristic classes of the two systems are clustered and analyzed using a hierarchical clustering method, and the clustering steps are consistent with the steps for obtaining the address classes.

[0071] As an optional embodiment, features belonging to different systems are obtained in the same category, and text similarity is calculated from samples represented by the features in the same category. The two addresses with the highest similarity are a pair of addresses that match each other in the two systems.

[0072] It should be noted that in each category, addresses belonging to two different systems are found, and the addresses uploaded by both parties are obtained by decryption. Then, the two sets of addresses are matched on the public server for text similarity to obtain the address with the highest similarity in the two sets of addresses. The similarity can be measured by the cosine similarity indicator applicable to text data. a and b are the word vectors of the two texts after word segmentation. The cosine similarity calculation formula between a and b is as follows:

[0073]

[0074] As an optional embodiment, among the 1092 addresses in system 2, 813 correct system 1 addresses were successfully matched, with an accuracy rate of 75.73%.

[0075] Through the above steps, text data (for example, address data) in two or more systems are matched. While protecting most of the data privacy, the user intersections between multiple systems can be screened only by matching addresses. In the process of address matching, each enterprise can use unsupervised clustering methods to extract the features of the address information of its own system, and then perform clustering learning of multiple data sources for federated learning based on the extracted features. At the same time, the applicability of federated learning itself to unsupervised learning is also improved, and the push feedback of results is completed under the premise of ensuring data privacy.

[0076] Example 2

[0077] According to an embodiment of the present invention, there is also provided an apparatus embodiment for implementing the above-mentioned multi-source text data intelligent matching method. Figure 4 is a schematic diagram of the structure of a multi-source text data intelligent matching device according to an embodiment of the present invention. Figure 4 As shown, the above device includes: an acquisition module 40, an analysis module 42, an extraction module 44 and a matching module 46, wherein:

[0078] An acquisition module 40 is used to acquire first text data corresponding to different information systems, wherein the first text data of each of the above information systems includes text information for recording the address of the above information system;

[0079] An analysis module 42, configured to perform cluster analysis on the first text data using a hierarchical clustering algorithm to obtain a plurality of address classes;

[0080] An extraction module 44 is used to extract address feature information of each of the above address classes to obtain a plurality of the above address feature information;

[0081] The matching module 46 is used to match the first text data corresponding to different information systems based on the plurality of address feature information to obtain a matching result.

[0082] It should be noted here that the above-mentioned acquisition module 40, analysis module 42, extraction module 44 and matching module 46 correspond to steps S102 to S108 in Example 1, and the instances and application scenarios implemented by the four modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned Example 1.

[0083] It should be noted that the preferred implementation of this embodiment can refer to the relevant description in Example 1, which will not be repeated here.

[0084] According to an embodiment of the present invention, an embodiment of a computer-readable storage medium is also provided. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the text data matching method provided in the above embodiment 1.

[0085] Optionally, in this embodiment, the computer-readable storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0086] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: obtaining first text data corresponding to different information systems, wherein the first text data of each of the above-mentioned information systems includes text information for recording the address of the above-mentioned information system; performing clustering analysis on the above-mentioned first text data using a hierarchical clustering algorithm to obtain multiple address classes; extracting address feature information of each of the above-mentioned address classes to obtain multiple address feature information; matching the above-mentioned first text data corresponding to different above-mentioned information systems based on the multiple address feature information to obtain matching results.

[0087] Optionally, the computer-readable storage medium is configured to store program codes for executing the following steps: preprocessing the first text data, wherein the preprocessing at least includes: deleting the first text data containing only numbers; deleting the first text data whose data volume is lower than a preset threshold; deleting characters contained in the first text data that do not comply with preset regulations.

[0088] Optionally, the computer-readable storage medium is configured to store program codes for executing the following steps: transferring the first text data to a public server; performing word segmentation processing on the first text data using a word segmentation component to obtain a word list; and constructing a word bag model based on the word list.

[0089] Optionally, the computer-readable storage medium is configured to store program codes for executing the following steps: randomly selecting test samples from the bag-of-words model for cluster analysis, obtaining and recording the number of clusters; and determining multiple address classes based on the number of clusters.

[0090] Optionally, the computer-readable storage medium is configured to store program code for executing the following steps: using a weighted algorithm to calculate a weight vector for each of the above address classes to obtain a plurality of the above weight vectors; obtaining a target vector that is greater than a preset weight threshold value from the plurality of the above weight vectors; and using the above target vector as address feature information corresponding to the above address class.

[0091] Optionally, the computer-readable storage medium is configured to store program codes for executing the following steps: performing cluster analysis on the address feature information using a hierarchical clustering algorithm to obtain a plurality of feature classes; obtaining the first text data belonging to a plurality of different information systems in each of the feature classes to obtain the second text data; performing text similarity matching processing on the second text data to obtain a text similarity value; and selecting the second text data having the highest text similarity value as the matching result.

[0092] According to an embodiment of the present invention, an embodiment of a processor is also provided. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the text data matching method provided in the above embodiment 1.

[0093] An embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, the following steps are implemented: obtaining first text data corresponding to different information systems, wherein the first text data of each of the above-mentioned information systems includes text information for recording the address of the above-mentioned information system; performing cluster analysis on the above-mentioned first text data using a hierarchical clustering algorithm to obtain multiple address classes; extracting address feature information of each of the above-mentioned address classes to obtain multiple address feature information; matching the above-mentioned first text data corresponding to different above-mentioned information systems based on the multiple address feature information to obtain matching results.

[0094] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that is initialized with the following method steps: obtaining first text data corresponding to different information systems, wherein the first text data of each of the above-mentioned information systems includes text information for recording the address of the above-mentioned information system; performing clustering analysis on the above-mentioned first text data using a hierarchical clustering algorithm to obtain multiple address classes; extracting address feature information of each of the above-mentioned address classes to obtain multiple address feature information; matching the above-mentioned first text data corresponding to different above-mentioned information systems based on the multiple address feature information to obtain matching results.

[0095] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0096] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0097] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0098] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0099] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0100] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0101] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A multi-source text data intelligent matching method, characterized in that: include: Acquire first text data corresponding to different information systems, wherein the first text data of each information system includes text information for recording the address of the information system; Performing cluster analysis on the first text data using a hierarchical clustering algorithm to obtain multiple address classes; Extracting address feature information of each address class to obtain a plurality of address feature information; Matching the first text data corresponding to different information systems based on the plurality of address feature information to obtain a matching result; The step of using a hierarchical clustering algorithm to perform cluster analysis on the first text data to obtain a plurality of address classes includes: using a hierarchical clustering algorithm to perform cluster analysis on the first text data corresponding to each of the different information systems to obtain a plurality of address classes; The method of matching the first text data corresponding to different information systems based on the plurality of address feature information to obtain a matching result includes: performing cluster analysis on the address feature information using a hierarchical clustering algorithm to obtain a plurality of feature classes; obtaining the first text data belonging to a plurality of different information systems in each feature class to obtain second text data; performing text similarity matching processing on the second text data to obtain a text similarity value; and selecting the second text data with the highest text similarity value as the matching result; The performing text similarity matching processing on the second text data to obtain text similarity values ​​includes: performing text similarity matching processing on text data belonging to different information systems in the second text data in pairs to obtain multiple text similarity values; The selecting the second text data with the highest text similarity value as the matching result includes: A group of text data corresponding to the highest text similarity value among the multiple text similarity values ​​is selected as the matching result.

2. The method according to claim 1, characterized in that Before performing cluster analysis on the first text data using a hierarchical clustering algorithm to obtain a plurality of address classes, the method further includes: The first text data is preprocessed, wherein the preprocessing at least includes: deleting the first text data containing only numbers; deleting the first text data whose data volume is lower than a preset threshold; and deleting characters contained in the first text data that do not comply with preset regulations.

3. The method according to claim 1, characterized in that Before performing cluster analysis on the first text data using a hierarchical clustering algorithm to obtain a plurality of address classes, the method further includes: transferring the first text data to a public server; Using a word segmentation component to perform word segmentation processing on the first text data to obtain a word list; A bag-of-words model is constructed based on the list of terms.

4. The method according to claim 3, characterized in that The hierarchical clustering algorithm is used to perform cluster analysis on the first text data to obtain multiple address classes, including: Randomly select test samples from the bag-of-words model to perform cluster analysis, and obtain and record the number of clusters; A plurality of address classes are determined based on the number of clusters.

5. The method according to claim 1, characterized in that The extracting of address feature information of each address class to obtain a plurality of address feature information includes: A weighted algorithm is used to calculate a weight vector of each address class to obtain a plurality of weight vectors; Obtaining a target vector greater than a preset weight threshold value from among the plurality of weight vectors; The target vector is used as address feature information corresponding to the address class.

6. A multi-source text data intelligent matching device, characterized in that: include: An acquisition module, used for acquiring first text data corresponding to different information systems, wherein the first text data of each information system includes text information for recording the address of the information system; An analysis module, configured to perform cluster analysis on the first text data using a hierarchical clustering algorithm to obtain a plurality of address classes; An extraction module, used for extracting address feature information of each address class to obtain a plurality of address feature information; A matching module, used for matching the first text data corresponding to different information systems based on the plurality of address feature information to obtain a matching result; The analysis module is further used to perform cluster analysis on the first text data corresponding to the different information systems respectively using a hierarchical clustering algorithm to obtain a plurality of address classes; The matching module is also used to perform cluster analysis on the address feature information using a hierarchical clustering algorithm to obtain multiple feature classes; obtain the first text data belonging to multiple different information systems in each feature class to obtain second text data; perform text similarity matching processing on the second text data to obtain a text similarity value; and select the second text data with the highest text similarity value as the matching result; The matching module is further used to perform text similarity matching between two text data belonging to different information systems in the second text data to obtain multiple text similarity values; The matching module is further configured to select a group of text data corresponding to the highest text similarity value among the multiple text similarity values ​​as the matching result.

7. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the multi-source text data intelligent matching method according to any one of claims 1 to 5.

8. A processor, characterized in that: The processor is used to run a program, wherein the program is configured to execute the multi-source text data intelligent matching method according to any one of claims 1 to 5 when running.

9. An electronic device, comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute the multi-source text data intelligent matching method according to any one of claims 1 to 5.

Images