Data matching method, device and equipment and computer readable storage medium
By extracting preset and non-preset labels from the data to be matched and the optional matching data, and combining cosine similarity and Euclidean distance calculations, the problems of high complexity, low efficiency and limited accuracy in existing data matching methods are solved, and efficient and accurate data matching is achieved.
Patent Information
- Application Number
- CN202511125430.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-12
- Publication Date
- 2025-11-14
AI Technical Summary
Existing data matching methods are cumbersome, costly, and inefficient when faced with large amounts of data to be matched and optional matching data, and their accuracy is greatly affected by the user's subjective experience.
By extracting preset and non-preset labels from the data to be matched and the optional matching data, calculating cosine similarity and Euclidean distance, and combining a weighted fusion method, matching data is automatically filtered out, thus achieving data matching.
It reduces the complexity of data matching, minimizes the influence of users' subjective experience, and improves the efficiency and accuracy of data matching.
Smart Images
Figure CN120950995A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a data matching method, apparatus, device, and computer-readable storage medium. Background Technology
[0002] With the development of big data, the amount of data generated is constantly increasing, and there will be interaction between data. Data matching is one of the many ways data can interact.
[0003] Currently, the commonly used data matching method involves users finding alternative matching data based on their accumulated experience, and then performing the data matching. When the amount of data to be matched and alternative matching data is large, the data acquisition and matching process becomes cumbersome, costly, inefficient, and time-consuming. Furthermore, the accuracy of the data matching is significantly affected by the user's subjective experience.
[0004] In summary, effectively addressing the current data matching methods' problems of cumbersome matching processes, high costs, low efficiency, long processing times, and significant reliance on user subjective experience for accuracy is a pressing issue for those skilled in the art. Summary of the Invention
[0005] The purpose of this application is to provide a data matching method that reduces the complexity of data matching, reduces the influence of user subjective experience, improves data matching efficiency, and improves data matching accuracy; another purpose of this application is to provide a data matching device, equipment, and computer-readable storage medium.
[0006] To solve the above-mentioned technical problems, this application provides the following technical solution:
[0007] A data matching method, comprising:
[0008] Retrieve the optional matching data for the data to be matched;
[0009] For each optional matching data, tags are extracted to obtain a first tag set corresponding to each optional matching data; wherein, the first tag set includes each first preset tag and each first non-preset tag;
[0010] Obtain historical matching data corresponding to the data to be matched, and obtain a second tag set corresponding to the historical matching data; wherein, the second tag set includes each second preset tag and each second non-preset tag corresponding to the historical matching data;
[0011] Calculate the first cosine similarity and the first Euclidean distance between each first label set and the second label set;
[0012] The matching data corresponding to the data to be matched is determined from each of the optional matching data based on each first cosine similarity and each first Euclidean distance.
[0013] In one specific embodiment of this application, determining the matching data corresponding to the data to be matched from each optional matching data based on each cosine similarity and each Euclidean distance includes:
[0014] Obtain the preset cosine similarity fusion weight and Euclidean distance fusion weight;
[0015] The cosine similarity and Euclidean distance of each optional matching data are weighted and fused according to the cosine similarity fusion weight and the Euclidean distance fusion weight respectively to obtain the matching degree between each optional matching data and the historical matching data respectively.
[0016] The optional matching data are sorted according to the size of each matching degree to obtain the sorting result;
[0017] The first preset number of selectable matching data with the larger matching degree in the sorting results are determined as the matching data corresponding to the data to be matched.
[0018] In one specific embodiment of this application, after determining the matching data corresponding to the data to be matched from each of the optional matching data based on each first cosine similarity and each first Euclidean distance, the method further includes:
[0019] Obtain the third tag set corresponding to each preset additional data; wherein, the third tag set includes each third preset tag;
[0020] Obtain historical additional data corresponding to the data to be matched, and obtain a fourth tag set corresponding to the historical additional data; wherein, the fourth tag set includes each fourth preset tag;
[0021] Calculate the second cosine similarity and the second Euclidean distance between each third tag set and the fourth tag set;
[0022] The additional data corresponding to the data to be matched is determined from each preset additional data based on each second cosine similarity and each second Euclidean distance.
[0023] In one specific embodiment of this application, before calculating the second cosine similarity and the second Euclidean distance between each third tag set and the fourth tag set, the method further includes:
[0024] Select the third preset label for excluding additional data from each third label set, and determine the third preset label for excluding additional data as the additional data condition label;
[0025] When the fourth tag set does not satisfy the additional data condition tag, the preset additional data corresponding to the third tag set to which the additional data condition tag belongs is excluded to obtain each optional additional data;
[0026] Accordingly, the second cosine similarity and the second Euclidean distance between each third tag set and the fourth tag set are calculated, including:
[0027] Calculate the second cosine similarity and the second Euclidean distance between the third tag set and the fourth tag set corresponding to each optional additional data;
[0028] Accordingly, additional data corresponding to the data to be matched is determined from each preset additional data based on each second cosine similarity and each second Euclidean distance, including:
[0029] The additional data corresponding to the data to be matched is determined from each of the optional additional data based on each second cosine similarity and each second Euclidean distance.
[0030] In one specific embodiment of this application, after obtaining the first tag set corresponding to each optional matching data, the method further includes:
[0031] Normalize and encode the numerical labels and comparative non-numerical labels in each of the first preset labels and each of the first non-preset labels to obtain the first encoded sub-vector;
[0032] Hot encoding is performed on the independent non-numerical tags in each of the first preset tags and each of the first non-preset tags to obtain the second encoding sub-vector;
[0033] Obtain the first encoding vector corresponding to the second tag set;
[0034] For each optional matching data, according to the arrangement order of the elements corresponding to each second preset label and each second non-preset label in the first encoding vector, the elements in the first encoding sub-vector and the second encoding sub-vector are merged and rearranged to obtain the second encoding vector;
[0035] Accordingly, the first cosine similarity and the first Euclidean distance between each first label set and the second label set are calculated, including:
[0036] Calculate the first cosine similarity and the first Euclidean distance between each second encoding vector and the first encoding vector.
[0037] In one specific embodiment of this application, after obtaining the first tag set corresponding to each optional matching data, and before calculating the first cosine similarity and the first Euclidean distance between each first tag set and the second tag set, the method further includes:
[0038] For each set of first tags, deduplication is performed on each set of first preset tags and each set of first non-preset tags.
[0039] In one specific embodiment of this application, tags are extracted for each optional matching data to obtain a first tag set corresponding to each optional matching data, including:
[0040] Get the current data matching type;
[0041] Search the tag template library for the preset tag extraction template corresponding to the current data matching type;
[0042] The preset tag extraction template is used to extract tags for each optional matching data to obtain each first preset tag corresponding to each optional matching data.
[0043] For each optional matching data, perform unlabel extraction to obtain the first non-preset label corresponding to each optional matching data;
[0044] The first preset tags and the first non-preset tags corresponding to each optional matching data are merged to obtain the first tag set corresponding to each optional matching data.
[0045] A data matching device, comprising:
[0046] The optional matching data acquisition module is used to acquire each optional matching data of the data to be matched;
[0047] The first tag set acquisition module is used to extract tags from each optional matching data to obtain a first tag set corresponding to each optional matching data; wherein, the first tag set includes each first preset tag and each first non-preset tag;
[0048] The second tag set acquisition module is used to acquire historical matching data corresponding to the data to be matched, and to acquire a second tag set corresponding to the historical matching data; wherein, the second tag set includes each second preset tag and each second non-preset tag corresponding to the historical matching data;
[0049] The first cosine similarity and Euclidean distance calculation module is used to calculate the first cosine similarity and the first Euclidean distance between each first label set and the second label set, respectively.
[0050] The matching data determination module is used to determine the matching data corresponding to the data to be matched from each of the selectable matching data based on each first cosine similarity and each first Euclidean distance.
[0051] A data matching device, comprising:
[0052] Memory, used to store computer programs;
[0053] A processor for executing the computer program to implement the steps of the data matching method as described above.
[0054] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the data matching method as described above.
[0055] The data matching method provided in this application obtains each optional matching data of the data to be matched; extracts labels from each optional matching data to obtain a first label set corresponding to each optional matching data; wherein the first label set includes each first preset label and each first non-preset label; obtains historical matching data corresponding to the data to be matched, and obtains a second label set corresponding to the historical matching data; wherein the second label set includes each second preset label and each second non-preset label corresponding to the historical matching data; calculates the first cosine similarity and the first Euclidean distance between each first label set and the second label set respectively; and determines the matching data corresponding to the data to be matched from each optional matching data based on each first cosine similarity and each first Euclidean distance.
[0056] As can be seen from the above technical solution, by extracting preset tags and non-preset tags from each optional matching data of the data to be matched, a first tag set including each first preset tag and each first non-preset tag is obtained. A second tag set corresponding to the historical matching data of the data to be matched is also obtained. Matching data is obtained by filtering based on the cosine similarity and Euclidean distance between each first tag set and the second tag set. By automatically extracting preset and non-preset tags and automatically matching data based on the extracted tag sets, the tediousness of data matching is greatly reduced, the influence of user subjective experience is reduced, and the efficiency of data matching is improved. By combining cosine similarity and Euclidean distance for data matching, the accuracy of data matching is improved.
[0057] Accordingly, this application also provides a data matching apparatus, device, and computer-readable storage medium corresponding to the above-mentioned data matching method, which have the above-mentioned technical effects, and will not be described in detail here. Attached Figure Description
[0058] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0059] Figure 1This is a flowchart illustrating one implementation of the data matching method in this application.
[0060] Figure 2 This is a flowchart illustrating another implementation of the data matching method in this application.
[0061] Figure 3 This is a schematic diagram of the overall framework of an improved collaborative filtering algorithm with hybrid topic tags in an embodiment of this application;
[0062] Figure 4 This is a structural block diagram of a data matching device according to an embodiment of this application;
[0063] Figure 5 This is a structural block diagram of a data matching device according to an embodiment of this application;
[0064] Figure 6 This is a schematic diagram of the specific structure of a data matching device provided in an embodiment of this application. Detailed Implementation
[0065] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0066] See Figure 1 , Figure 1 This is a flowchart illustrating one implementation of the data matching method in this application. The method may include the following steps:
[0067] S101: Obtain the optional matching data for the data to be matched.
[0068] Multiple optional matching data are provided in advance for the data to be matched, and each optional matching data is obtained from the data to be matched.
[0069] S102: Extract labels for each optional matching data to obtain the first label set corresponding to each optional matching data.
[0070] The first tag set includes each first preset tag and each first non-preset tag.
[0071] After obtaining the optional matching data of the data to be matched, tags are extracted for each optional matching data to obtain a first tag set corresponding to each optional matching data. For example, preset tags can be extracted for each optional matching data to obtain first preset tags, and non-preset tags can be extracted for each optional matching data to obtain first non-preset tags. The first preset tags and the first non-preset tags are then merged to obtain a first tag set.
[0072] S103: Obtain the historical matching data corresponding to the data to be matched, and obtain the second tag set corresponding to the historical matching data.
[0073] The second tag set includes each second preset tag and each second non-preset tag corresponding to the historical matching data.
[0074] The data to be matched contains historical matching data obtained from previous matching. The historical matching data corresponding to the data to be matched is obtained, and a second tag set including each second preset tag and each second non-preset tag corresponding to the historical matching data is obtained.
[0075] S104: Calculate the first cosine similarity and the first Euclidean distance between each first label set and the second label set.
[0076] After extracting the first tag set corresponding to each optional matching data and obtaining the second tag set corresponding to the historical matching data, the first cosine similarity and the first Euclidean distance between each first tag set and the second tag set are calculated respectively.
[0077] S105: Determine the matching data corresponding to the data to be matched from each of the optional matching data based on each first cosine similarity and each first Euclidean distance.
[0078] After calculating the first cosine similarity and the first Euclidean distance between each first label set and the second label set, the matching data corresponding to the data to be matched is determined from each optional matching data based on the first cosine similarity and the first Euclidean distance. For example, the weight information corresponding to the cosine similarity and Euclidean distance can be pre-set, and the matching data corresponding to the data to be matched is determined from each optional matching data based on the weight information corresponding to the cosine similarity and Euclidean distance, as well as the first cosine similarity and the first Euclidean distance.
[0079] As can be seen from the above technical solution, by extracting preset tags and non-preset tags from each optional matching data of the data to be matched, a first tag set including each first preset tag and each first non-preset tag is obtained. A second tag set corresponding to the historical matching data of the data to be matched is also obtained. Matching data is obtained by filtering based on the cosine similarity and Euclidean distance between each first tag set and the second tag set. By automatically extracting preset and non-preset tags and automatically matching data based on the extracted tag sets, the tediousness of data matching is greatly reduced, the influence of user subjective experience is reduced, and the efficiency of data matching is improved. By combining cosine similarity and Euclidean distance for data matching, the accuracy of data matching is improved.
[0080] It should be noted that, based on the above embodiments, this application also provides corresponding improvement solutions. In subsequent embodiments, steps that are the same as or corresponding to those in the above embodiments can be referred to each other, and the corresponding beneficial effects can also be referred to each other. These improvements will not be elaborated upon in the following improved embodiments.
[0081] See Figure 2 , Figure 2 This is a flowchart illustrating another implementation of the data matching method in this application. The method may include the following steps:
[0082] S201: Retrieve the optional matching data for the data to be matched.
[0083] S202: Extract labels for each optional matching data to obtain the first label set corresponding to each optional matching data.
[0084] The first tag set includes each first preset tag and each first non-preset tag.
[0085] S203: Obtain the historical matching data corresponding to the data to be matched, and obtain the second tag set corresponding to the historical matching data.
[0086] The second tag set includes each second preset tag and each second non-preset tag corresponding to the historical matching data.
[0087] S204: Calculate the first cosine similarity and the first Euclidean distance between each first label set and the second label set.
[0088] S205: Obtain the preset cosine similarity fusion weight and Euclidean distance fusion weight.
[0089] Pre-set the cosine similarity fusion weight and the Euclidean distance fusion weight, and obtain the preset cosine similarity fusion weight and Euclidean distance fusion weight.
[0090] S206: The cosine similarity and Euclidean distance of each optional matching data are weighted and fused according to the cosine similarity fusion weight and the Euclidean distance fusion weight respectively to obtain the matching degree between each optional matching data and the historical matching data.
[0091] After obtaining the preset cosine similarity fusion weights and Euclidean distance fusion weights, the cosine similarity and Euclidean distance of each optional matching data are weighted and fused according to these weights to obtain the matching degree between each optional matching data and the historical matching data. By weighting and fusing cosine similarity and Euclidean distance, the two similarity calculation methods are integrated, significantly improving the accuracy of the calculated matching degree.
[0092] S207: Sort the optional matching data according to the size of each matching degree to obtain the sorting result.
[0093] After obtaining the matching degree corresponding to each optional matching data and the historical matching data, the optional matching data are sorted according to the size of each matching degree. For example, the optional matching data can be sorted in descending order of matching degree, or in ascending order of matching degree, to obtain the sorting result.
[0094] S208: Determine the first preset number of optional matching data with the larger matching degree in the sorting results as the matching data corresponding to the data to be matched.
[0095] After sorting the optional matching data according to the size of each matching degree, the top preset number of optional matching data with the larger matching degree in the sorted results are determined as the matching data corresponding to the data to be matched, thus realizing the recommendation of TOP-N matching data.
[0096] S209: Obtain the set of third tags corresponding to each preset additional data.
[0097] The third tag set includes each third preset tag.
[0098] Matching data between the target data and the matching data sometimes requires the participation of additional data. There may be multiple preset additional data options. Obtain the set of third tags, including each preset third tag, corresponding to each preset additional data.
[0099] S210: Obtain the historical additional data corresponding to the data to be matched, and obtain the fourth tag set corresponding to the historical additional data.
[0100] The fourth tag set includes each of the fourth preset tags.
[0101] The data to be matched has corresponding historical additional data. Obtain the historical additional data corresponding to the data to be matched, and obtain the fourth tag set corresponding to the historical additional data, including each fourth preset tag.
[0102] S211: Calculate the second cosine similarity and the second Euclidean distance between each third label set and the fourth label set.
[0103] After obtaining the third tag set corresponding to each preset additional data and the fourth tag set corresponding to the historical additional data, the second cosine similarity and the second Euclidean distance between each third tag set and the fourth tag set are calculated respectively.
[0104] S212: Determine the supplementary data corresponding to the data to be matched from each preset supplementary data based on each second cosine similarity and each second Euclidean distance.
[0105] After calculating the second cosine similarity and second Euclidean distance between each third label set and the fourth label set, supplementary data corresponding to the data to be matched is determined from each preset supplementary data based on the second cosine similarity and the second Euclidean distance. By combining cosine similarity and Euclidean distance for supplementary data filtering, the accuracy of the filtered supplementary data is significantly improved.
[0106] In one specific embodiment of this application, before step S211, the method may further include the following steps:
[0107] Step 1: Select the third preset label for excluding additional data from each third label set, and determine the third preset label for excluding additional data as the additional data condition label;
[0108] Step 2: When the fourth tag set does not meet the additional data condition tag, exclude the preset additional data corresponding to the third tag set to which the additional data condition tag belongs, and obtain each optional additional data;
[0109] Accordingly, step S211 may include the following steps:
[0110] Calculate the second cosine similarity and the second Euclidean distance between the third and fourth label sets corresponding to each optional additional data;
[0111] Accordingly, step S212 may include the following steps:
[0112] The supplementary data corresponding to the data to be matched is determined from each of the optional supplementary data based on each second cosine similarity and each second Euclidean distance.
[0113] For ease of description, the above steps can be combined for explanation.
[0114] Each third tag set includes a third preset tag for excluding additional data. After obtaining the third tag set corresponding to each preset additional data and the fourth tag set corresponding to historical additional data, third preset tags for excluding additional data are selected from each third tag set and determined as additional data condition tags. When the fourth tag set does not meet the additional data condition tags, the preset additional data corresponding to the third tag set to which the additional data condition tags belong is excluded, resulting in each optional additional data. The second cosine similarity and the second Euclidean distance between the third tag set and the fourth tag set corresponding to each optional additional data are calculated. Based on the second cosine similarity and the second Euclidean distance, the additional data corresponding to the data to be matched is determined from each optional additional data. By first using the third preset tags for excluding additional data to exclude the preset additional data, matching redundant preset additional data is avoided, the amount of matching preset additional data is reduced, and the matching efficiency of additional data is improved.
[0115] In one specific embodiment of this application, after obtaining the first tag set corresponding to each optional matching data, the method may further include the following steps:
[0116] Step 1: Normalize and encode the numerical labels and comparative non-numerical labels in each of the first preset labels and each of the first non-preset labels to obtain the first encoded sub-vector;
[0117] Step 2: Perform hot encoding on the independent non-numerical tags in each of the first preset tags and each of the first non-preset tags to obtain the second encoding sub-vector;
[0118] Step 3: Obtain the first encoding vector corresponding to the second tag set;
[0119] Step 4: For each optional matching data, according to the arrangement order of the elements corresponding to each second preset label and each second non-preset label in the first encoding vector, merge and rearrange the elements in the first encoding sub-vector and the second encoding sub-vector to obtain the second encoding vector;
[0120] Accordingly, calculating the first cosine similarity and the first Euclidean distance between each first label set and the second label set may include the following steps:
[0121] Calculate the first cosine similarity and the first Euclidean distance between each second encoding vector and the first encoding vector.
[0122] For ease of description, the above steps can be combined for explanation.
[0123] Each first preset label and each first non-preset label may be a numerical label, a comparative non-numerical label, or an independent non-numerical label. After obtaining the first label set corresponding to each optional matching data, the numerical labels and comparative non-numerical labels in each first preset label and each first non-preset label are normalized and encoded to obtain a first encoded sub-vector. The independent non-numerical labels in each first preset label and each first non-preset label are hot-coded to obtain a second encoded sub-vector. The first encoded vector corresponding to the second label set is obtained. For each optional matching data, according to the arrangement order of the elements corresponding to each second preset label and each second non-preset label in the first encoded vector, the elements in the first encoded sub-vector and the second encoded sub-vector are merged and rearranged to obtain a second encoded vector. The first cosine similarity and the first Euclidean distance between each second encoded vector and the first encoded vector are calculated respectively. Then, the matching data corresponding to the data to be matched can be determined from each optional matching data based on the first cosine similarity and the first Euclidean distance between each second encoded vector and the first encoded vector. By normalizing and encoding numerical labels and comparative non-numerical labels, and by hot-coding mutually independent non-numerical labels, the first preset labels and the first non-preset labels are converted into normalized data, thereby improving the accuracy of the calculated first cosine similarity and first Euclidean distance, and thus improving the accuracy of the matched data.
[0124] In one specific embodiment of this application, after obtaining the first tag set corresponding to each optional matching data, and before calculating the first cosine similarity and the first Euclidean distance between each first tag set and the second tag set, the method may further include the following steps:
[0125] For each set of first tags, deduplication is performed on each set of first preset tags and each set of first non-preset tags.
[0126] After obtaining the first tag set corresponding to each optional matching data, before calculating the first cosine similarity and the first Euclidean distance between each first tag set and the second tag set, for each first tag set, deduplication is performed on each first preset tag and each first non-preset tag. For example, deduplication can be performed on each first preset tag and each first non-preset tag with high similarity or repetition, thereby reducing the computational load of the data matching process and improving the data matching efficiency.
[0127] In one specific embodiment of this application, tag extraction is performed on each optional matching data to obtain a first tag set corresponding to each optional matching data, which may include the following steps:
[0128] Step 1: Obtain the current data matching type;
[0129] Step 2: Search the tag template library for the preset tag extraction template corresponding to the current data matching type;
[0130] Step 3: Use the preset tag extraction template to extract tags for each optional matching data to obtain the first preset tags corresponding to each optional matching data;
[0131] Step 4: Extract the non-label from each optional matching data to obtain the first non-preset label corresponding to each optional matching data;
[0132] Step 5: Merge each first preset tag and each first non-preset tag corresponding to each optional matching data to obtain the first tag set corresponding to each optional matching data.
[0133] For ease of description, the five steps above can be combined for explanation.
[0134] The process involves obtaining the current data matching type, searching for the corresponding preset tag extraction template in the tag template library, extracting tags for each optional matching data using the preset tag extraction template, obtaining the first preset tags for each optional matching data, and then extracting non-preset tags for each optional matching data. Finally, the first preset tags and first non-preset tags for each optional matching data are merged to obtain the first tag set for each optional matching data. By pre-setting a tag template library containing multiple preset tag extraction templates, and using these preset tag extraction templates for tag extraction, the accuracy and efficiency of preset tag extraction are significantly improved, thereby enhancing the accuracy and efficiency of data matching.
[0135] To facilitate understanding of the technical solutions provided in the embodiments of the present invention, the following uses a specific scenario of applying intelligent transaction matching in the noodle processing industry as an example to provide a detailed description of the technical solutions provided in the embodiments of the present invention.
[0136] It should be noted that the fresh noodle processing industry is a significant economic pillar in some regions and a prime customer base for commercial banks' asset, liability, and intermediary services. However, this customer base is scattered across the country, characterized by its small size and dispersed nature. This leads to difficulties in information exchange and supply-demand matching between buyers and sellers in transactions involving noodle workshops, raw materials such as flour, and fresh noodle products. Consequently, transaction costs are high, operational efficiency is low, and time-consuming processes are lengthy. Furthermore, because the traditional fresh noodle processing industry relies on offline operations and lacks an information technology infrastructure, commercial banks and other financial institutions lack data support and credit criteria when providing financial services to this customer base. This makes it difficult to accurately identify high-quality clients and provide efficient and accurate credit services. Consequently, commercial banks face high labor costs, low efficiency, long processing times, inaccurate business recommendations, and low customer satisfaction when dealing with the fresh noodle processing industry.
[0137] The above problems can be solved using the data matching method provided in the embodiments of this application. Taking flour mill buying and selling transactions as an example, the specific implementation process is as follows:
[0138] Step 1: Obtain the description text of each noodle shop's seller;
[0139] Step 2: Extract tags from the description text of each noodle shop seller to obtain the first tag set corresponding to the description text of each noodle shop seller; wherein, the first tag set includes each first preset tag and each first non-preset tag;
[0140] Step 3: Obtain the historical description text of the flour mill corresponding to the successful transactions of the flour mill buyer, and obtain the second tag set corresponding to the historical flour mill description text; wherein, the second tag set includes each second preset tag and each second non-preset tag corresponding to the historical flour mill description text;
[0141] Step 4: Calculate the first cosine similarity and the first Euclidean distance between each first label set and the second label set;
[0142] Step 5: Determine the flour mill seller corresponding to the flour mill buyer from among the flour mill sellers based on the first cosine similarity and the first Euclidean distance.
[0143] See Figure 3 , Figure 3 This is a schematic diagram of the overall framework of an improved collaborative filtering algorithm with hybrid topic tags, as described in an embodiment of this application. Figure 3In noodle workshop transactions, most key matching tags are fixed and preset, analyzed and summarized by industry experts. Taking noodle workshop buying and selling transactions as an example, when the transaction type is noodle workshop buying and selling, the fixed keywords mainly include four categories: geographical location type (bustling city center, general city center, general suburbs), years of operation, and average daily sales volume (unit: package). The system has multiple sets of fixed tag templates according to the matching transaction type (such as noodle workshop buying and selling transactions, noodle processing machinery and equipment transactions, noodle processing consumables transactions), and the fixed tags are consistent for the same matching transaction type.
[0144] [ Figure 3[II] Using only fixed tags has limitations in matching accuracy. This application also establishes a non-preset tag extraction capability, meaning the system needs to extract tags based on the buyer and seller requirements collected by the platform. For example: Suppose the descriptive text for seller A's noodle shop is: "A noodle shop located in the bustling city center, operating for 10 years, with an area of 300 square meters, selling an average of 35 packs per day, rated A in hygiene, mainly processing fresh noodles, using advanced equipment, and engaging in both wholesale and retail." The result after word segmentation is: ["located in", "bustling", "city center", "of", "noodle shop", "operating", "10 years", "area", "300 square meters", "daily average", "sales", "35 packs", "hygiene", "environment", "A grade", "mainly", "operating", "fresh noodles", "processing", "using", "advanced", "equipment", "wholesale", "retail", "concurrently operating"]. The result after removing stop words is: ["bustling", "city center", "noodle shop", "operating", "10 years", "area", "300 square meters", "daily average", "sales", "35 packs", "hygiene", "environment", "A grade", "mainly", "operating", "fresh noodles", "processing", "using", "advanced", "equipment", "wholesale", "retail", "concurrently operating"]. Construct a document-word matrix, assuming the vocabulary is: ["bustling", "city center", "noodle shop"]. The system extracts high-frequency keywords as non-preset labels based on the LDA (Latent Dirichlet Allocation) algorithm, thereby significantly increasing the accuracy of transaction matching. Keywords include: "operation," "10 years," "area," "300 square meters," "daily average," "sales," "35 packs," "hygiene," "environment," "Grade A," "main," "operation," "fresh noodles," "processing," "adopted," "advanced," "equipment," "wholesale," "retail," and "concurrent operation." Tags extracted by LDA generally include geographic location information (codes or representations of geographic locations such as province, city, district, county, and street; for example, the degree of matching between the buyer's desired geographic location and the seller's actual geographic location), scale information (size of the noodle shop, such as square meters), scale-related indicators such as the number of employees or production capacity, years of operation, business type or category, main types of noodles the noodle shop mainly produces (such as dried noodle processing, fresh noodle processing, other types of products), business mode of operation (such as wholesale, retail), hygiene and environmental information (such as Grade A, Grade B, etc.), and equipment information (the type and quantity of equipment used in the noodle shop). In actual business matching, different matching transaction types and each specific transaction will generate different non-preset tags.
[0145] [ Figure 3[III] In the third step, the system needs to perform vectorization processing of mixed topic words. First, the preset tags and non-preset tags need to be merged into a single table. The system will automatically remove duplicate tags, that is, if non-preset tags and preset tags are similar or duplicated, they will be automatically removed. The merged table is shown in Table 1 below. Table 1 is the tag table obtained by merging non-preset tags and preset tags. Table 1
[0146] Then, based on the different types of label values, normalized label encoding and one-hot encoding are performed. Specifically, there are three cases:
[0147] (1) Case 1: For numerical labels, such as size and area, the system directly performs normalization, using the normalization formula to convert the vectorized result into data between 0 and 1;
[0148] (2) Case 2: For non-numerical labels with comparative significance, such as geographical location (bustling city center, general city center, general suburbs). The system determines the business meaning, maps the labels to the values 0, 1, 2 according to the business order, and then performs normalization processing to unify them into data between 0 and 1.
[0149] (3) Case 3: For mutually independent non-numerical labels, such as noodle types (dry noodles, wet noodles, dried noodles). The system uses one-hot encoding to expand each type into a dimension [dry, wet, dried], with wet noodles being [0, 1, 0] and dry noodles being [1, 0, 0].
[0150] After combining normalized label encoding and one-hot encoding, the hybrid topic tags seller topic words are vectorized, as shown in Table 2 below. Table 2 is a tag table obtained after normalized label encoding and one-hot encoding. Table 2
[0151] The vectorized hybrid topic tags will have their similarity calculated using cosine similarity and Euclidean distance. The Euclidean distance, after normalization, will be used to form a matching ranking p(c, d) through a weighted fusion algorithm provided in this embodiment. Finally, the matching ranking will be rearranged in a matrix to derive the Top-N matching recommendation. The subsequent process is as follows:
[0152] [ Figure 3 -IV] Wherein, the cosine similarity is calculated as follows:
[0153] ;
[0154] In the above formula, A and B represent the vectors of the two element fields matched in the transaction matching. The similarity range is (-1, 1), with larger values indicating higher similarity.
[0155] [ Figure 3 The Euclidean distance is calculated as follows:
[0156] ;
[0157] In the above formula, x and y represent similar comparison objects, and n represents the total number of label fields in the flour mill.
[0158] [ Figure 3 -VI] Normalization of Euclidean distance: The range of Euclidean distance and cosine similarity are aligned to (-1, 1) using the following normalization formula for subsequent calculation.
[0159] ;
[0160] [ Figure 3 [VII] Finally, the system uses different weights to perform a weighted fusion of the two similarities, with the weights (w1, w2) dynamically adjusted based on historical successful transactions. In this way, the system obtains the matching degree between flour mill sellers, as shown below:
[0161] ;
[0162] In the above formula, ω1 and ω2 are the similarity fusion weights, and ω1 + ω2 = 1 is required. c and d refer to the chordal similarity cos(θ) and the Euclidean distance d(x, y) respectively (after normalization). The similarity matrix of p(c, d) is derived through the weight fusion algorithm, and the matrix is shown in Table 3 below. Table 3 is the similarity matrix derived through the weight fusion algorithm. Table 3
[0163] [ Figure 3 When a buyer inquires about a flour mill, the system locates their preferred seller (e.g., seller C) based on their historical transaction records. Then, using a similarity matrix, it ranks the remaining sellers (e.g., seller E has a similarity p(c53, d53) of 0.95 with seller C, making it the most similar seller and ranked second). The Top-N matching ranking recommendation is shown in Table 4 below. Table 4 is the seller recommendation table obtained by performing Top-N matching ranking on the sellers. Table 4
[0164] Commercial banks offer various financial products with different categories, attributes, and application requirements. Loan product matching tags are all derived from preset tags, specifically including: applicable population, loan purpose, age restrictions, occupational restrictions, maximum loan amount, maximum loan term, interest calculation method, interest rate level, approval timeframe, repayment method, and whether collateral is required. The system performs reverse filtering based on financial product exclusion criteria, pre-excluding buyers and sellers and their corresponding financial products who do not meet the application requirements. As shown in Table 2, Loan Product 1 is a collateralized product; if the matched buyer and seller are uncollateralized, Loan Product 1 will not be included in the ranking. This approach avoids recommending products to ineligible customers, further improving matching accuracy. In this stage, the system still uses the improved collaborative filtering algorithm described earlier to generate ranking numbers for matched transactions in the loan product list, as shown in Table 5 below. Table 5
[0165] In this step, the system reorders the buyer-seller matching scores generated by an improved collaborative filtering algorithm based on hybrid topic tags, combined with the loan product matching scores generated by an improved collaborative filtering algorithm based on preset tags, to generate an overall recommendation for matching transactions and loan products for the buyer. Finally, based on the overall recommendation for matching transactions and loan products obtained from testing, the system pushes transaction suggestion messages to the buyer via in-app messages and SMS through the noodle industry information system (APP). For example, "Dear A, the system has matched and recommended seller product B and loan product 1 for you. Please log in to the noodle industry information system (APP) for details." The intermediate business logic table is shown in Table 6 below, where Table 6 is the loan product recommendation ranking table. Table 6
[0166] This application's embodiments improve transaction efficiency and reduce transaction costs for practitioners in the fresh noodle processing industry, and enhance the work efficiency, reduce labor costs, and improve service quality and customer satisfaction for commercial banks serving customers in the fresh noodle processing industry. The improved collaborative filtering algorithm using a hybrid topic tagging approach combining intelligent transaction and intelligent recommendation algorithms employed in this application's embodiments exhibits higher response speed, more accurate matching, and stronger stability compared to traditional recommendation transaction algorithms. It also possesses compatibility with non-preset tags and requires less industry expert experience. This application's embodiments are highly versatile, applicable not only to noodle industry matching transactions and loan recommendations but also to other fields combining "transaction matching + financial product recommendation," and possess strong scalability.
[0167] Corresponding to the above method embodiments, this application also provides a data matching device, and the data matching device described below can be referred to in correspondence with the data matching method described above.
[0168] See Figure 4 , Figure 4 This is a structural block diagram of a data matching device according to an embodiment of the present application. The device may include:
[0169] Optional matching data acquisition module 41 is used to acquire each optional matching data of the data to be matched;
[0170] The first tag set acquisition module 42 is used to extract tags for each optional matching data to obtain a first tag set corresponding to each optional matching data; wherein, the first tag set includes each first preset tag and each first non-preset tag;
[0171] The second tag set acquisition module 43 is used to acquire historical matching data corresponding to the data to be matched, and to acquire the second tag set corresponding to the historical matching data; wherein, the second tag set includes each second preset tag and each second non-preset tag corresponding to the historical matching data;
[0172] The first cosine similarity and Euclidean distance calculation module 44 is used to calculate the first cosine similarity and the first Euclidean distance between each first label set and the second label set respectively.
[0173] The matching data determination module 45 is used to determine the matching data corresponding to the data to be matched from each of the optional matching data based on each first cosine similarity and each first Euclidean distance.
[0174] As can be seen from the above technical solution, by extracting preset tags and non-preset tags from each optional matching data of the data to be matched, a first tag set including each first preset tag and each first non-preset tag is obtained. A second tag set corresponding to the historical matching data of the data to be matched is also obtained. Matching data is obtained by filtering based on the cosine similarity and Euclidean distance between each first tag set and the second tag set. By automatically extracting preset and non-preset tags and automatically matching data based on the extracted tag sets, the tediousness of data matching is greatly reduced, the influence of user subjective experience is reduced, and the efficiency of data matching is improved. By combining cosine similarity and Euclidean distance for data matching, the accuracy of data matching is improved.
[0175] In one specific embodiment of this application, the matching data determination module 45 includes:
[0176] The weight acquisition submodule is used to obtain the preset cosine similarity fusion weight and Euclidean distance fusion weight;
[0177] The matching degree acquisition submodule is used to perform weighted fusion of the cosine similarity and Euclidean distance corresponding to each optional matching data according to the cosine similarity fusion weight and the Euclidean distance fusion weight, respectively, to obtain the matching degree corresponding to each optional matching data and the historical matching data.
[0178] The sorting result acquisition submodule is used to sort each optional matching data according to the size of each matching degree to obtain the sorting result;
[0179] The matching data determination submodule is used to determine the first preset number of optional matching data with the larger matching degree in the sorting results as the matching data corresponding to the data to be matched.
[0180] In one specific embodiment of this application, the device may further include:
[0181] The third tag set acquisition module is used to acquire the third tag set corresponding to each preset additional data after determining the matching data corresponding to the data to be matched from each optional matching data based on each first cosine similarity and each first Euclidean distance; wherein, the third tag set includes each third preset tag;
[0182] The fourth tag set acquisition module is used to acquire the historical additional data corresponding to the data to be matched, and to acquire the fourth tag set corresponding to the historical additional data; wherein, the fourth tag set includes each fourth preset tag;
[0183] The second cosine similarity and Euclidean distance calculation module is used to calculate the second cosine similarity and the second Euclidean distance between each third label set and the fourth label set, respectively.
[0184] The additional data determination module is used to determine the additional data corresponding to the data to be matched from each preset additional data based on each second cosine similarity and each second Euclidean distance.
[0185] In one specific embodiment of this application, the device may further include:
[0186] The additional data condition label determination module is used to select third preset labels for additional data exclusion from each third label set before calculating the second cosine similarity and the second Euclidean distance between each third label set and the fourth label set, and to determine the third preset labels for additional data exclusion as additional data condition labels.
[0187] The optional additional data acquisition module is used to exclude the preset additional data corresponding to the third tag set to which the additional data condition tag belongs when the fourth tag set does not meet the additional data condition tag, and obtain each optional additional data;
[0188] Correspondingly, the second cosine similarity and Euclidean distance calculation module is specifically a module that calculates the second cosine similarity and the second Euclidean distance between the third label set and the fourth label set corresponding to each optional additional data.
[0189] Accordingly, the additional data determination module is specifically a module that determines the additional data corresponding to the data to be matched from each of the optional additional data based on each second cosine similarity and each second Euclidean distance.
[0190] In one specific embodiment of this application, the device may further include:
[0191] The normalization encoding acquisition module is used to normalize and encode the numerical labels and comparative non-numerical labels in each first preset label and each first non-preset label after obtaining the first label set corresponding to each optional matching data, so as to obtain the first encoding sub-vector.
[0192] The second encoding sub-vector acquisition module is used to perform hot encoding on the independent non-numerical tags in each first preset tag and each first non-preset tag to obtain the second encoding sub-vector.
[0193] The first encoding vector acquisition module is used to acquire the first encoding vector corresponding to the second tag set;
[0194] The second encoding vector acquisition module is used to merge and rearrange the elements in the first encoding sub-vector and the second encoding sub-vector according to the arrangement order of the elements corresponding to each second preset label and each second non-preset label in the first encoding vector for each optional matching data, so as to obtain the second encoding vector.
[0195] The first cosine similarity and Euclidean distance calculation module 44 is specifically a module that calculates the first cosine similarity and the first Euclidean distance between each second encoding vector and the first encoding vector.
[0196] In one specific embodiment of this application, the device may further include:
[0197] The tag deduplication module is used to deduplicate each first preset tag and each first non-preset tag for each first tag set after obtaining the first tag set corresponding to each optional matching data and before calculating the first cosine similarity and the first Euclidean distance between each first tag set and the second tag set.
[0198] In one specific embodiment of this application, the first tag set acquisition module 42 includes:
[0199] The current data matching type retrieval submodule is used to retrieve the current data matching type;
[0200] The preset tag extraction template search submodule is used to search for the preset tag extraction template corresponding to the current data matching type from the tag template library;
[0201] The first preset tag acquisition submodule is used to extract tags from each optional matching data using a preset tag extraction template, and obtain each first preset tag corresponding to each optional matching data.
[0202] The first non-preset label acquisition submodule is used to extract non-labels from each optional matching data to obtain each first non-preset label corresponding to each optional matching data.
[0203] The first tag set acquisition submodule is used to merge each first preset tag and each first non-preset tag corresponding to each optional matching data to obtain the first tag set corresponding to each optional matching data.
[0204] For the method embodiments described above, see [link to relevant documentation]. Figure 5 , Figure 5 This is a schematic diagram of the data matching device provided in this application, which may include:
[0205] Memory 332 is used to store computer programs;
[0206] The processor 322 is used to implement the steps of the data matching method in the above method embodiments when executing a computer program.
[0207] For details, please refer to Figure 6 , Figure 6This is a schematic diagram illustrating the specific structure of a data matching device provided in this embodiment. The data matching device can vary significantly due to different configurations or performance. It may include a processor (central processing unit, CPU) 322 (e.g., one or more processors) and a memory 332. The memory 332 stores one or more computer programs 342 or data 344. The memory 332 can be temporary or persistent storage. The program stored in the memory 332 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the data processing device. Furthermore, the processor 322 may be configured to communicate with the memory 332 and execute the series of instruction operations stored in the memory 332 on the data matching device 301.
[0208] The data matching device 301 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input / output interfaces 358, and / or one or more operating systems 341.
[0209] The steps in the data matching method described above can be implemented by the structure of the data matching device.
[0210] Corresponding to the above method embodiments, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the following steps:
[0211] Obtain all optional matching data for the data to be matched; extract labels for each optional matching data to obtain a first label set corresponding to each optional matching data; wherein, the first label set includes each first preset label and each first non-preset label; obtain historical matching data corresponding to the data to be matched, and obtain a second label set corresponding to the historical matching data; wherein, the second label set includes each second preset label and each second non-preset label corresponding to the historical matching data; calculate the first cosine similarity and the first Euclidean distance between each first label set and the second label set respectively; determine the matching data corresponding to the data to be matched from each optional matching data based on each first cosine similarity and each first Euclidean distance.
[0212] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0213] For a description of the computer-readable storage medium provided in this application, please refer to the above method embodiments; further details will not be repeated here.
[0214] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatuses, devices, and computer-readable storage media disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.
[0215] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the technical solutions and core ideas of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of this application.
Claims
1. A data matching method, characterized in that, include: Retrieve the optional matching data for the data to be matched; For each optional matching data, tags are extracted to obtain a first tag set corresponding to each optional matching data; wherein, the first tag set includes each first preset tag and each first non-preset tag; Obtain historical matching data corresponding to the data to be matched, and obtain a second tag set corresponding to the historical matching data; wherein, the second tag set includes each second preset tag and each second non-preset tag corresponding to the historical matching data; Calculate the first cosine similarity and the first Euclidean distance between each first label set and the second label set; The matching data corresponding to the data to be matched is determined from each of the optional matching data based on each first cosine similarity and each first Euclidean distance.
2. The data matching method according to claim 1, characterized in that, Based on each cosine similarity and each Euclidean distance, the matching data corresponding to the data to be matched is determined from each available matching data, including: Obtain the preset cosine similarity fusion weight and Euclidean distance fusion weight; The cosine similarity and Euclidean distance of each optional matching data are weighted and fused according to the cosine similarity fusion weight and the Euclidean distance fusion weight respectively to obtain the matching degree between each optional matching data and the historical matching data respectively. The optional matching data are sorted according to the size of each matching degree to obtain the sorting result; The first preset number of selectable matching data with the larger matching degree in the sorting results are determined as the matching data corresponding to the data to be matched.
3. The data matching method according to claim 1 or 2, characterized in that, After determining the matching data corresponding to the data to be matched from each of the optional matching data based on each first cosine similarity and each first Euclidean distance, the method further includes: Obtain the third tag set corresponding to each preset additional data; wherein, the third tag set includes each third preset tag; Obtain historical additional data corresponding to the data to be matched, and obtain a fourth tag set corresponding to the historical additional data; wherein, the fourth tag set includes each fourth preset tag; Calculate the second cosine similarity and the second Euclidean distance between each third tag set and the fourth tag set; The additional data corresponding to the data to be matched is determined from each preset additional data based on each second cosine similarity and each second Euclidean distance.
4. The data matching method according to claim 3, characterized in that, Before calculating the second cosine similarity and second Euclidean distance between each third tag set and the fourth tag set, the method further includes: Select the third preset label for excluding additional data from each third label set, and determine the third preset label for excluding additional data as the additional data condition label; When the fourth tag set does not satisfy the additional data condition tag, the preset additional data corresponding to the third tag set to which the additional data condition tag belongs is excluded to obtain each optional additional data; Accordingly, the second cosine similarity and the second Euclidean distance between each third tag set and the fourth tag set are calculated, including: Calculate the second cosine similarity and the second Euclidean distance between the third tag set and the fourth tag set corresponding to each optional additional data; Accordingly, additional data corresponding to the data to be matched is determined from each preset additional data based on each second cosine similarity and each second Euclidean distance, including: The additional data corresponding to the data to be matched is determined from each of the optional additional data based on each second cosine similarity and each second Euclidean distance.
5. The data matching method according to claim 1, characterized in that, After obtaining the first tag set corresponding to each optional matching data, it also includes: Normalize and encode the numerical labels and comparative non-numerical labels in each of the first preset labels and each of the first non-preset labels to obtain the first encoded sub-vector; Hot encoding is performed on the independent non-numerical tags in each of the first preset tags and each of the first non-preset tags to obtain the second encoding sub-vector; Obtain the first encoding vector corresponding to the second tag set; For each optional matching data, according to the arrangement order of the elements corresponding to each second preset label and each second non-preset label in the first encoding vector, the elements in the first encoding sub-vector and the second encoding sub-vector are merged and rearranged to obtain the second encoding vector; Accordingly, the first cosine similarity and the first Euclidean distance between each first label set and the second label set are calculated, including: Calculate the first cosine similarity and the first Euclidean distance between each second encoding vector and the first encoding vector.
6. The data matching method according to claim 1, characterized in that, After obtaining the first label set corresponding to each optional matching data, and before calculating the first cosine similarity and the first Euclidean distance between each first label set and the second label set, the process also includes: For each set of first tags, deduplication is performed on each set of first preset tags and each set of first non-preset tags.
7. The data matching method according to claim 1, characterized in that, For each optional matching data, tags are extracted to obtain the first tag set corresponding to each optional matching data, including: Get the current data matching type; Search the tag template library for the preset tag extraction template corresponding to the current data matching type; The preset tag extraction template is used to extract tags for each optional matching data to obtain each first preset tag corresponding to each optional matching data. For each optional matching data, perform unlabel extraction to obtain the first non-preset label corresponding to each optional matching data; The first preset tags and the first non-preset tags corresponding to each optional matching data are merged to obtain the first tag set corresponding to each optional matching data.
8. A data matching device, characterized in that, include: The optional matching data acquisition module is used to acquire each optional matching data of the data to be matched; The first tag set acquisition module is used to extract tags from each optional matching data to obtain a first tag set corresponding to each optional matching data; wherein, the first tag set includes each first preset tag and each first non-preset tag; The second tag set acquisition module is used to acquire historical matching data corresponding to the data to be matched, and to acquire a second tag set corresponding to the historical matching data; wherein, the second tag set includes each second preset tag and each second non-preset tag corresponding to the historical matching data; The first cosine similarity and Euclidean distance calculation module is used to calculate the first cosine similarity and the first Euclidean distance between each first label set and the second label set, respectively. The matching data determination module is used to determine the matching data corresponding to the data to be matched from each of the selectable matching data based on each first cosine similarity and each first Euclidean distance.
9. A data matching device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the data matching method as described in any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the data matching method as described in any one of claims 1 to 7.