Multi-platform resource matching method and device and readable storage medium
By preprocessing and multiple sorting of resource information of multiple e-commerce platforms, the problems of low matching accuracy and lack of results in the prior art are solved, and resource matching with high accuracy and efficiency are achieved.
Patent Information
- Application Number
- CN202411691232.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-22
- Publication Date
- 2025-03-18
AI Technical Summary
The prior art is difficult to accurately locate the same resource in the matching of multi-source services or products, and due to different platform specifications, some matching results may be missing.
By obtaining relevant matching objects from multiple e-commerce platforms, performing data cleaning, vectorization, entity extraction and post-processing, triple sorting based on text semantic similarity and custom weight feature similarity, and determining the target matching objects.
It realizes accurate matching of resources within e-commerce and different platforms, improves the accuracy and efficiency of resource information consistency judgment, and avoids the lack of matching results due to different platform specifications.
Smart Images
Figure CN119646532A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the fields of Internet technology and financial technology, and specifically to a multi-platform resource matching method, device, and readable storage medium. Background Art
[0002] With the development of e-commerce, there are more and more online shopping platforms. For the same products or services, there is a need to compare prices on multiple platforms. For example, in order to provide users with cost-effective services and products, an e-commerce platform needs to compare the prices of the services and products on the shelves with the same services and products on other e-commerce platforms. If a product is found to be priced too high, the reference price of the service product outside is provided to guide the operation staff to adjust the price of the high-priced product, which not only protects the rights and interests of customers but also increases the competitiveness of the enterprise.
[0003] In the prior art, multi-source service or product information is matched mainly through text similarity or text keywords. Although text similarity and keyword search can find related services or products, it is difficult to accurately locate the same products or services. Operators need to manually screen from multiple matched results. In addition, data from different platforms have different specifications, and the text descriptions of products and services are different, which may cause missing matching results on some platforms. Summary of the invention
[0004] In view of the above problems, the embodiments of the present application provide a multi-platform resource matching method, device, and readable storage medium to overcome or partially overcome the shortcomings of the prior art.
[0005] In a first aspect, the present application provides a multi-platform resource matching method, comprising:
[0006] Acquire multiple matching objects that are relevant to the resources to be matched from multiple e-commerce platforms;
[0007] Preprocessing the resource information of the plurality of matching objects, wherein the preprocessing includes: data cleaning, vectorization, entity extraction, and entity post-processing;
[0008] Based on the text semantic similarity, multiple matching objects are recalled and sorted to form a candidate matching object set;
[0009] Based on the self-defined weight feature similarity, multiple candidate matching objects in the candidate matching object set are re-ranked to obtain a refined matching object set;
[0010] Based on the relative number of e-commerce platforms and candidate matching objects, several target matching objects are determined from the refined matching object set.
[0011] In a second aspect, the present application further provides a multi-platform resource matching device, the device comprising:
[0012] An acquisition unit, used to acquire multiple matching objects that are relevant to the resources to be matched from multiple e-commerce platforms;
[0013] A preprocessing unit, configured to preprocess the resource information of the plurality of matching objects, wherein the preprocessing sequentially includes: data cleaning, vectorization, entity extraction, and entity post-processing;
[0014] A recall unit, used for recalling and sorting a plurality of matching objects from the plurality of matching objects based on text semantic similarity to form a candidate matching object set;
[0015] A refined ranking unit, used for re-ranking a plurality of candidate matching objects in the candidate matching object set based on a user-defined weight feature similarity to obtain a refined ranking matching object set;
[0016] The selection unit is used to determine a number of target matching objects from the refined matching object set based on the relative number of e-commerce platforms and candidate matching objects.
[0017] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor; and a memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes any of the above-mentioned multi-platform resource matching methods.
[0018] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs, and when the one or more programs are executed by an electronic device including multiple applications, the electronic device executes any of the above-mentioned multi-platform resource matching methods.
[0019] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:
[0020] This application first obtains a large number of matching objects that are relevant to the resources to be matched from multiple e-commerce platforms through rough retrieval, and then performs pre-processing such as data cleaning, vectorization, entity extraction, and entity post-processing on the resource information of these matching objects; then the following three-fold sorting and selection process is performed: first, based on the text semantic similarity, multiple matching objects are recalled and sorted from multiple matching objects to form a preliminary candidate matching object set; second, based on the self-defined weight feature similarity, multiple candidate matching objects in the above preliminary candidate matching object set are sorted twice to obtain a refined matching object set; third, based on the relative number of e-commerce platforms and candidate matching objects, several target matching objects are determined from the refined matching object set. This application can accurately match the internal resource information of e-commerce with the resource information from different platforms, output high-similarity matching results from multiple sources, and quickly locate the same resources from different sources, which not only saves a lot of manpower and time costs, but also improves the accuracy and efficiency of the consistency judgment of multi-source resource information, and is highly practical; and avoids the problem of missing matching results of individual platforms due to different platform specifications. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0022] Figure 1 A schematic diagram of a process of a multi-platform resource matching method according to an embodiment of the present application is shown;
[0023] Figure 2 A schematic diagram of the structure of a multi-platform resource matching device according to an embodiment of the present application is shown;
[0024] Figure 3 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION
[0025] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0026] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.
[0027] With the development of e-commerce, there are more and more online shopping platforms. For the same products or services, there is a need to compare prices on multiple platforms. For example, in order to provide users with cost-effective services and products, an e-commerce platform needs to compare the prices of the services and products on the shelves with the same services and products on other e-commerce platforms. If a product is found to be priced too high, the reference price of the service product outside is provided to guide the operation staff to adjust the price of the high-priced product, which not only protects the rights and interests of customers but also increases the competitiveness of the enterprise.
[0028] In the prior art, multi-source service or product information is matched mainly through text similarity or text keywords. Although text similarity and keyword search can find related services or products, it is difficult to accurately locate the same products or services. Operators need to manually screen from multiple matched results. In addition, data from different platforms have different specifications, and the text descriptions of products and services are different, which may cause missing matching results on some platforms.
[0029] In view of the problems in the prior art such as low matching accuracy, fuzzy positioning, and easy loss of goods or services, this application provides a multi-platform resource matching method. Figure 1 A schematic diagram of a multi-platform resource matching method according to an embodiment of the present application is shown. Figure 1 It can be seen that this embodiment includes steps S110 to S150:
[0030] Step S110, obtaining multiple matching objects that are relevant to the resources to be matched from multiple e-commerce platforms.
[0031] For the convenience of explanation, in the embodiment of the present application, the resource to be matched is recorded as resource to be matched A, which comes from e-commerce platform A.
[0032] In actual scenarios, the resources to be matched mainly refer to goods or services. The so-called "goods" usually refer to physical goods, such as shoes, hats, medicines, daily necessities, etc.; the so-called "services" usually refer to the professional support provided by enterprises or individuals to customers, such as maintenance services, financial services, consulting services, medical services, etc. This application does not limit the types and scope of application of the resources to be matched. The above is only an exemplary description.
[0033] First, based on the information of the resource to be matched, multiple matching objects that are relevant to the resource to be matched are obtained from multiple e-commerce platforms. The "acquisition" here can be understood as a rough and extensive acquisition, such as pulling from different e-commerce platforms through category search, obtaining information on all goods or services under the category, and taking each product or service as a matching object. The volume of data obtained in this way is usually very large, and it is not easy to miss through category search. For example, if the resource A to be matched is a down jacket, it can be searched through the down jacket category or by keyword search. This form can ensure data coverage.
[0034] The resources to be matched are not limited to commodities or services, and the following description is mainly based on commodities or services.
[0035] Step S120, preprocessing the resource information of the plurality of matching objects, wherein the preprocessing includes: data cleaning, vectorization, entity extraction, and entity post-processing.
[0036] Since the matching objects come from different platforms, the resource information of the matching objects is usually not standardized, and the resource information of the matching objects is structured preprocessed. Preferably, in some embodiments of the present application, the preprocessing includes but is not limited to: data cleaning, vectorization, entity extraction, and entity post-processing.
[0037] Specifically, in some embodiments, the data cleaning is performed according to the following method: if a key field value in a piece of resource information is missing, the corresponding matching object is deleted; if, except for the field date, all field values in multiple pieces of resource information are repeated, only one of the corresponding multiple matching objects is retained; if there is a field value type error or a field name error in a piece of resource information, the corresponding matching object is deleted.
[0038] Delete matching objects with missing key field values. A resource to be matched is usually described by multiple fields. Some of these fields are necessary and can be understood as key fields. If the information of a key field is missing, the data cannot be applied. For example, if the information of a key field such as product name, service description, SKU details, or price is missing, the corresponding matching object will be deleted.
[0039] Delete duplicate matching objects. If two or more matching objects have duplicate fields except the acquisition date, it means the data is duplicated. Only the latest acquisition date is retained.
[0040] Delete abnormal and erroneous data. Delete erroneous data in the data set. For example, if the prices of goods and services are non-numeric, or if the descriptions of goods and services contain descriptions such as "make up the price difference", "contact customer service to place an order", or "consult customer service", these are unreasonable, and the corresponding matching objects will be deleted.
[0041] After data cleaning, more standardized data without redundancy is obtained. Then vectorization processing is performed. In some embodiments of the present application, vectorization processing can be performed by the following method: use a pre-trained text embedding model to vectorize the resource information of the resources to be matched and the matching objects, and use the vector to represent the semantic information of the entire sentence. The input of the text embedding model is a text description, and the output is a multi-dimensional vector representation, such as a 768-dimensional vector. The text embedding model can adopt any model in the prior art, such as Word2Vec, BERT, GPT, etc.
[0042] Then, entity extraction is performed. It is recommended that, in some embodiments of the present application, the entity extraction is performed according to the following method: an entity extraction model is used to perform entity extraction on the resource information of the matching object. If the resource to be matched is a commodity, the extracted entity name includes: commodity name, commodity brand, model, specification, and color; if the resource to be matched is a service, the extracted entity name includes: service name, supplier, number of services, service duration, service area, and service city; wherein the entity extraction model is constructed based on Bert combined with CRF, and is trained using labeled entity data, wherein the labeling of the entity data is implemented using a generative large model.
[0043] Construct an entity extraction model. In this application, a dual entity extraction model of Bert combined with CRF is constructed. The main function of the Bert model is word segmentation, and the main function of CRF is scoring (which can be understood as output probability or confidence).
[0044] During training, the training data is first labeled. Preferably, the data can be labeled manually or automatically by a model. In some embodiments of the present application, semi-automatic labeling can be used, and a generative large model is used for pre-labeling. The input of the generative large model is a prompt (prompt question) and the text to be labeled, and the output is a text sequence labeled in the BIOES method (a marking system for labeling sequences). Then, the results of the large model labeling are manually reviewed and corrected. This semi-automatic method can ensure both the efficiency and accuracy of labeling.
[0045] Then, the entity extraction model is trained using the labeled training data. The general process of the data is as follows: the vectorized resources to be matched and the matching objects are input into the pre-trained Bert model to obtain the feature vector; then the feature vector is input into the CRF to obtain the entity and the corresponding score. Through the back propagation of these results, after multiple rounds of training, a trained entity extraction model is obtained, which can be used for career extraction. The multiple combination model can significantly improve the generalization ability of the model.
[0046] For different types of resources to be matched, different entities need to be extracted. In some embodiments of the present application, if the resource to be matched is a commodity, the extracted entity name includes: commodity name, commodity brand, model, specification, and color; if the resource to be matched is a service, the extracted entity name includes: service name, supplier, service times, service duration, service area, and service city. This application does not limit this and can be set according to business needs.
[0047] After extracting several entities, these entity data may also have redundancy or inconsistency problems or inconsistent units, etc., which will affect the subsequent recall. Therefore, in preprocessing, these problems can be solved through entity post-processing.
[0048] Specifically, in some embodiments of the present application, it is checked whether the extracted entity name is correct. If a product name or service name contains Chinese characters, and / or a service city is not in the preconfigured city list, and / or a specification does not contain numbers, it is determined that the extracted entity name is incorrect, and the corresponding matching object is deleted; it is checked whether there are synonyms for the same entity extracted from different resource information. If so, the synonyms are unified according to the preconfigured synonym standard table; it is checked whether the units of the same entity extracted from different resource information are the same. If not, the units are unified.
[0049] That is, post-processing mainly includes but is not limited to entity word verification, synonym replacement, and specification unit unification. The entity word verification stage is mainly used to check whether the extracted entity words are correct, such as checking whether the extracted product model contains Chinese characters, checking whether the extracted city is in the city list, and checking whether the specifications contain numbers. The synonym replacement stage replaces the extracted entity words with synonyms. For suppliers, product names, colors, etc., the same entity words may have different expressions, such as "vitamin C" and "vitamin C". For existing synonyms, the extracted entity words are replaced using the accumulated synonym table. The specification unit unification stage unifies the measurement of the extracted product specifications, service times, service time, and service area. For example, the extracted service time unit may be "hours" or "minutes", such as "1 hour" and "60 minutes". The extracted product weight specification unit may be "kg", "g", or "jin", such as "1kg", "1000g", and "2 jin". For entities with different measurement units, each measurement unit is unified into one through numerical conversion.
[0050] Step S130: based on the text semantic similarity, multiple matching objects are recalled and sorted to form a candidate matching object set.
[0051] After the entity is obtained through the above processing, the first recall of the matching object is performed, and this round of recall is based on the semantics of text similarity. Specifically, in some embodiments of the present application, based on the text semantic similarity, multiple matching objects are recalled and sorted to form a candidate matching object set, including: if the resource to be matched is a commodity, then find the matching object with the same commodity name and brand name as the resource to be matched from the multiple matching objects, and add it to the candidate matching object set; if the resource to be matched is a service, then find the matching object with the same service name and supplier as the resource to be matched from the multiple matching objects, and add it to the candidate matching object set; determine the cosine similarity value of the text vector of the resource to be matched and each matching object in the candidate matching object set, and sort each matching object in descending order according to the relative size of the cosine similarity value; intercept the multiple matching objects ranked first to form the candidate matching object set.
[0052] The goal of this step is to recall products and services that are highly similar to the products and services to be matched from multiple matching objects. If the resource to be matched is a product, then for the product to be matched, select products with the same "product name" and "brand name" as the product to be matched from multiple matching objects and put them into the candidate matching object set; if the resource to be matched is a service, then for the service to be matched, select services with the same "service name" and "supplier" as the service to be matched from multiple matching objects and put them into the candidate matching object set.
[0053] Then calculate the product and service to be matched and each product and service T in the candidate matching object set in turn. i ,i=1,2,…n, the value of the cosine similarity Cosine_Similarity(S,T) of the text description corresponding vector, and sort the multiple matching objects in the candidate matching object set in descending order according to the cosine similarity value from large to small.
[0054] Finally, the top N products and services with high similarity are selected from the candidate matching object set to form the final candidate matching object set, denoted as {T 1 ,T 2 ,…T N}.
[0055] Through text semantic similarity, similar products or services can be preliminarily selected from massive amounts of data, laying the foundation for further screening.
[0056] Step S140: re-ranking the plurality of candidate matching objects in the candidate matching object set based on the user-defined weight feature similarity to obtain a refined matching object set.
[0057] Then, secondary sorting is performed based on the user-defined weight feature similarity. Specifically, in some embodiments, the multiple candidate matching objects in the candidate matching object set are re-sorted based on the user-defined weight feature similarity to obtain a refined matching object set, including: determining the weight feature similarity value between the resource to be matched and each matching object in the candidate matching object set; sorting each matching object in descending order according to the relative size of the weight feature similarity value;
[0058] The calculation formula of the weighted feature similarity value is shown in Formula 1;
[0059] Similarity(S,T)=γ×Feature_Similarity(S,T)+(1-γ)×Cosine_Similarity(S,T) Formula (1);
[0060] Where, Similarity(S,T) represents the weighted feature similarity between the resource to be matched and the matching object; γ is used to reduce the weight of cosine similarity, 0.5<γ<1; Cosine_Similarity(S,T) is the cosine similarity between the resource to be matched and the matching object; Feature_Similarity(S,T) is the feature similarity between the resource to be matched and the matching object;
[0061] Among them, the feature similarity is shown in formula 2:
[0062]
[0063] Among them, w i represents the weight of the i-th feature, and S i represents the i-th feature of the resource to be matched, T i Represents the i-th feature of the matching object T; s i (S i ,T i ) represents the feature similarity of the i-th feature.
[0064] The goal of this step is to use a more refined similarity algorithm to calculate the similarity between the goods and services recalled in the previous step and the goods and services to be matched, and then re-sort them according to the new similarity.
[0065] In this regard, this application sets a custom weight feature similarity, the formula of which is shown in Formula 1;
[0066] Similarity(S,T)=γ×Feature_Similarity(S,T)+(1-γ)×Cosine_Similarity(S,T) Formula (1);
[0067] Among them, Similarity(S,T) represents the weighted feature similarity between the resource to be matched and the matching object; γ is used to reduce the weight of cosine similarity, 0.5<γ<1; Cosine_Similarity(S,T) is the cosine similarity between the resource to be matched and the matching object; Feature_Similarity(S,T) is the feature similarity between the resource to be matched and the matching object, and its calculation formula is shown in Formula 2:
[0068]
[0069] where w i represents the weight of the i-th feature, and w i Set it according to the importance of the feature. For unimportant features, a smaller weight can be set. i represents the i-th feature of the product or service S to be matched, T i represents the i-th feature of matching product and service T; s i (S i ,T i ) represents the similarity of the i-th feature, and 0≤s i (S i ,T i )≤1.
[0070] If the feature result is a single enumeration value, such as product model, product color, service times, etc., when a feature of the resource to be matched is the same as the corresponding feature of the matching object, then s i (S i ,T i )=1, otherwise s i (S i ,T i )=0.
[0071] If the feature result is multiple enumeration values, such as specifications and service cities, multiple values may appear in a product or service record. For this feature, the overlap degree is used to measure the similarity.
[0072] For a feature i, S i ,T i The calculation formula of the overlap degree (feature similarity) is shown in Formula 3:
[0073] s i (S i ,T i )=len(S i ∩T i ) / α i ×len(S i )+(1-α i )×len(T i ) formula (3);
[0074] where len(·) represents the number of elements, 0.5<α i <1,α i It is used to enhance the influence of feature i in the resource to be matched S on the similarity result.
[0075] If the resource to be matched is a commodity, the characteristics of the commodity to be matched are the processed model, specification, and color entities, that is, S i Select from model, specification, and color entities; if the resource to be matched is a service, the service characteristics to be matched are the number of services, service duration, service area, and service city entities after processing, that is, S i Selected from number of services, service duration, service area, and service city entities, etc.
[0076] Using the aforementioned custom weighted feature similarity formula, calculate the feature similarity value of the product or service to be matched with each product or service in the candidate matching object set in turn, and perform secondary sorting in descending order according to the feature similarity value from large to small to obtain a refined matching object set.
[0077] Step S150, based on the relative number of e-commerce platforms and candidate matching objects, a number of target matching objects are determined from the refined matching object set.
[0078] The goal of this step is to screen k matching results (k≥1) from the refined ranking matching object set reordered from step S140 of the refined ranking phase, and the returned results should come from different platforms as much as possible.
[0079] Specifically, in some embodiments of the present application, based on the relative number of e-commerce platforms and candidate matching objects, several target matching objects are determined from the refined ranking matching object set, including: determining whether the first number of retained matching objects is greater than a preset number threshold. If not, directly use the retained matching objects as the target matching objects; if so, determining whether the number threshold is less than or equal to the second number of the multiple e-commerce platforms. If so, intercept the number threshold of matching objects as the target matching objects according to the ranking in the refined ranking matching object set; if the number threshold is greater than the second number, use the one with the largest weight feature similarity value in each e-commerce platform as the target matching object, and supplement the target matching objects from the remaining matching objects according to the relative size of the weight feature similarity, and the supplementary number is the difference between the number threshold and the second number.
[0080] A similarity threshold is set in advance, and the matching objects with a weight feature similarity less than or equal to this similarity threshold in the refined ranking matching object set obtained in step S140 are excluded, and a part of the top-ranked ones are retained. Denote the number of retained matching objects as the first number.
[0081] If the first number is less than or equal to the number threshold k required for the matching results, directly use the first k matching objects in the ranking as the target matching objects and return them.
[0082] If the first number is greater than the number k required for the matching results, first calculate the number p of platforms that appear in the refined ranking matching object set, and denote this number as the second number p. Then compare the relative sizes of the second number p and the number threshold k. If p≥k, select the results of the number threshold k platforms from the candidate set in descending order of scores.
[0083] If p<k, select the result with the highest score for each platform as the target matching object. In this case, there are still k-p results missing. Then, select k-p matching objects from the remaining matching objects in descending order of the ranking as the target matching objects and return them.
[0084] By Figure 1It can be seen from the method shown that the present application first obtains a large number of matching objects that are relevant to the resources to be matched from a rough search of multiple e-commerce platforms, and then performs data cleaning, vectorization, entity extraction, and entity post-processing on the resource information of these matching objects; then the following three-fold sorting and selection process is performed: first, based on the text semantic similarity, multiple matching objects are recalled and sorted from multiple matching objects to form a preliminary candidate matching object set; second, based on the self-defined weight feature similarity, multiple candidate matching objects in the above preliminary candidate matching object set are sorted twice to obtain a refined matching object set; third, based on the relative number of e-commerce platforms and candidate matching objects, several target matching objects are determined from the refined matching object set. The present application can accurately match the internal resource information of e-commerce with the resource information from different platforms, output high-similarity matching results from multiple sources, and quickly locate the same resources from different sources, which not only saves a lot of manpower and time costs, but also improves the accuracy and efficiency of the consistency judgment of multi-source resource information, and is highly practical; and avoids the problem of missing matching results of individual platforms due to different platform specifications.
[0085] Figure 2 A schematic diagram of the structure of a multi-platform resource matching device according to an embodiment of the present application is shown. Figure 2 It can be seen that the multi-platform resource matching device 200 includes:
[0086] An acquisition unit 210 is used to acquire multiple matching objects that are relevant to the resources to be matched from multiple e-commerce platforms;
[0087] A preprocessing unit 220, configured to preprocess the resource information of the plurality of matching objects, wherein the preprocessing sequentially includes: data cleaning, vectorization, entity extraction, and entity post-processing;
[0088] A recall unit 230, configured to recall and sort multiple matching objects from the multiple matching objects based on text semantic similarity to form a candidate matching object set;
[0089] A refined ranking unit 240, configured to re-rank the plurality of candidate matching objects in the candidate matching object set based on a user-defined weight feature similarity to obtain a refined ranking matching object set;
[0090] The selection unit 250 is used to determine a number of target matching objects from the refined matching object set based on the relative number of e-commerce platforms and candidate matching objects.
[0091] In some embodiments of the present application, in the above-mentioned device, the data cleaning is performed according to the following method: if a key field value in a piece of resource information is missing, the corresponding matching object is deleted; if, except for the field date, all field values in multiple pieces of resource information are repeated, only one of the corresponding multiple matching objects is retained; if there is a field value type error or a field name error in a piece of resource information, the corresponding matching object is deleted.
[0092] In some embodiments of the present application, in the above-mentioned device, the entity extraction is performed according to the following method: an entity extraction model is used to perform entity extraction on the resource information of the matching object. If the resource to be matched is a commodity, the extracted entity name includes: commodity name, commodity brand, model, specification, and color; if the resource to be matched is a service, the extracted entity name includes: service name, supplier, number of services, service duration, service area, and service city; wherein the entity extraction model is constructed based on Bert combined with CRF, and is trained using labeled entity data, wherein the labeling of the entity data is implemented using a generative large model.
[0093] In some embodiments of the present application, in the above-mentioned device, the entity post-processing is performed according to the following method: check whether the extracted entity name is correct. If a product name or service name contains Chinese characters, and / or a service city is not in the preconfigured city list, and / or a specification does not contain numbers, it is determined that the extracted entity name is incorrect, and the corresponding matching object is deleted; detect whether there are synonyms for the same entity extracted from different resource information. If so, unify the synonyms according to the preconfigured synonym standard table; detect whether the units of the same entity extracted from different resource information are the same. If not, unify the units.
[0094] In some embodiments of the present application, in the above-mentioned device, the recall unit 230 is used to, if the resource to be matched is a commodity, find a matching object with the same commodity name and brand name as the resource to be matched from the multiple matching objects, and add it to the candidate matching object set; if the resource to be matched is a service, find a matching object with the same service name and supplier as the resource to be matched from the multiple matching objects, and add it to the candidate matching object set; determine the cosine similarity value of the text vector of the resource to be matched and each matching object in the candidate matching object set, and sort each matching object in descending order according to the relative size of the cosine similarity value; intercept multiple matching objects that are sorted first to form the candidate matching object set.
[0095] In some embodiments of the present application, in the above-mentioned device, the fine sorting unit 240 is used to determine the weight feature similarity value between the to-be-matched resource and each matching object in the candidate matching object set; and sort each matching object in descending order according to the relative size of the weight feature similarity value;
[0096] The calculation formula of the weighted feature similarity value is shown in Formula 1;
[0097] Similarity(S,T)=γ×Feature_Similarity(S,T)+(1-γ)×Cosine_Similarity(S,T) Formula (1);
[0098] Where, Similarity(S,T) represents the weighted feature similarity between the resource to be matched and the matching object; γ is used to reduce the weight of cosine similarity, 0.5<γ<1; Cosine_Similarity(S,T) is the cosine similarity between the resource to be matched and the matching object; Feature_Similarity(S,T) is the feature similarity between the resource to be matched and the matching object;
[0099] Among them, the feature similarity is shown in formula 2:
[0100]
[0101] Among them, w i represents the weight of the i-th feature, and S i represents the i-th feature of the resource to be matched, T i Represents the i-th feature of the matching object T; s i (S i ,T i ) represents the feature similarity of the i-th feature.
[0102] In some embodiments of the present application, in the above device, the fine sorting unit 240 is used to calculate the feature similarity of a feature as shown in Formula 3 if the feature result of a feature is multiple enumeration values:
[0103] s i (S i ,T i )=len(S i ∩T i ) / α i ×len(S i )+(1-α i )×len(T i ) formula (3);
[0104] where len(·) represents the number of elements, 0.5<α i <1,α i It is used to enhance the influence of feature i in the service to be matched S on the feature similarity result.
[0105] In some embodiments of the present application, in the above-mentioned device, the selection unit 250 is used to eliminate several matching objects whose weight feature similarity in the refined matching object set is less than or equal to a preset similarity threshold; determine whether the first number of retained matching objects is greater than the preset number threshold, if not, directly use the retained matching objects as the target matching objects; if so, determine whether the number threshold is less than or equal to the second number of the multiple e-commerce platforms, if so, according to the ranking in the refined matching object set, intercept the number threshold number of matching objects as the target matching objects; if the number threshold is greater than the second number, the one with the largest weight feature similarity value in each of the e-commerce platforms is used as the target matching object, and the target matching objects are supplemented from the remaining matching objects according to the relative size of the weight feature similarity, and the supplemented number is the difference between the number threshold and the second number.
[0106] It can be understood that the above-mentioned multi-platform resource matching device can implement the various steps of the multi-platform resource matching method provided in the aforementioned embodiment, and the relevant explanations on the multi-platform resource matching method are applicable to the multi-platform resource matching device, which will not be repeated here.
[0107] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 3 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.
[0108] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0109] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.
[0110] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a multi-platform resource matching device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:
[0111] Acquire multiple matching objects that are relevant to the resources to be matched from multiple e-commerce platforms;
[0112] Preprocessing the resource information of the plurality of matching objects, wherein the preprocessing includes: data cleaning, vectorization, entity extraction, and entity post-processing;
[0113] Based on the text semantic similarity, multiple matching objects are recalled and sorted to form a candidate matching object set;
[0114] Based on the self-defined weight feature similarity, multiple candidate matching objects in the candidate matching object set are re-ranked to obtain a refined matching object set;
[0115] Based on the relative number of e-commerce platforms and candidate matching objects, several target matching objects are determined from the refined matching object set.
[0116] It should be noted that the above-mentioned multi-platform resource matching device can implement the above-mentioned multi-platform resource matching method one by one, and will not be described in detail.
[0117] The above application Figure 2The method performed by the multi-platform resource matching device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in a decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.
[0118] The electronic device may also perform Figure 2 A method for executing a resource matching device for multiple platforms in the system, and realizing a resource matching device for multiple platforms in the system Figure 2 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.
[0119] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 2 The method executed by the multi-platform resource matching device in the illustrated embodiment is specifically used to execute:
[0120] Acquire multiple matching objects that are relevant to the resources to be matched from multiple e-commerce platforms;
[0121] Preprocessing the resource information of the plurality of matching objects, wherein the preprocessing includes: data cleaning, vectorization, entity extraction, and entity post-processing;
[0122] Based on the text semantic similarity, multiple matching objects are recalled and sorted to form a candidate matching object set;
[0123] Based on the self-defined weight feature similarity, multiple candidate matching objects in the candidate matching object set are re-ranked to obtain a refined matching object set;
[0124] Based on the relative number of e-commerce platforms and candidate matching objects, several target matching objects are determined from the refined matching object set. Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0125] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0126] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0127] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0128] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0129] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0130] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0131] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0132] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0133] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A multi-platform resource matching method, characterized in that: include: Acquire multiple matching objects that are relevant to the resources to be matched from multiple e-commerce platforms; Preprocessing the resource information of the plurality of matching objects, wherein the preprocessing includes: data cleaning, vectorization, entity extraction, and entity post-processing; Based on the text semantic similarity, multiple matching objects are recalled and sorted to form a candidate matching object set; Based on the self-defined weight feature similarity, multiple candidate matching objects in the candidate matching object set are re-ranked to obtain a refined matching object set; Based on the relative number of e-commerce platforms and candidate matching objects, several target matching objects are determined from the refined matching object set.
2. The method according to claim 1, characterized in that The data cleaning is performed according to the following method: If the key field value in a resource information is missing, the corresponding matching object is deleted; If, except for the date field, all other field values in multiple resource information are repeated, only one of the corresponding multiple matching objects is retained; If a field value type error or a field name error exists in a resource information, the corresponding matching object will be deleted.
3. The method according to claim 1, characterized in that The entity extraction is performed according to the following method: An entity extraction model is used to extract entity information from the resource information of the matching object. If the resource to be matched is a commodity, the extracted entity name includes: commodity name, commodity brand, model, specification, and color; if the resource to be matched is a service, the extracted entity name includes: service name, supplier, service number, service duration, service area, and service city; The entity extraction model is constructed based on Bert combined with CRF and is trained using annotated entity data, wherein the annotation of the entity data is implemented using a generative large model.
4. The method according to claim 3, characterized in that The entity post-processing is carried out according to the following method: Verify whether the extracted entity name is correct. If a product name or service name contains Chinese characters, and / or a service city is not in the preconfigured city list, and / or a specification does not contain a number, it is determined that the extracted entity name is incorrect, and the corresponding matching object is deleted; Detect whether there are synonyms for the same entity extracted from different resource information. If so, unify the synonyms using the pre-configured synonym standard table; Check whether the units of the same entity extracted from different resource information are the same. If they are not the same, unify the units.
5. The method according to claim 1, characterized in that The method of recalling and sorting multiple matching objects from the multiple matching objects based on text semantic similarity to form a candidate matching object set includes: If the resource to be matched is a commodity, a matching object having the same commodity name and brand name as the resource to be matched is found from the multiple matching objects, and the matching object is added to the candidate matching object set; if the resource to be matched is a service, a matching object having the same service name and supplier as the resource to be matched is found from the multiple matching objects, and the matching object is added to the candidate matching object set; Determine the cosine similarity value between the text vectors of the resource to be matched and each matching object in the candidate matching object set, and sort each matching object in descending order according to the relative size of the cosine similarity value; A plurality of matching objects ranked first are intercepted to form the candidate matching object set.
6. The method according to claim 1, characterized in that The method of reordering the plurality of candidate matching objects in the candidate matching object set based on the self-defined weight feature similarity to obtain a refined matching object set includes: Determine a weighted feature similarity value between the to-be-matched resource and each matching object in the candidate matching object set; Sorting the matching objects in descending order according to the relative magnitudes of the weighted feature similarity values; The calculation formula of the weighted feature similarity value is shown in Formula 1; Similarity(S,T)=γ×Feature_Similarity(S,T)+(1-γ)×Cosine_Similarity(S,T) Formula (1); Where, Similarity(S,T) represents the weighted feature similarity between the resource to be matched and the matching object; γ is used to reduce the weight of cosine similarity, 0.5<γ<1; Cosine_Similarity(S,T) is the cosine similarity between the resource to be matched and the matching object; Feature_Similarity(S,T) is the feature similarity between the resource to be matched and the matching object; Wherein, the feature similarity is as shown in Formula 2: Among them, w i represents the weight of the i-th feature, and S i represents the i-th feature of the resource to be matched, T i represents the i-th feature of the matching object T; s i (S i ,T i ) represents the feature similarity of the i-th feature.
7. The method according to claim 6, characterized in that If the feature result of a feature is multiple enumeration values, the formula for the feature similarity of the feature is as shown in Formula 3: s i (S i ,T i ) = len(S i ∩T i ) / α i × len(S i ) + (1 - α i ) × len(T i ) Equation (3); where len(·) represents the number of elements, 0.5<α i <1,α i It is used to enhance the influence of feature i in the service to be matched S on the feature similarity result.
8. The method according to claim 1, characterized in that The method of determining a number of target matching objects from the refined matching object set based on the relative number of e-commerce platforms and candidate matching objects includes: Eliminate a number of matching objects whose weighted feature similarity is less than or equal to a preset similarity threshold value in the refined matching object set; Determine whether the first number of the reserved matching objects is greater than a preset number threshold, and if not, directly use the reserved matching objects as the target matching objects; If so, determining whether the quantity threshold is less than or equal to the second quantity of the plurality of e-commerce platforms; if so, intercepting the matching objects of the quantity threshold as the target matching objects according to the ranking in the refined matching object set; If the quantity threshold is greater than the second quantity, the one with the largest weight feature similarity value in each of the e-commerce platforms will be used as the target matching object, and the target matching objects will be supplemented among the remaining matching objects according to the relative size of the weight feature similarity, and the supplementary quantity is the difference between the quantity threshold and the second quantity.
9. A multi-platform resource matching device, characterized in that: The device comprises: An acquisition unit, used to acquire multiple matching objects that are relevant to the resources to be matched from multiple e-commerce platforms; A preprocessing unit, configured to preprocess the resource information of the plurality of matching objects, wherein the preprocessing sequentially includes: data cleaning, vectorization, entity extraction, and entity post-processing; A recall unit, used for recalling and sorting a plurality of matching objects from the plurality of matching objects based on text semantic similarity to form a candidate matching object set; A refined ranking unit, used for re-ranking a plurality of candidate matching objects in the candidate matching object set based on a user-defined weight feature similarity to obtain a refined ranking matching object set; The selection unit is used to determine a number of target matching objects from the refined matching object set based on the relative number of e-commerce platforms and candidate matching objects.
10. A computer-readable storage medium storing one or more programs, characterized in that: When the one or more programs are executed by an electronic device including a plurality of application programs, the electronic device executes the multi-platform resource matching method according to any one of claims 1 to 8.