Generation method and device for query tag data, electronic equipment and storage medium

Abstract

The invention relates to a query tag data generation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining initial retrieval data, obtaining first query tag data matched with the initial retrieval data from a commodity transaction log, screening the first query tag data based on a preset heuristic rule, and obtaining a first query tag data set; obtaining second query label data, determining a reasonable probability value of the second query label data through a preset discrimination model, and determining the second query label data as target query label data when the reasonable probability value reaches a set threshold value. Therefore, the query label data matched with the retrieval data is screened through the preset discrimination model, the query label data matched with the user demand is mined, and the problem of recall omission caused by a semantic gap is solved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of data mining, and in particular to a method, device, electronic device and storage medium for generating query tag data. Background Art

[0002] In related technologies, with the continuous development of the field of artificial intelligence, knowledge graphs are seen as an important cornerstone to support the development of artificial intelligence. At the same time, with the continuous breakthroughs in various applications of artificial intelligence technology, the application of knowledge graphs has also received more and more attention. Knowledge engines are regarded as the brains of machines in the era of artificial intelligence, and are constantly accelerating the implementation of various vertical applications. The combination of knowledge graphs and e-commerce platforms is based on e-commerce product data. Using knowledge graph technology, the labels, scenes, core product words, attribute feature words, synonyms, etc. of products are calculated and associated, thereby building a real-time product graph for mining product hypernyms, synonyms, and scene concept words. At the same time, based on user preferences, purchase history, browsing footprints, repeat purchase rates and other characteristics, products of interest to users are recommended, providing targeted content for search and recommendation, thereby accurately meeting user needs and improving the value of commercial applications. Summary of the invention

[0003] In order to overcome the problems existing in the related art, the present disclosure provides a method, device, electronic device and storage medium for generating query tag data.

[0004] According to a first aspect of an embodiment of the present disclosure, a method for generating query tag data is provided, comprising:

[0005] Get initial search data;

[0006] Acquire first query tag data matching the initial search data from the commodity transaction log;

[0007] Filter the first query tag data based on a preset heuristic rule to obtain second query tag data;

[0008] Determining a reasonable probability value of the second query tag data by using a preset discriminant model, wherein the preset discriminant model is used to identify the query tag data and generate a reasonable probability value, and the reasonable probability value is used to characterize the accuracy of the query tag data;

[0009] When the reasonable probability value reaches a set threshold, the second query tag data is determined to be the target query tag data.

[0010] Optionally, the preset discriminant model is used to linearly combine the features of the second query tag data to generate a first vector, and encode the semantic information of the second query tag data to generate a second vector, and after splicing the first vector and the second vector, generate the reasonable probability value.

[0011] Optionally, linearly combining the features of the second query tag data to generate a first vector includes:

[0012] Performing feature recognition on the second query tag data to determine feature data of the second query tag data, the feature data including: commodity attribute feature data, length feature data, part-of-speech combination feature data, perplexity data, and popularity value data;

[0013] The feature data are linearly combined and encoded to generate the first vector.

[0014] Optionally, the feature data includes perplexity data, and the performing feature recognition on the second query tag data to determine the feature data of the second query tag data includes:

[0015] Performing feature recognition on the second query tag data by using a preset language model to determine first perplexity data;

[0016] Performing feature recognition on the second query tag data by using a preset product corpus model to determine second perplexity data;

[0017] The perplexity data is determined according to the first perplexity data and the second perplexity data.

[0018] Optionally, the feature data includes popularity value data, and the performing feature recognition on the second query tag data to determine the feature data of the second query tag data includes:

[0019] generating first popular value data according to the number of matches between the second query tag data and a preset general corpus;

[0020] generating second popularity value data according to the number of matches between the second query tag data and a preset commodity corpus;

[0021] The popularity value data is determined according to the first popularity value data and the second popularity value data.

[0022] Optionally, encoding the semantic information of the second query tag data to generate a second vector includes:

[0023] Encoding the semantic information of the second query tag data to generate initial semantic information;

[0024] Determining contextual semantic information of the second query tag data according to the initial semantic information;

[0025] The second vector is generated according to the initial semantic information and the context semantic information.

[0026] Optionally, the method further comprises:

[0027] Rewriting the initial search data according to a preset rewriting rule to generate a plurality of third query tag data;

[0028] Determining multiple semantic similarities between the multiple third query tag data and the target query tag data;

[0029] Filter out a plurality of fourth query tag data having a semantic similarity greater than a similarity threshold from the plurality of third query tag data;

[0030] A product recommendation list is generated according to the plurality of fourth query tag data and the target query tag data.

[0031] Optionally, obtaining first query tag data matching the initial search data from the commodity transaction log includes:

[0032] Acquire a plurality of fifth query tag data matching the initial search data from the commodity transaction log;

[0033] Obtaining a plurality of page views corresponding to the plurality of fifth query tag data one by one;

[0034] According to the plurality of page views, the first query tag data having a page view amount greater than a set view amount threshold is determined from the plurality of fifth query tag data.

[0035] Optionally, the preset heuristic rule includes:

[0036] If the first query tag data contains commodity attribute characteristics, determining the first query tag data as the second query tag data;

[0037] If the first query tag data matches a preset commodity word list and / or a preset brand word list, determining that the first query tag data is not the second query tag data;

[0038] If the part-of-speech combination of the first query tag data matches a preset part-of-speech combination, the first query tag data is determined to be the second query tag data.

[0039] According to a second aspect of an embodiment of the present disclosure, a device for generating query tag data is provided, including:

[0040] A first acquisition module is configured to acquire initial search data;

[0041] A second acquisition module is configured to acquire first query tag data matching the initial search data from the commodity transaction log;

[0042] A third acquisition module is configured to filter the first query tag data based on a preset heuristic rule to obtain second query tag data;

[0043] a determination module configured to determine a reasonable probability value of the second query tag data through a preset discriminant model, wherein the preset discriminant model is used to linearly combine features of the second query tag data to generate a first vector, and encode semantic information of the second query tag data to generate a second vector, and concatenate the first vector and the second vector to generate the reasonable probability value;

[0044] The generating module is configured to determine that the second query tag data is the target query tag data when the reasonable probability value reaches a set threshold.

[0045] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0046] a memory having a computer program stored thereon;

[0047] A processor is used to execute the computer program in the memory to implement the steps of any one of the methods described in the first aspect of the present disclosure.

[0048] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored, and when the program instructions are executed by a processor, the steps of any one of the methods described in the first aspect of the present disclosure are implemented.

[0049] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0050] Through the above method, the initial search data is obtained, the first query label data matching the initial search data is obtained from the commodity transaction log, the first query label data is screened based on the preset heuristic rules, the second query label data is obtained, and the reasonable probability value of the second query label data is determined by the preset discriminant model, wherein the preset discriminant model is used to identify the query label data and generate a reasonable probability value, and the reasonable probability value is used to characterize the accuracy of the query label data. When the reasonable probability value reaches the set threshold, the second query label data is determined to be the target query label data. Thus, the query label data matching the search data is screened by the preset discriminant model, and the query label data matching the user needs is mined, solving the missed recall problem caused by the semantic gap.

[0051] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0053] Figure 1 is a flowchart of an unsupervised method according to an exemplary embodiment.

[0054] Figure 2 The figure is a flowchart of a method for generating query tag data according to an exemplary embodiment.

[0055] Figure 3 It is a schematic diagram showing a space collapse problem according to an exemplary embodiment.

[0056] Figure 4 The figure is a flowchart of a method for generating a vector according to an exemplary embodiment.

[0057] Figure 5 is a schematic diagram of a preset discrimination model according to an exemplary embodiment.

[0058] Figure 6 The figure is a block diagram of a device for generating query tag data according to an exemplary embodiment.

[0059] Figure 7 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0060] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0061] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0062] In the practice of improving the search experience, the mining of tagquery (tag query data) is an important task. In the relevant technology, query (retrieval data) refers to the text entered by the user in the purchase search box. Since the product title is short and concise, when the user searches for the target product, the product title cannot completely cover all the attributes in the user's query, resulting in the inability to recall the corresponding product or the user's expected product is ranked low, thus affecting the user's search experience. For example, when searching for "gaming mobile phone" in the query, the click-through rate of product results based on the query is currently only 33%, far less than the actual recall expectation.

[0063] Tag query mining is usually based on the expansion and mining of queries based on models. In related technologies, tag query mining can be summarized into two categories: the first category is an unsupervised method based on dictionaries, linguistic characteristics, and heuristic rules; the second category is a supervised learning method based on machine learning models, mainly based on sequence labeling models.

[0064] Figure 1 is a flowchart of an unsupervised method according to an exemplary embodiment. Figure 1 As shown, the method includes the following steps.

[0065] The search data in the user query database is identified by sentence tagging, word segmentation tagging, part-of-speech tagging, etc., and the dependency analysis is performed on the identification results; the scene words are extracted from the analysis results based on the heuristic rule set, and the scene word compliance is judged based on the custom filtering rule set, and finally the mining results of the tag query are output. The part-of-speech plus rule is adopted as a preliminary method that is easier to implement. According to the heuristic rules, the text word segmentation and part-of-speech can be annotated. After the word segmentation and part-of-speech analysis, many non-compliant query phrases and texts that are too colloquial and not written enough are excluded, for example, adverbs are followed by verbs (such as: installment payment, installment interest-free, etc.), and verbs are followed by auxiliary words (such as: used in cars, children's food, etc.). The advantages of this method are fast and easy to understand. The rule-based learning method can extract and learn frequent rules or text patterns from a large amount of natural language text data. By extracting information from texts using heuristic rules or custom dictionaries, frequent patterns of information to be mined can be summarized from the corpus, thus achieving higher extraction accuracy. However, due to the limitations of the corpus, the learned patterns are relatively fixed and lack scalability.

[0066] This mining method faces problems such as the complexity of Chinese grammar and the many intrinsic semantic connections between words. When faced with the casualness and colloquialism of user query language, as well as many grammatically irregular sentences, the extraction of linguistic features is not accurate enough, thus affecting the accuracy of tag query mining.

[0067] In the related art, another tag query mining method is also provided. In the supervised learning method of the machine learning model, the tag query mining task can be regarded as a text sequence labeling problem. The sequence labeling model can learn the text relationship between sequence positions and decode the result path of the maximum probability reasonable label. For example, there are k output labels, if the input sequence length is n, then there are K n The problem of sequence labeling is to select the path with the highest probability from these probability paths as the reasonable path for tag query mining. The CRF (Conditional Random Field, natural language processing) sequence labeling model can capture contextual information well and more comprehensively consider the linear weighted summation result of the local features of the entire text. It is one of the machine learning methods with better results in information extraction tasks. In the field of natural language processing, CRF can be used to solve many common tasks such as word segmentation, named body recognition, and part-of-speech tagging.

[0068] However, after combining word features, part-of-speech features, word spacing features and dependency syntactic features based on CRF for corpora from multiple different fields, it was found that the accuracy, recall rate and F1 value (F1 score, a comprehensive evaluation indicator) results are greatly affected by the field, that is, they are closely related to the scale of the training corpus in the field. The larger the training corpus, the better the effect. When the diversity of the training corpus is low, the tag query mining effect is poor and it is impossible to recall products that meet user needs.

[0069] Based on the above-mentioned problems in related fields, this patent makes the attributes corresponding to the query data into tags, and specifically solves the problem that the user query data containing attribute tags (including: product features, selling points, etc.) cannot recall the corresponding products or the expected products are ranked at the back, so as to obtain better recall and sorting effects, improve product information by mining tag queries, and enhance the search experience. For example, Figure 2 is a flowchart of a method for generating query tag data according to an exemplary embodiment. Figure 2 As shown, applied to a terminal, the method includes the following steps.

[0070] In step S11, initial search data is acquired.

[0071] By way of example, this embodiment is applied to the terminal to collect the initial search data entered by the user in the search box. The search box may be the input position corresponding to the search text when the user searches for the desired product through the search text in a shopping app, and the search text is the initial search data. By way of example, the search scenario may not be limited to the product search scenario, but may also be applicable to other application scenarios that analyze the user's search intent based on the user's search content and return the desired product data to the user through the analysis results, including: app search scenarios, content search scenarios, and data search scenarios.

[0072] The initial search data is text data input by the user for retrieving the desired product. Different users have different usage habits, and the corresponding initial search data may be different. The terminal can determine the user's intention based on the initial search data and the user's behavior in the usage scenario, and return the desired product data to the user based on the user's intention.

[0073] In step S12, first query tag data matching the initial search data is obtained from the commodity transaction log.

[0074] Normally, the user will enter the initial search data in the search dialog box to retrieve the desired product data. The terminal analyzes the user's intention based on the initial search data entered by the user and returns a matching recommended product list to the user. The user selects the target product that meets the needs based on the returned recommended product list and completes the product transaction. Therefore, in this embodiment, the terminal can establish a corresponding relationship between the product transaction log and the query tag data. The product transaction log can include the transaction data completed by all users under the app based on the search content. From the product transaction dimension, the first query tag data that matches the initial search data entered by the user is obtained from a large number of user query transaction logs.

[0075] After the terminal receives the initial search data input by the user, it identifies the product based on the initial search data to determine the type of product the user wishes to search for. Normally, for the same product, different users enter different initial search data in the search box. The product transaction log records the search data of all users, that is, the correspondence between a product and multiple first query label data can be recorded in the product transaction log. After the corresponding product is determined through the search data, the first query label data of the product type can be determined based on the product transaction log through the correspondence.

[0076] It should be noted that in this embodiment, the first query tag data corresponding to the commodity transaction log is selected as the extended data of the initial search data because the commodity transaction log is more concise than the user search log, and the correspondence between the commodity and the query is more accurate.

[0077] Optionally, in one implementation, step S12 includes:

[0078] Acquire a plurality of fifth query tag data matching the initial search data from the commodity transaction log;

[0079] Obtain multiple page views corresponding to multiple fifth query tag data one by one;

[0080] According to the plurality of page views, the first query tag data having a page view amount greater than a set view amount threshold is determined from the plurality of fifth query tag data.

[0081] For example, in this embodiment, there are accidental and personalized query tags in the query tag data corresponding to the initial search data. In order to ensure the accuracy of the query tag data during the query tag mining process, the query tag data corresponding to the initial search data needs to be cleaned and processed, thereby improving the accuracy of the first query tag data.

[0082] In this embodiment, the commodity category corresponding to the initial search data is first determined, and then the fifth query tag data corresponding to the commodity category is obtained from the commodity transaction log. Data is screened by the page views corresponding to the query tag data, wherein the page views reflect the degree to which the commodity list returned by the query tag data meets the user's needs. According to the multiple page views corresponding to the multiple fifth query tag data, the multiple fifth query tag data are screened, and the fifth query tag data with a page view greater than the view threshold is selected as the first query tag data. For example, in the commodity transaction log, the multiple fifth query label data corresponding to the commodity "No. 6 children's electric scooter" include: "Electric scooter 1690", "Scooter 629", "Scooter children 574", "Children's scooter 444", "Electric skateboard 365", "Children's electric scooter 202", "Children's electric car 146", "Electric car cool play skateboard 136", "Balance car children 125"..., where the numbers represent the page views under the query label data text, and the page views are frequency data, that is, the search PV (Page View) value. Based on the page views, multiple fifth query label data are screened. For example, the page views threshold can be set based on different data cleaning standards, and the query label data with a page views threshold greater than 200 is selected as the first query label data corresponding to the initial retrieval data.

[0083] Through the above method, the query tag data matching the retrieval data is cleaned based on the page views, and some query tag data with page views below the page views threshold are removed, so that the obtained query tag data is more accurate, which is convenient for the subsequent recall of the products expected by the user based on the query tag data.

[0084] In step S13, the first query tag data is screened based on a preset heuristic rule to obtain second query tag data.

[0085] For example, in this embodiment, after the first query tag data is determined through the above steps, in order to improve the accuracy of the query tag data, the query tag data needs to be screened. In this embodiment, a series of heuristic rules are used to filter the second query tag data from the first query tag data. Among them, the preset heuristic rules are based on relevant information, past search results and experience, and the word grouping rules, sentence structure and word meaning information in the accurate query tag data are identified to generate the screening rules for the query tag data.

[0086] Optionally, in another implementation, the preset heuristic rule includes:

[0087] If the commodity attribute feature exists in the first query tag data, determining the first query tag data as the second query tag data;

[0088] If the first query tag data matches the preset commodity word list and / or the preset brand word list, determining that the first query tag data is not the second query tag data;

[0089] If the part-of-speech combination of the first query tag data matches the preset part-of-speech combination, the first query tag data is determined to be the second query tag data.

[0090] For example, in this embodiment, a series of heuristic rules are used to filter out the second query tag data from the first query tag data. The data quality is ensured by preset heuristic rules, and the second query tag data that meets the identification requirements is obtained. For example, the preset heuristic rules include:

[0091] (1) Identify the product attribute features in the first query tag data. When the product attribute features exist in the first query tag data, determine that the first query tag data is the second query tag data; product attributes are an important part of the tag query, reflecting the important features of the user's intended products. Filter out some representative attributes and their corresponding attribute values ​​in the product map, including applicable population attributes, efficacy attributes, applicable season attributes and style attributes. If the first query tag data contains attribute values, it can be used as a candidate for the tag query, and the first query tag data with product attribute features is used as the second query tag data. For example, if the first query tag data is: children's tables and chairs, compressed biscuits, summer short sleeves, business shirts, "children" is the population attribute, "compressed" and "business" are efficacy attributes, and "summer" is the applicable season attribute, then the first query tag data can be directly used as the second query tag data.

[0092] (2) In the current era of abundant commodities, pure commodity words correspond to more commodity options, so the commodity options based on pure commodity words are more complicated, resulting in pure commodity words affecting the query accuracy of the query tag. Therefore, the appropriate query tag data cannot refer specifically to a certain commodity and cannot be a pure commodity word. Therefore, in this embodiment, the first query tag data is filtered based on the preset commodity word table to remove the pure commodity words. For example, if the first query tag data is a relatively broad pure commodity word such as "mobile phone", "thermos cup", "computer", etc., the first query tag data is discarded.

[0093] (3) When searching for products, users usually use brand search to carry the brand name of the required brand in the first query tag data to search for products. However, when the query tag data contains a brand name, the search is more limited and the applicability of the corresponding query tag data is lower. Therefore, in this embodiment, a preset brand word table is compared with the first query tag data. If the first query tag data contains a brand word, the first query tag data is discarded.

[0094] (4) Using the part-of-speech and lexical rules of the three-party Chinese word segmentation database as an aid, the first query tag data is screened. For example, part-of-speech combinations of participle + verb, numeral + numeral + noun + noun, noun + adjective, place name + noun + noun, etc. in the first query tag data cannot form a reasonable query tag data, while the part-of-speech combination of adjective + noun can form a reasonable query tag data. Therefore, the part-of-speech combination in the first query tag data is identified, and the query tag that satisfies the part-of-speech combination of adjective + noun is screened out as the second query tag data.

[0095] For example, it is worth mentioning that in this embodiment, one or more of the above preset heuristic rules can be selected based on the data cleaning standard to filter the first query tag data. When the data cleaning requirement is high, the corresponding filtered second query tag data satisfies the above four preset heuristic rules. The first query tag data is filtered by the preset heuristic rules to improve the accuracy of the second query tag data.

[0096] In step S14, a reasonable probability value of the second query tag data is determined by a preset discriminant model, wherein the preset discriminant model is used to identify the query tag data and generate a reasonable probability value, and the reasonable probability value is used to characterize the accuracy of the query tag data;

[0097] For example, in this embodiment, after the second query tag data is screened out through the above steps, the second query tag data is judged for rationality through a preset discriminant model to determine a reasonable probability value of the second query tag data. Among them, the reasonable probability value is used to characterize the accuracy of the query tag data. When the reasonable probability value of the query tag data is less than the set threshold, it means that the query tag data is inaccurate, and the products that meet the user's expectations cannot be recalled through the query tag data, so the query tag data needs to be discarded; when the reasonable probability value of the query tag data is greater than or equal to the set threshold, it means that the query tag data meets the rationality requirements, and the products that meet the user's expectations can be recalled through the query tag data.

[0098] Optionally, the preset discriminant model is used to linearly combine the features of the second query label data to generate a first vector, and encode the semantic information of the second query label data to generate a second vector, and after splicing the first vector and the second vector, a reasonable probability value is generated.

[0099] For example, in this embodiment, the preset discriminant model is used to linearly combine the features of the second query label data to generate a first vector, and encode the semantic information of the second query label data to generate a second vector. The first vector and the second vector are concatenated and enter the fully connected layer, and a reasonable probability value of the second query label data is output through the fully connected layer.

[0100] In step S15, when the reasonable probability value reaches a set threshold, the second query tag data is determined to be the target query tag data.

[0101] In this embodiment, a threshold is set for the reasonable probability value of the query tag data, and the second query tag data is screened based on the reasonable probability value. When the reasonable probability value reaches the set threshold, the second query tag data is determined to be the target query tag data. The second query tag data corresponding to the reasonable probability value that meets the condition is screened from the second query tag data according to the set threshold as the target query tag data. The terminal can recall a list of products that meet the user's expectations based on the target query tag data, thereby increasing the user's transaction probability.

[0102] Through the above method, the initial search data is obtained, the first query label data matching the initial search data is obtained from the commodity transaction log, the first query label data is screened based on the preset heuristic rules, the second query label data is obtained, and the reasonable probability value of the second query label data is determined by the preset discriminant model, wherein the preset discriminant model is used to identify the query label data and generate a reasonable probability value, and the reasonable probability value is used to characterize the accuracy of the query label data. When the reasonable probability value reaches the set threshold, the second query label data is determined to be the target query label data. Thus, the query label data matching the search data is screened by the preset discriminant model, and the query label data matching the user needs is mined, solving the missed recall problem caused by the semantic gap.

[0103] Optionally, in another embodiment, after the above step S15, the method further includes:

[0104] Rewriting the initial search data using a preset rewriting rule to generate a plurality of third query tag data;

[0105] Determining multiple semantic similarities between multiple third query tag data and target query tag data;

[0106] Filtering out a plurality of fourth query tag data having a semantic similarity greater than a similarity threshold from the plurality of third query tag data;

[0107] A product recommendation list is generated according to the plurality of fourth query tag data and the target query tag data.

[0108] For example, in this embodiment, in order to improve the accuracy of recalled products and meet the actual expectations of users, the initial search data needs to be rewritten and expanded to increase the richness of the search text. The application method of rewriting the initial search data is to expand the query tag data, generate rewritten words with high correlation with user needs, and search multiple rewritten words together with user search words, so as to better express user expectations and help users retrieve more products that meet their needs to solve the problem of missed recall caused by the semantic gap. The preset rewriting rules include: semantic extension, links between user expressions and merchant expressions, and related missed recall problems. Semantic expansion mainly includes synonyms, hyponyms, and common uppercase and lowercase numbers and traditional and simplified Chinese conversions, for example: synonyms are expanded according to grammatical rules, such as "cooling patch", "fever patch", "cooling patch", "heat dissipation patch", etc.; the link between user expression and merchant expression is based on the different expressions of the same product at the user level and at the merchant level, and the expression vocabulary input by the user is expanded through the link relationship, for example: the "soup bag" in the user's expression is described as "soup ingredients" at the merchant level. Based on the link relationship, "soup bag" is expanded to "soup ingredients"; the related missing recall problem is to perform semantic analysis on the initial search data with missing words, and recall the initial search data according to the semantic information and contextual semantic information of the initial search data, for example: the search data "children's warm" and "children's lining" correspond to the recall of "children's warm" and "children's shirt".

[0109] For example, in this embodiment, the initial search data is expanded and rewritten by the preset rewriting rules to generate multiple third query label data. In this embodiment, the search terms related to the products corresponding to the initial search data can be used to build a correlation relationship by using the similar clicks, product details and the initial search data. The rewritten words can be mined through the correlation relationship. For example, the product "No. 6 children's electric scooter" corresponds to the search terms: "electric scooter", "scooter electric", "scooter children", "children's scooter", "electric skateboard", "children's electric scooter", etc. can be rewritten results. The synonym dictionary and automatic rewriting module can also be used to rewrite the initial search data in fine-grained segments to obtain high-precision rewritten words. For example, when it is determined through the synonym dictionary that "double door" and "side door" are synonyms, "double door refrigerator" can be rewritten as "side door refrigerator". For the problem of missed recall, the semantic information of the initial search data can be identified based on the BERT model, and the initial search data with missing words can be completed through the semantic information and rewritten into complete query label data.

[0110] For example, there are erroneous rewriting or rewriting deviation problems in the generated multiple third query tag data, so the multiple third query tag data need to be screened. In this embodiment, the multiple third query tag data are compared with the target query tag data for similarity, and the target query tag data generated in the above steps are used to screen the multiple third query tag data to determine the semantic similarity between each query tag data and the target query tag data. Multiple fourth query tag data with semantic similarity greater than the similarity threshold are screened from the multiple third query tag data.

[0111] In the process of judging the rewriting results, the main focus is on judging the similarity between the third query label data and the target query label data, including basic similarity features such as word level, similarity coefficient, and Hamming distance. The third query label data and the target query label data can be converted into sentence vectors through feature analysis, and then the sentence vectors are used to determine the semantic similarity between the target query label data and the third query label data using the improved contrastive learning method of simCSE.

[0112] It should be noted that when similarity verification is performed based on the BERT model, since the BERT model tends to encode all sentences into a smaller spatial area, most sentence pairs have high similarity scores. Even semantically irrelevant sentence pairs can collapse in this spatial area. For example, Figure 3 is a schematic diagram of a space collapse problem according to an exemplary embodiment. Figure 3 As shown in the figure, the left side is the similarity distribution determined by BERT model recognition, and the right side is the similarity distribution determined by contrastive learning, where the horizontal axis in the coordinate system is the similarity score and the vertical axis is the cosine similarity predicted by the model. The similarity results calculated by contrastive learning have higher discrimination and accuracy than the similarity results determined by BERT model recognition.

[0113] For example, based on multiple semantic similarities corresponding to multiple third query tag data, multiple fourth query tag data with semantic similarities greater than a similarity threshold are filtered out from the third query tag data, and then based on the multiple fourth query tag data and the target query tag data, products are recalled to generate a product recommendation list that meets user expectations.

[0114] Through the above method, the initial retrieval data is expanded, and then the rationality of the query label data generated by the expansion is judged based on the target query label data to generate a rewriting result with higher rationality. This can significantly solve the missed recall problem caused by the semantic gap and recall products that meet the needs of users.

[0115] Figure 4is a flow chart of a vector generation method according to an exemplary embodiment. Figure 4 As shown, when applied to the terminal, the above step of "linearly combining the features of the second query tag data to generate a first vector" includes:

[0116] In step S141, feature recognition is performed on the second query tag data to determine feature data of the second query tag data, the feature data including: commodity attribute feature data, length feature data, part-of-speech combination feature data, perplexity data and popularity value data.

[0117] In step S142, the feature data is linearly combined and encoded to generate a first vector.

[0118] In this embodiment, the preset discriminant model includes BERT (Bidirectional Encoder Representation from Transformers, an unsupervised pre-trained language model). The BERT model is a deep bidirectional encoding text based on the Transformer model. It mainly goes through two steps of model pre-training and task fine-tuning to determine the suitability of the second query label data screened and determined in the above steps.

[0119] The second query tag data is judged by a preset discriminant model, and the preset discriminant model is mixed based on the structure of wide (linear model) and deep learning model (deep). For example, Figure 5 is a schematic diagram of a preset discrimination model according to an exemplary embodiment. Figure 5 As shown, the preset discriminant model includes a linear model structure and a deep learning model structure. The wide part is a generalized linear model. Taking advantage of the simplicity and interpretability of the wide model, a linear combination of a series of important features of the user query is used to generate the first vector corresponding to the query label data. For example, the important features include: whether it contains product words, whether it contains attribute features, the length feature of the query label data, the part-of-speech combination feature of the query label data, the ppl (perplexity) value and pop value (popular value data) of the query label data. According to the feature value of the feature data corresponding to the second query label data, the feature data is linearly combined and encoded to generate the first vector of the second query label data.

[0120] Optionally, in one implementation, the feature data includes perplexity data, and the step S141 includes:

[0121] Performing feature recognition on the second query tag data by using a preset language model to determine the first perplexity data;

[0122] Performing feature recognition on the second query tag data by using a preset product corpus model to determine second perplexity data;

[0123] The perplexity data is determined based on the first perplexity data and the second perplexity data.

[0124] For example, the preset discriminant model in this embodiment includes two BERT models, one is a common preset language model BERT, and the other is a commodity corpus model GoodsBERT generated based on commodity corpus training. The second query label data is feature-recognized by the BERT model to determine the second perplexity data, and the second query label data is feature-recognized by the Goods BERT model to determine the second perplexity data, wherein the perplexity data may be a perplexity value, and the higher the perplexity, the worse the rationality of the second query label data. In this embodiment, the first perplexity data and the second perplexity data may be weighted averaged to generate perplexity data corresponding to the second query label data.

[0125] The BERT model identifies the universal perplexity features of the second query label data based on the universal grammatical rules of the query label data and generates the PPL value of the language model. The Goods BERT model is a language model generated by training the commodity corpus. The model can identify the second perplexity of the second query label data from the perspective of the commodity corpus and generate the corresponding Goods PPL value. The PPL value and the Goods PPL value are then weighted averaged to determine the perplexity data of the second query label data.

[0126] Optionally, in another embodiment, the characteristic data includes popular value data, and the above step S141 includes:

[0127] Generate first popular value data according to the number of matches between the second query tag data and the preset general corpus;

[0128] Generate second popular value data according to the number of matches between the second query tag data and the preset commodity corpus;

[0129] The popularity value data is determined according to the first popularity value data and the second popularity value data.

[0130] For example, the preset discrimination model of this embodiment is configured with a preset general corpus and a preset commodity corpus, the preset general corpus records the general search terms of various commodities, and the preset commodity corpus is configured with various commodity search terms. The second query label data is compared with the preset general corpus, and the first popular value data is generated according to the matching degree between the second query label data and the preset general corpus, the second query label data is compared with the preset commodity corpus, and the second popular value data is generated according to the matching number, and the first popular value data and the second popular value data are weighted to generate the popular value data corresponding to the second query label data.

[0131] Pop value and goods pop value refer to the number of times the second query label data appears in the general corpus and the goods corpus. The more matches there are, the more reasonable the second query label data is, and the higher the probability that it corresponds to appropriate query label data. In this embodiment, the popular value features of the second query label data are determined by the general corpus and the preset goods corpus, and the popular value data is determined according to the degree of matching.

[0132] Through the above steps, the wide model in the preset discriminant model is used to perform linear combination of the attribute features in the query label data, and the first vector corresponding to the query label data is generated according to the characteristic value of the query label data on the corresponding attribute, so that the query label data is characterized by the attribute features, and the rationality of the query label data in terms of part of speech is determined using the first vector.

[0133] Optionally, in another implementation manner, the above “encoding the semantic information of the second query tag data to generate a second vector” includes:

[0134] Encoding the semantic information of the second query tag data to generate initial semantic information;

[0135] Determining contextual semantic information of the second query tag data according to the initial semantic information;

[0136] A second vector is generated according to the initial semantic information and the context semantic information.

[0137] For example, the preset discriminant model of this embodiment is configured with a Deep feedforward neural network. The second query tag data is first input into the BERT model of the Deep feedforward neural network to encode the semantic information of the query tag data and generate initial semantic information. In order to enhance the concept of context and allow the model to learn richer information, the initial semantic information output by the BERT model is input into the BiLSTM (Bi-directional Long Short-Term Memory) of the Deep feedforward neural network. It should be noted that in the process of model training, the LSTM model in the BiLSTM can learn to choose which training data information to remember or forget during the training process, so it can capture the longer-distance dependencies in the training samples, and the BiLSTM model can learn the semantic information of the two directions before and after the sentence. After obtaining the sentence encoding information of the context information, based on the semantic information and the context semantic information, the second vector corresponding to the query data is generated.

[0138] For example, in this embodiment, LSTM is used to determine the dependency relationship between each word in the query tag data, and the BiLSTM model is used to determine the contextual semantic information in the query tag data in both the front and back directions of the sentence based on the dependency relationship, and the second vector of the query tag data is generated based on the initial semantic information and the contextual semantic information.

[0139] Through the above method, a feedforward neural network is used to identify semantic features in query tag data, and the query tag data is annotated in combination with the contextual semantics of the semantic features to generate a second vector for representing the semantic information of the query tag data, thereby accurately representing the semantic information of the query tag data.

[0140] Figure 6 is a block diagram of a device for generating query tag data according to an exemplary embodiment. Figure 6 As shown, the generating device 100 includes: a first acquiring module 110 , a second acquiring module 120 , a third acquiring module 130 , a determining module 140 and a generating module 150 .

[0141] A first acquisition module 110 is configured to acquire initial search data;

[0142] The second acquisition module 120 is configured to acquire first query tag data matching the initial search data from the commodity transaction log;

[0143] The third acquisition module 130 is configured to filter the first query tag data based on a preset heuristic rule to obtain the second query tag data;

[0144] The determination module 140 is configured to determine a reasonable probability value of the second query tag data through a preset discriminant model, wherein the preset discriminant model is used to identify the query tag data and generate a reasonable probability value, and the reasonable probability value is used to characterize the accuracy of the query tag data;

[0145] The generating module 150 is configured to determine the second query tag data as the target query tag data when the reasonable probability value reaches a set threshold.

[0146] Optionally, the preset discriminant model is used to linearly combine the features of the second query label data to generate a first vector, and encode the semantic information of the second query label data to generate a second vector, and after splicing the first vector and the second vector, a reasonable probability value is generated.

[0147] Optionally, the determination module 140 includes:

[0148] A determination submodule is configured to perform feature recognition on the second query tag data to determine feature data of the second query tag data, wherein the feature data includes: commodity attribute feature data, length feature data, part-of-speech combination feature data, perplexity data, and popularity value data;

[0149] The generating submodule is configured to linearly combine and encode the feature data to generate a first vector.

[0150] Optionally, the feature data includes perplexity data, and the determination submodule is configured as follows:

[0151] Performing feature recognition on the second query tag data by using a preset language model to determine the first perplexity data;

[0152] Performing feature recognition on the second query tag data by using a preset product corpus model to determine second perplexity data;

[0153] The perplexity data is determined based on the first perplexity data and the second perplexity data.

[0154] Optionally, the characteristic data includes popularity value data, and the determination submodule is configured to:

[0155] Generate first popular value data according to the number of matches between the second query tag data and the preset general corpus;

[0156] Generate second popular value data according to the number of matches between the second query tag data and the preset commodity corpus;

[0157] The popularity value data is determined according to the first popularity value data and the second popularity value data.

[0158] Optionally, the determination module 140 is configured to:

[0159] Encoding the semantic information of the second query tag data to generate initial semantic information;

[0160] Determining contextual semantic information of the second query tag data according to the initial semantic information;

[0161] A second vector is generated according to the initial semantic information and the context semantic information.

[0162] Optionally, the apparatus 100 further includes an execution module, which is configured to:

[0163] Rewriting the initial search data using a preset rewriting rule to generate a plurality of third query tag data;

[0164] Determining multiple semantic similarities between multiple third query tag data and target query tag data;

[0165] Filtering out a plurality of fourth query tag data having a semantic similarity greater than a similarity threshold from the plurality of third query tag data;

[0166] A product recommendation list is generated according to the plurality of fourth query tag data and the target query tag data.

[0167] Optionally, the second acquisition module 120 is configured to:

[0168] Acquire a plurality of fifth query tag data matching the initial search data from the commodity transaction log;

[0169] Obtain multiple page views corresponding to multiple fifth query tag data one by one;

[0170] According to the plurality of page views, the first query tag data having a page view amount greater than a set view amount threshold is determined from the plurality of fifth query tag data.

[0171] Optionally, the preset heuristic rules include:

[0172] If the commodity attribute feature exists in the first query tag data, determining the first query tag data as the second query tag data;

[0173] If the first query tag data matches the preset commodity word list and / or the preset brand word list, determining that the first query tag data is not the second query tag data;

[0174] If the part-of-speech combination of the first query tag data matches the preset part-of-speech combination, the first query tag data is determined to be the second query tag data.

[0175] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0176] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, and when the program instructions are executed by a processor, the steps of the method for generating query tag data provided by the present disclosure are implemented.

[0177] Figure 7 7 is a block diagram of an electronic device according to an exemplary embodiment. For example, the electronic device 700 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0178] Reference Figure 7 , the electronic device 700 may include one or more of the following components: a processing component 702 , a memory 704 , a power component 706 , a multimedia component 707 , an audio component 710 , an input / output interface 712 , a sensor component 714 , and a communication component 716 .

[0179] The processing component 702 generally controls the overall operation of the electronic device 700, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 702 may include one or more processors 720 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 702 may include one or more modules to facilitate the interaction between the processing component 702 and other components. For example, the processing component 702 may include a multimedia module to facilitate the interaction between the multimedia component 707 and the processing component 702.

[0180] The memory 704 is configured to store various types of data to support operations on the electronic device 700. Examples of such data include instructions for any application or method operating on the electronic device 700, contact data, phone book data, messages, pictures, videos, etc. The memory 704 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0181] The power supply component 706 provides power to the various components of the electronic device 700. The power supply component 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 700.

[0182] The multimedia component 707 includes a screen that provides an output interface between the electronic device 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 707 includes a front camera and / or a rear camera. When the electronic device 700 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0183] The audio component 710 is configured to output and / or input audio signals. For example, the audio component 710 includes a microphone (MIC), and when the electronic device 700 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 704 or sent via the communication component 716. In some embodiments, the audio component 710 also includes a speaker for outputting audio signals.

[0184] The input / output interface 712 provides an interface between the processing component 702 and the peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0185] The sensor assembly 714 includes one or more sensors for providing various aspects of status assessment for the electronic device 700. For example, the sensor assembly 714 can detect the open / closed state of the electronic device 700, the relative positioning of components, such as the display and keypad of the electronic device 700, and the sensor assembly 714 can also detect the position change of the electronic device 700 or a component of the electronic device 700, the presence or absence of user contact with the electronic device 700, the orientation or acceleration / deceleration of the electronic device 700, and the temperature change of the electronic device 700. The sensor assembly 714 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 714 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 714 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0186] The communication component 716 is configured to facilitate wired or wireless communication between the electronic device 700 and other devices. The electronic device 700 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 716 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 716 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0187] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0188] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 704 including instructions, and the instructions can be executed by a processor 720 of an electronic device 700 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0189] In addition to being an independent electronic device, the above-mentioned device can also be a part of an independent electronic device. For example, in one embodiment, the device can be an integrated circuit (IC) or a chip, wherein the integrated circuit can be an IC or a collection of multiple ICs; the chip can include but is not limited to the following types: GPU (Graphics Processing Unit), CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), SOC (System on Chip, SoC, system on chip or system-level chip), etc. The above-mentioned integrated circuit or chip can be used to execute executable instructions (or codes) to implement the above-mentioned query tag data generation method. The executable instructions can be stored in the integrated circuit or chip, or can be obtained from other devices or equipment, such as the integrated circuit or chip includes a processor, a memory, and an interface for communicating with other devices. The executable instruction can be stored in the memory, and when the executable instruction is executed by the processor, the above-mentioned method for generating query tag data is implemented; alternatively, the integrated circuit or chip can receive the executable instruction through the interface and transmit it to the processor for execution, so as to implement the above-mentioned method for generating query tag data.

[0190] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program executable by a programmable device. The computer program has a code portion for executing the above-mentioned method for generating query tag data when executed by the programmable device.

[0191] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0192] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for generating query tag data, characterized in that: include: Get initial search data; Acquire first query tag data matching the initial search data from the commodity transaction log; Filter the first query tag data based on a preset heuristic rule to obtain second query tag data; Determining a reasonable probability value of the second query tag data by using a preset discriminant model, wherein the preset discriminant model is used to identify the query tag data and generate a reasonable probability value, and the reasonable probability value is used to characterize the accuracy of the query tag data; When the reasonable probability value reaches a set threshold, the second query tag data is determined to be the target query tag data.

2. The method according to claim 1, characterized in that The preset discriminant model is used to linearly combine the features of the second query tag data to generate a first vector, and encode the semantic information of the second query tag data to generate a second vector, and after splicing the first vector and the second vector, generate the reasonable probability value.

3. The generation method according to claim 2, characterized in that: The linearly combining the features of the second query tag data to generate a first vector includes: Performing feature recognition on the second query tag data to determine feature data of the second query tag data, the feature data including: commodity attribute feature data, length feature data, part-of-speech combination feature data, perplexity data, and popularity value data; The feature data are linearly combined and encoded to generate the first vector.

4. The generation method according to claim 3, characterized in that: The feature data includes perplexity data, and the performing feature recognition on the second query tag data to determine the feature data of the second query tag data includes: Performing feature recognition on the second query tag data by using a preset language model to determine first perplexity data; Performing feature recognition on the second query tag data by using a preset product corpus model to determine second perplexity data; The perplexity data is determined according to the first perplexity data and the second perplexity data.

5. The generation method according to claim 3, characterized in that: The characteristic data includes popular value data, and the performing characteristic recognition on the second query tag data to determine the characteristic data of the second query tag data includes: generating first popular value data according to the number of matches between the second query tag data and a preset general corpus; generating second popularity value data according to the number of matches between the second query tag data and a preset commodity corpus; The popularity value data is determined according to the first popularity value data and the second popularity value data.

6. The generation method according to claim 1, characterized in that: The encoding of the semantic information of the second query tag data to generate a second vector includes: Encoding the semantic information of the second query tag data to generate initial semantic information; Determining contextual semantic information of the second query tag data according to the initial semantic information; The second vector is generated according to the initial semantic information and the context semantic information.

7. The generation method according to claim 1, characterized in that: The method further comprises: Rewriting the initial search data according to a preset rewriting rule to generate a plurality of third query tag data; Determining multiple semantic similarities between the multiple third query tag data and the target query tag data; Filter out a plurality of fourth query tag data having a semantic similarity greater than a similarity threshold from the plurality of third query tag data; A product recommendation list is generated according to the plurality of fourth query tag data and the target query tag data.

8. The generation method according to claim 1, characterized in that: The step of acquiring first query tag data matching the initial search data from the commodity transaction log includes: Acquire a plurality of fifth query tag data matching the initial search data from the commodity transaction log; Obtaining a plurality of page views corresponding to the plurality of fifth query tag data one by one; According to the plurality of page views, the first query tag data having a page view amount greater than a set view amount threshold is determined from the plurality of fifth query tag data.

9. The generation method according to any one of claims 1 to 8, characterized in that: The preset heuristic rules include: If the first query tag data contains commodity attribute characteristics, determining the first query tag data as the second query tag data; If the first query tag data matches a preset commodity word list and / or a preset brand word list, determining that the first query tag data is not the second query tag data; If the part-of-speech combination of the first query tag data matches a preset part-of-speech combination, the first query tag data is determined to be the second query tag data.

10. A device for generating query tag data, characterized in that: include: A first acquisition module is configured to acquire initial search data; A second acquisition module is configured to acquire first query tag data matching the initial search data from the commodity transaction log; A third acquisition module is configured to filter the first query tag data based on a preset heuristic rule to obtain second query tag data; A determination module is configured to determine a reasonable probability value of the second query tag data through a preset discriminant model, wherein the preset discriminant model is used to identify the query tag data and generate a reasonable probability value; The generating module is configured to determine that the second query tag data is the target query tag data when the reasonable probability value reaches a set threshold.

11. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the method described in any one of claims 1 to 9 are implemented.

Images