Text push method, device, electronic device and storage medium
Through deep learning models, the text collection associated with the target category is obtained in the news data, and the confidence and semantic similarity are combined to sort it, which solves the problem of keyword matching in the prior art that does not consider semantics, and improves the mapping of news data and Sorting accuracy reduces maintenance costs and manual intervention.
Patent Information
- Application Number
- CN202310369222.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-07
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2043-04-07
AI Technical Summary
In the prior art, when mapping and sorting news data, keyword matching does not take into account text semantics, resulting in low accuracy and high keyword maintenance costs. Manual strategies are affected by data quality and low sorting accuracy.
Using a natural language processing model based on deep learning, we use the target text collection associated with the target category, determine the candidate text based on confidence and similarity, calculate the semantic similarity in combination with TF-IDF weights, and perform fine sorting and recommendation.
It improves the accuracy and sorting accuracy of news data category classification, reduces keyword maintenance costs, reduces manual intervention, and enhances the reliability and precision of text recommendations.
Smart Images

Figure CN116881432B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to natural language processing, deep learning, and pre-training model technology, which can be applied in smart city and smart government scenarios. More specifically, the present disclosure provides a text push method, device, electronic device, and storage medium. Background Art
[0002] The Internet generates tens of millions of pieces of news, event reports and other data every day. How to push and display massive amounts of news data is a hot topic. Summary of the invention
[0003] The present disclosure provides a text recommendation method, apparatus, device and storage medium.
[0004] According to a first aspect, a text recommendation method is provided, the method comprising: obtaining a target text set associated with a target category, wherein the target category includes a keyword group and the target text has a confidence associated with the target category; determining a plurality of candidate texts from the target text set according to the confidence; for each candidate text, determining a similarity between the candidate text and the keyword group; determining at least one candidate text from the plurality of candidate texts as a recommended text associated with the target category according to the similarity and the confidence; and pushing the recommended text.
[0005] According to a second aspect, a text recommendation device is provided, which includes: a target text acquisition module, used to acquire a target text set associated with a target category, wherein the target category includes a keyword group and the target text has a confidence associated with the target category; a candidate text determination module, used to determine multiple candidate texts from the target text set according to the confidence; a similarity determination module, used to determine the similarity between the candidate text and the keyword group for each candidate text; a recommended text determination module, used to determine at least one candidate text from the multiple candidate texts as a recommended text associated with the target category according to the similarity and the confidence; and a push module, used to push the recommended text.
[0006] According to a third aspect, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method provided according to the present disclosure.
[0007] According to a fourth aspect, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute the method provided according to the present disclosure.
[0008] According to a fifth aspect, a computer program product is provided, comprising a computer program, wherein the computer program is stored on at least one of a readable storage medium and an electronic device, and wherein the computer program implements the method provided according to the present disclosure when executed by a processor.
[0009] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.
[0011] Figure 1 is a schematic diagram of an exemplary system architecture to which a text push method and apparatus can be applied according to an embodiment of the present disclosure;
[0012] Figure 2 is a flowchart of a text push method according to an embodiment of the present disclosure;
[0013] Figure 3 is a flowchart of a text push method according to another embodiment of the present disclosure;
[0014] Figure 4 is a system diagram of a text push method according to an embodiment of the present disclosure;
[0015] Figure 5 is a block diagram of a text push device according to an embodiment of the present disclosure;
[0016] Figure 6 The present invention is a block diagram of an electronic device of a text push method according to an embodiment of the present invention. DETAILED DESCRIPTION
[0017] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.
[0018] News data may include text data such as event reports. To push news data, you can first map the news data to the corresponding category (or label), and push and display the news data according to the category. For example, the categories may include sports, entertainment, government affairs, etc. By dividing the news data according to the category, you can get sports news data, entertainment news data, government affairs news data, etc. Next, you can sort the news data under each category by relevance, and push and display the data most relevant to the category, so that users can understand the key news information under the corresponding category more accurately and quickly.
[0019] At present, the method of mapping news data to the corresponding category generally adopts the method of keyword matching based on preset keywords. For example, in the sports category system, the category is "sports", and the preset keywords under this category include ["football", "basketball", "table tennis", "star", ...]. The existing mapping method is to traverse each news data. If this data hits one or more keywords in the keyword group corresponding to "sports", the news data is mapped to the "sports" category. The formulation of keyword groups often requires professional personnel to select, and the maintenance cost of keywords in the later stage is high, including the addition, deletion, and update of keywords.
[0020] This keyword matching method hardly considers the semantic information of the text itself, and it is easy to produce mapping errors, so the accuracy is often not high. In addition, news data changes rapidly, and keyword groups also need to be updated accordingly, which has a relatively high update cost.
[0021] In addition, the current method of sorting the relevance of news data under a category generally uses some manually formulated rules and strategies to score each piece of news data, and then sorts them according to the scores. For example, according to the number of keywords hit in each piece of data, the corresponding score increases for each additional keyword hit, and then combines some data release time, data length and other text features to perform comprehensive sorting.
[0022] However, the scoring mechanism of the method of using manually formulated strategies for relevance ranking is easily affected by data quality. For example, repeated descriptions in a certain part of the text will lead to a falsely high number of keywords in the text. The physical characteristics of the text (such as non-semantic features such as length and release time) are also difficult to reflect the relevance between the text and the category, so the ranking accuracy is low.
[0023] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0024] In the technical solution of the present disclosure, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.
[0025] Figure 1 is a schematic diagram of an exemplary system architecture to which a text push method and apparatus can be applied according to an embodiment of the present disclosure. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.
[0026] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.
[0027] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Terminal devices 101, 102, 103 can be various electronic devices, including but not limited to smart phones, tablet computers, laptops, etc.
[0028] The text push method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the text push device provided in the embodiment of the present disclosure can generally be set in the server 105. The text push method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the text push device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.
[0029] Figure 2 is a flowchart of a text push method according to an embodiment of the present disclosure.
[0030] like Figure 2 As shown, the text push method 200 includes operations S210 to S250.
[0031] In operation S210 , a target text set associated with a target category is acquired.
[0032] For example, news data generated daily may include multiple categories, such as "sports", "entertainment", "government affairs", etc. News data may include news text.
[0033] Each category can be preset with a keyword group, which includes multiple keywords. For example, the keyword group of "sports" includes ["football", "basketball", "table tennis", "star", ...]. News texts under each category can be stored in a database, and by querying the database using the keywords of the category, a text collection associated with the category can be obtained.
[0034] For example, the target category may be "sports". By querying the database using one or more keywords of "football", "basketball", "table tennis", and "star", a text collection under the "sports" category may be retrieved from the database as the target text collection.
[0035] In operation S220 , a plurality of candidate texts are determined from the target text set according to the confidence levels.
[0036] For example, each news text in the database has a confidence level, which indicates the relevance between the news text and the category to which it belongs. Specifically, the category of news text a is "sports", and the confidence level of news text a can indicate the probability that news text a belongs to the "sports" category.
[0037] The category and confidence of the news text can be obtained through a natural language processing model based on deep learning. The natural language processing model can be trained using historical news texts, which have annotated category labels. Through training, the model learns the commonality of semantic representations of news texts belonging to the same category, so that the model has the ability to classify news texts.
[0038] For example, for the new news texts generated daily, the trained natural language processing model can be input for classification to obtain the categories of the news texts. The model can also output the probability that the news text belongs to each category, and the category corresponding to the maximum probability value is the category to which the news text belongs. The maximum probability value can be used as the confidence of the news text.
[0039] The news text content, category and confidence level under each category can be stored in the database.
[0040] When the target text set associated with the target category is recalled from the database using the keywords of the target category, the target texts may also be roughly sorted based on the confidence, for example, the target texts may be sorted in descending order of confidence.
[0041] The first N (N is an integer greater than 1, for example, N=100) target texts may be selected from the rough sorting results as candidate texts, and the candidate texts are used for subsequent further fine sorting and text recommendation.
[0042] In operation S230 , for each candidate text, a similarity between the candidate text and the keyword group is determined.
[0043] For example, a keyword group can be represented as a keyword string [keyword a-keyword b-keyword c]. By performing word segmentation processing on the candidate text, multiple word segments of the candidate text can be obtained, and the word segmentation string constituting the candidate text is, for example, [word segmentation a-word segmentation b-word segmentation c-word segmentation d-......].
[0044] For each keyword in the keyword string [keyword a-keyword b-keyword c], the TF-IDF (Term Frequency Inverse Document Frequency) weight of the keyword can be calculated based on the frequency of the keyword in the candidate text. The higher the TF-IDF weight of the keyword, the greater the semantic contribution of the keyword to the candidate text. According to the weights of the multiple keywords in the keyword group, the feature vector of the keyword group can be determined.
[0045] For example, the weight of keyword a is 0.75, the weight of keyword b is 0.1, and the weight of keyword c is 0.65. It can be determined that the feature vector of the keyword group is [0.75, 0.10, 0.65].
[0046] Similarly, for each word in the word string [word a-word b-word c-word d-......] of the candidate text, the TF-IDF weight of the word can be calculated according to the frequency of the word in the candidate text. The higher the TF-IDF weight of the word, the greater the semantic contribution of the word to the candidate text. According to the weights of multiple words in the word string, the feature vector of the candidate text can be determined.
[0047] For example, the weight of word a is 0.70, the weight of word b is 0.25, the weight of word c is 0.15, and the weight of word d is 0.50. The feature vector of the candidate text can be determined to be [0.70, 0.25, 0.15, 0.50, ...].
[0048] According to the similarity between the feature vector of the keyword group and the feature vector of the candidate text, since the larger the TF-IDF weight in the feature vector, the greater the semantic contribution of the keyword or word segment to the candidate text, the similarity between the feature vector of the keyword group and the feature vector of the candidate text can be used as the semantic similarity between the keyword group and the candidate text.
[0049] In operation S240 , at least one candidate text is determined from the plurality of candidate texts as a recommended text associated with the target category according to the similarity and the confidence.
[0050] In operation S250, the recommended text is pushed.
[0051] The similarity and confidence can be combined to jointly determine the comprehensive evaluation value of the candidate text, and multiple candidate texts can be precisely sorted based on the comprehensive evaluation value. For example, the candidate texts can be sorted in descending order according to the comprehensive evaluation value.
[0052] The first M (M is an integer greater than 1, for example, M=20) candidate texts may be selected from the refined sorting results as recommended texts, and the recommended texts may be used for push and display.
[0053] The confidence is obtained by a natural language processing model based on deep learning. The natural language processing model divides the text and determines the confidence based on the semantic commonalities of texts belonging to the same category. Therefore, the confidence is also related to the semantics of the candidate text.
[0054] Therefore, the comprehensive evaluation value of the candidate text is determined by combining similarity and confidence. The comprehensive evaluation value can reflect the correlation between the candidate text and the category from a semantic level. Compared with the sorting method based on manual strategies, this embodiment can improve the sorting accuracy by precisely sorting multiple candidate texts based on the comprehensive evaluation value of semantic relevance.
[0055] This embodiment precisely sorts the candidate texts according to the confidence of the candidate texts and the similarity between the candidate texts and the category keyword groups. The sorting result is more precise and reliable, which can improve the text recommendation effect.
[0056] Figure 3 is a flowchart of a text push method according to another embodiment of the present disclosure.
[0057] like Figure 3 As shown, the text push method 300 includes operations S310 to S330, operations S341 to S342, operations S351 to S353, and operations S360 to S380.
[0058] In operation S310 , a category of the original text is determined, and a confidence level associated with the category of the original text is determined.
[0059] The original text may be a new news text generated daily. The original text may include a title and content. The original text may be classified using a trained natural language processing model based on deep learning.
[0060] For example, the title and content of the original text can be concatenated together and input into a natural language processing model to obtain the category of the original text and the probability that the original text belongs to each category.
[0061] For example, the natural language processing model can output the probability that the original text belongs to sports, the probability that it belongs to entertainment, and the probability that it belongs to government affairs. If the probability that it belongs to sports is the largest, the category of the original text can be determined to be sports, and the probability that the original text belongs to sports can be used as the confidence of the original text.
[0062] In operation S320, the original text, the category of the original text, and the confidence level are added to the database.
[0063] For the raw text newly generated daily, after the category is determined, the text content, category and confidence of the raw text can be stored in a database in association with each other. The database is, for example, an Elastic Search database.
[0064] Table 1 below shows the type and meaning of each field of the original text stored in the database.
[0065] Table 1
[0066] Field Name Type Name meaning id keyword Original text logo text text Original text content industry keyword Original text category confidence double Confidence insert_time data Original text insertion time
[0067] As shown in Table 1, each original text has an identifier, and the identifier, text content, confidence, and insertion time of the same original text can be associated and stored in the database.
[0068] In operation S330 , a target text set associated with the target category is acquired from a database using the keywords of the target category, and a plurality of candidate texts are determined from the target text set according to confidence levels.
[0069] Operation S330 is used to recall and roughly sort the stock data in the database.
[0070] Recall means inputting keywords of the target category in the database retrieval platform. The database retrieval platform can filter all the data in the database according to the input keywords to obtain an initial text set belonging to the target category as the target text set.
[0071] For example, the target category may be "sports", and the keywords of the target category may be one or more keywords in the keyword group of the target category, such as one or more of "football", "basketball", and "table tennis". You may also directly use "sports" as a keyword for query to obtain a target text set of the sports category.
[0072] Rough sorting refers to sorting the recalled target text set from high to low according to the confidence of the target text to obtain a rough sorting result.
[0073] The purpose of recall and rough sorting is to determine the candidate range of data to be pushed from the full amount of data in the database. For example, the first N (for example, N=100) target texts in the rough sorting results can be selected as candidate texts from tens of thousands of full amount of data.
[0074] The above operations S310 to S330 use a deep learning natural language processing model to determine the category and confidence of the original text. Compared with the related art in which the text category is determined by determining whether the text hits a keyword (keyword matching), this embodiment uses the semantic commonality of texts belonging to the same category learned by the model to classify the text. The classification results and confidence can better reflect the relevance between the text and the category, thereby improving the accuracy of category determination. In addition, the use of a deep learning model to classify the text category only requires one-time manual annotation data, and there is no restriction on the professional level of the annotation personnel, and there is no need to repeat the maintenance of keyword groups in the later stage.
[0075] Next is the fine sorting step, which includes the steps of determining the feature vector of the keyword group in operations S341 and S342, determining the feature vector of the candidate text in operations S341 and S342, and calculating the similarity between the keyword group and the candidate text in operation S360.
[0076] In operation S341 , for each keyword, a weight of the keyword is determined according to the word frequency of the keyword in the candidate texts and the number of candidate texts containing the keyword.
[0077] The keyword group includes multiple keywords, for example, keyword a, keyword b and keyword c. Multiple keywords are concatenated to obtain a keyword string, for example, [keyword a-keyword b-keyword c].
[0078] For each keyword, the TF-IDF weight of the keyword relative to the candidate text can be calculated. TF-IDF includes TF (term frequency) and IDF (inverse document frequency). TF-IDF can be expressed as follows:
[0079] TF-IDF=TF*IDF (1)
[0080] For example, when calculating the TF-IDF weight of keyword a, the TF of keyword a = the number of times keyword a appears in the current candidate text / the number of all word segments in the current candidate text, where all word segments in the current candidate text are obtained by performing word segmentation processing on the current candidate text.
[0081] The IDF of keyword a=log(|the number of all candidate texts| / (the number of candidate texts containing keyword a+1)).
[0082] Using the above formula (1), the TF-IDF weight of keyword a can be calculated.
[0083] Similarly, the TF-IDF weight of keyword b in the keyword group, the TF-IDF weight of keyword c, and so on may also be calculated.
[0084] The higher the TF-IDF weight of a keyword, the greater the semantic contribution of the keyword to the candidate text.
[0085] In operation S342, the characteristics of the keyword group are determined according to the weights of the plurality of keywords.
[0086] The TF-IDF weights of multiple keywords in a keyword group can form a feature vector of the keyword group. For example, if the weight of keyword a is 0.75, the weight of keyword b is 0.1, and the weight of keyword c is 0.65, the feature vector of the keyword group can be determined to be [0.75, 0.10, 0.65].
[0087] In operation S351, the candidate text is divided into a plurality of word segments.
[0088] For example, NLTK (Natural Language Toolkit) can be used to perform word segmentation on the recalled candidate text to obtain multiple word segments. Multiple word segments of the candidate text are concatenated to obtain a word segmentation string of the candidate text, such as [word segmentation a-word segmentation b-word segmentation c-word segmentation d-......].
[0089] In operation S352 , for each segmented word, a weight of the segmented word is determined according to the word frequency of the segmented word in the candidate texts and the number of candidate texts containing the segmented word.
[0090] For example, when calculating the TF-IDF weight of word a, the TF of word a = the number of occurrences of word a in the current candidate text / the number of all words in the current candidate text.
[0091] IDF of word a = log(|the number of all candidate texts| / (the number of candidate texts containing word a + 1)).
[0092] Using the above formula (1), the TF-IDF weight of word segment a can be calculated.
[0093] Similarly, the TF-IDF weight of word b, the TF-IDF weight of word c, and so on can be calculated.
[0094] The higher the TF-IDF weight of a word, the greater the semantic contribution of the word to the candidate text.
[0095] In operation S353, the features of the candidate text are determined according to the weights of the multiple word segments in the candidate text.
[0096] For example, the weight of word a is 0.70, the weight of word b is 0.25, the weight of word c is 0.15, and the weight of word d is 0.50. The feature vector of the candidate text can be determined to be [0.70, 0.25, 0.15, 0.50, ...].
[0097] In operation S360, the similarity between the candidate text and the keyword group is determined based on the characteristics of the candidate text and the characteristics of the keyword group.
[0098] For each candidate text, the cosine similarity between the feature vector of the candidate text and the feature vector of the keyword group may be calculated.
[0099] Since the TF-IDF weight in the feature vector of the keyword group and the TF-IDF weight in the feature vector of the candidate text are related to the semantics of the candidate text, the cosine similarity between the keyword group and the candidate text can be used as the semantic similarity.
[0100] In operation S370, the similarity and the confidence are weighted to obtain a comprehensive evaluation value, and at least one candidate text is determined from the plurality of candidate texts as a recommended text associated with the target category according to the comprehensive evaluation value.
[0101] In operation S380, the recommended text is output and displayed.
[0102] For example, the similarity and confidence can be weighted according to the following formula (2) to obtain a comprehensive evaluation value.
[0103] Comprehensive evaluation value = A*(similarity)+B*(confidence) (2)
[0104] Wherein, A and B are both numbers greater than 0 and less than 1, and A+B=1, for example, A=0.5, B=0.5.
[0105] Based on the comprehensive evaluation value, multiple candidate texts can be precisely sorted. For example, the candidate texts can be sorted in descending order according to the comprehensive evaluation value.
[0106] The first M (M is an integer greater than 1, for example, M=20) candidate texts may be selected from the refined sorting results as recommended texts, and the recommended texts may be used for push and display.
[0107] This embodiment precisely sorts the candidate texts according to the confidence of the candidate texts and the similarity between the candidate texts and the category keyword groups. The sorting result is more precise and reliable, which can improve the text recommendation effect.
[0108] Figure 4 is a system diagram of a text push method according to an embodiment of the present disclosure.
[0109] like Figure 4 As shown, this embodiment includes three stages.
[0110] The first stage is to train the natural language processing model based on deep learning, including data labeling module and model training module.
[0111] The main purpose of the data annotation module is to provide learnable samples for model training. The data source can be historical news data. When selecting the data to be annotated, the uniform sampling of historical data of each category should be achieved as much as possible to avoid the problem of poor model building effect caused by data distribution deviation. After selecting the samples to be annotated, you can perform manual annotation or use the model to annotate. By annotating the historical news data with the corresponding category label, you can obtain the annotated data.
[0112] The natural language processing model can adopt Bert, a mainstream text classification model in deep learning. Bert is a pre-trained model that can be applied to downstream tasks in various text fields such as text classification, entity extraction, and reading comprehension. This embodiment can apply Bert to text classification scenarios.
[0113] In the model training module, the Bert text classification model can receive news text and corresponding category labels as input. First, the Bert text classification model will encode the received text and convert it into a vector format. Then, through the internal self-attention mechanism and the calculation of the fully connected layer network, the final category output probability is obtained. The output probability and the real category label of the data are calculated for loss, and then the gradient is updated through back propagation. This is repeated to obtain the final trained text classification model.
[0114] The second stage is to use the trained text classification model to classify the news data and store the news data after the classification is determined. The second stage includes the model prediction module and the database.
[0115] The model prediction module is mainly based on the text classification model obtained by the first stage of training, and classifies each news data to be predicted. The news data to be predicted can be generated in real time every day, and the model prediction module can process the newly generated news data in real time. For example, for a piece of news data "XX basketball game was successfully held in XX Center", the category predicted by the text classification model can be "sports", and the probability of the predicted category is "0.95", then the category of the data is marked as "sports" and the confidence is "0.95".
[0116] You can choose the ElasticSearch database to store the original text. The ElasticSearch database can achieve a very good balance between capacity and interactivity. At the same time, the ElasticSearch database supports various operations such as full-text retrieval, inverted indexing, filtering, matching, and aggregation.
[0117] The text content, category and confidence of the news data can be stored in the ElasticSearch database according to the data type and format of Table 1 above.
[0118] The third stage is the relevance ranking of news data of the target category, which includes the recall module, the rough ranking module, the fine ranking module, and the output and display module.
[0119] The recall module filters all the data in the database according to the target category or keywords of the target category input by the user to obtain the original text set of the target category.
[0120] The rough sorting module sorts the recalled original texts according to the confidence of the original texts to obtain a rough sorting result. A plurality of (for example, the first 100) candidate texts can be selected from the rough sorting module for subsequent fine sorting.
[0121] The fine sorting module is used to calculate the similarity between the candidate text and the keyword group of the target category for the candidate text, and to perform sorting based on the confidence and similarity to obtain the fine sorting result.
[0122] A plurality of (eg, first 20) candidate texts may be selected from the refined sorting results as recommended texts, and the recommended texts may be output and displayed.
[0123] Figure 5 is a block diagram of a text push device according to an embodiment of the present disclosure.
[0124] like Figure 5 As shown, the text pushing device 500 includes a target text acquisition module 501 , a candidate text determination module 502 , a similarity determination module 503 , a recommended text determination module 504 and a pushing module 505 .
[0125] The target text acquisition module 501 is used to acquire a target text set associated with a target category, wherein the target category includes a keyword group, and the target text has a confidence level associated with the target category.
[0126] The candidate text determination module 502 is used to determine a plurality of candidate texts from the target text set according to confidence levels.
[0127] The similarity determination module 503 is used to determine the similarity between each candidate text and the keyword group.
[0128] The recommended text determination module 504 is used to determine at least one candidate text from a plurality of candidate texts as a recommended text associated with the target category according to the similarity and the confidence.
[0129] The push module 505 is used to push the recommended text.
[0130] According to an embodiment of the present disclosure, the keyword group includes a plurality of keywords, and the similarity determination module 503 includes a first feature determination unit, a second feature determination unit, and a similarity determination unit.
[0131] The first feature determination unit is used to determine the features of the keyword group according to the frequency of each of the multiple keywords in the candidate text.
[0132] The second feature determination unit is used to divide the candidate text into a plurality of segmented words, and determine the features of the candidate text according to the word frequencies of the plurality of segmented words in the candidate text.
[0133] The similarity determination unit is used to determine the similarity between the candidate text and the keyword group according to the characteristics of the candidate text and the characteristics of the keyword group.
[0134] The first feature determination unit includes a first weight determination subunit and a first feature determination subunit.
[0135] The first weight determination subunit is used to determine the weight of each keyword according to the word frequency of the keyword in the candidate texts and the number of candidate texts containing the keyword.
[0136] The first feature determination subunit is used to determine the features of the keyword group according to the respective weights of the multiple keywords.
[0137] The second feature determination unit includes a second weight determination subunit and a second feature determination subunit.
[0138] The second weight determination subunit is used to determine the weight of each segmentation word according to the word frequency of the segmentation word in the candidate text and the number of candidate texts containing the segmentation word.
[0139] The second feature determination subunit is used to determine the features of the candidate text according to the respective weights of multiple word segments in the candidate text.
[0140] The recommended text determination module 504 includes a processing unit and a recommended text determination unit.
[0141] The processing unit is used to perform weighted processing on the similarity and the confidence to obtain a comprehensive evaluation value.
[0142] The recommended text determination unit is used to determine at least one candidate text from a plurality of candidate texts as a recommended text associated with the target category according to the comprehensive evaluation value.
[0143] The text pushing device 500 further includes a category determination module and an adding module.
[0144] The category determination module is used to determine the category of the original text and determine the confidence level associated with the category.
[0145] The adding module is used to add the original text, the category and the confidence of the original text into the database.
[0146] The target text acquisition module 501 is used to use the keywords of the target category to acquire a target text set associated with the target category from a database.
[0147] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
[0148] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0149] like Figure 6As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0150] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0151] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the text push method. For example, in some embodiments, the text push method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the text push method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the text push method in any other appropriate manner (e.g., by means of firmware).
[0152] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0153] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0154] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0155] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0156] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0157] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.
[0158] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0159] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A text push method, include: Acquire a target text set associated with a target category, wherein the target category includes a keyword group, the target text in the target text set has a confidence associated with the target category, and the keyword group includes a plurality of keywords; Determining a plurality of candidate texts from the target text set according to the confidence, wherein the confidence is related to the semantics of the candidate texts; For each candidate text, determining the characteristics of the keyword group according to the word frequencies of each of the multiple keywords in the candidate text; Divide the candidate text into a plurality of segmented words, and determine the features of the candidate text according to the word frequencies of each of the plurality of segmented words in the candidate text; Determining the similarity between the candidate text and the keyword group according to the characteristics of the candidate text and the characteristics of the keyword group; Determining a comprehensive evaluation value of the candidate text according to the similarity and the confidence, wherein the comprehensive evaluation value is used to characterize the relevance between the candidate text and the target category from a semantic level; determining at least one candidate text from the plurality of candidate texts as a recommended text associated with the target category according to the comprehensive evaluation values of the candidate texts; and The recommended text is pushed.
2. The method according to claim 1, in, Determining the features of the keyword group according to the frequency of each of the multiple keywords in the candidate text includes: For each keyword, determine the weight of the keyword according to the frequency of the keyword in the candidate text and the number of candidate texts containing the keyword; and The characteristics of the keyword group are determined according to the weights of the multiple keywords.
3. The method according to claim 1, in, The step of dividing the candidate text into a plurality of segmented words and determining the features of the candidate text according to the word frequencies of the plurality of segmented words in the candidate text comprises: For each participle, determine the weight of the participle according to the word frequency of the participle in the candidate text and the number of candidate texts containing the participle; and The features of the candidate text are determined according to the weights of the multiple word segments in the candidate text.
4. The method according to any one of claims 1 to 3, in, Determining the comprehensive evaluation value of the candidate text according to the similarity and the confidence comprises: The similarity and the confidence are weighted to obtain the comprehensive evaluation value.
5. The method according to any one of claims 1 to 3, further comprising: include: Determining a category of the original text and determining a confidence level that the original text is associated with the category; as well as The original text, the category of the original text, and the confidence level are added to a database.
6. The method according to claim 5, in, The acquiring of a target text set associated with a target category comprises: Using the keywords of the target category, a target text set associated with the target category is acquired from the database.
7. A text push device, include: A target text acquisition module, used to acquire a target text set associated with a target category, wherein the target category includes a keyword group, texts in the target text set have confidence associated with the target category, and the keyword group includes a plurality of keywords; A candidate text determination module, configured to determine a plurality of candidate texts from the target text set according to the confidence level, wherein the confidence level is related to the semantics of the candidate text; A first feature determination unit, configured to determine a feature of the keyword group according to a frequency of each of the plurality of keywords in the candidate text; A second feature determination unit is used to divide the candidate text into a plurality of segmented words, and determine the features of the candidate text according to the word frequencies of the plurality of segmented words in the candidate text; and A similarity determination unit, configured to determine the similarity between the candidate text and the keyword group according to the characteristics of the candidate text and the characteristics of the keyword group; a recommended text determination module, configured to determine a comprehensive evaluation value of the candidate text according to the similarity and the confidence; and to determine at least one candidate text from the plurality of candidate texts as a recommended text associated with the target category according to the comprehensive evaluation value of the candidate text, wherein the comprehensive evaluation value is used to characterize the relevance between the candidate text and the target category from a semantic level; and A push module is used to push the recommended text.
8. The device according to claim 7, in, The first feature determination unit includes: A first weight determination subunit is used to determine the weight of each keyword according to the word frequency of the keyword in the candidate text and the number of candidate texts containing the keyword; and The first feature determination subunit is used to determine the features of the keyword group according to the respective weights of the multiple keywords.
9. The device according to claim 7, in, The second feature determination unit includes: A second weight determination subunit is used to determine the weight of each segmentation word according to the word frequency of the segmentation word in the candidate text and the number of candidate texts containing the segmentation word; and The second feature determination subunit is used to determine the feature of the candidate text according to the weights of each of the multiple word segments in the candidate text.
10. The device according to any one of claims 7 to 9, in, The recommended text determination module comprises: a processing unit, configured to perform weighted processing on the similarity and the confidence to obtain the comprehensive evaluation value; and The recommended text determining unit is used to determine at least one candidate text from the multiple candidate texts as a recommended text associated with the target category according to the comprehensive evaluation value.
11. The device according to any one of claims 7 to 9, further comprising: include: A category determination module, used to determine the category of the original text and determine the confidence level associated with the category; as well as An adding module is used to add the original text, the category of the original text and the confidence into a database.
12. The device according to claim 11, in, The target text acquisition module is used to use the keywords of the target category to acquire a target text set associated with the target category from the database.
13. An electronic device, include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.
14. A non-transitory computer-readable storage medium storing computer instructions, in, The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 6.
15. A computer program product, comprising a computer program, wherein the computer program is stored on at least one of a readable storage medium and an electronic device, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.
Citation Information
Patent Citations
Information pushing method and apparatus
CN108241667A
Information pushing method and device
CN112380331A
Commodity title keyword extraction method and device, equipment, medium and product
CN114818674A
Machine learning for categorizing text
US20230026656A1
