Method, device, equipment and storage medium for determining article concept tags

Abstract

This application discloses a method, apparatus, device, and storage medium for determining article concept tags, belonging to the field of Internet technology. The method comprises: determining at least one keyword corresponding to a target article; forming input data for a pre-trained language model based on the at least one keyword, a placeholder, and the target article; inputting the input data into the pre-trained language model, and having the pre-trained language model output a predicted word corresponding to the placeholder; and forming a concept tag for the target article with the predicted word and the at least one keyword. Using this application can improve the flexibility of obtaining concept tags corresponding to articles.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a method, apparatus, device, and storage medium for determining article concept tags. Background Art

[0002] A concept tag represents key information about an article. It's typically a word or phrase. For example, an article about actor X's movies might have the concept tag "actor X's movies." Concept tags can be used to categorize articles or recommend articles to users based on their concept tags and user profiles.

[0003] In related technologies, concept tags can be derived using existing classification models. Specifically, technicians can pre-set a certain number of concept tags, treating each concept tag as a category, and then train the classification model based on sample articles from different categories. To determine the concept tag corresponding to an article, the article's content is input into the trained classification model. The classification model then determines the confidence level for each category (i.e., concept tag) and selects the concept tag with the highest confidence level as the article's concept tag.

[0004] In the process of implementing this application, the inventors discovered that the related art has at least the following problems:

[0005] In related technologies, the concept labels of articles obtained through classification models are all pre-set by technicians. If for an article, there is no concept label that matches the article among the concept labels pre-set by technicians, the concept label obtained through the classification model is not accurate. It can be seen that the current method of obtaining the corresponding concept label of the article is not flexible. Summary of the Invention

[0006] The present invention provides a method, apparatus, device, and storage medium for determining article concept tags, which can improve the flexibility of obtaining corresponding article concept tags. The technical solution is as follows:

[0007] In a first aspect, a method for determining an article concept tag is provided, the method comprising:

[0008] At least one keyword corresponding to the determined target article;

[0009] Based on the at least one keyword, the placeholder, and the target article, forming input data for a pre-trained language model;

[0010] Inputting the input data into the pre-trained language model, and having the pre-trained language model output a predicted word corresponding to the placeholder;

[0011] The predicted word and the at least one keyword are combined into a concept tag for the target article.

[0012] Optionally, the input data for the pre-trained language model based on the at least one keyword, the placeholder, and the target article includes:

[0013] Add a preset number of placeholders before and after each keyword;

[0014] The keywords with the placeholder added are concatenated with the title and body of the target article to obtain input data for the pre-trained language model.

[0015] Optionally, before inputting the input data into the pre-trained language model, the method further includes:

[0016] Obtaining sample articles and sample concept tags corresponding to the sample articles;

[0017] Replacing other words in the sample concept labels except the keywords in the sample article with the placeholders to obtain training concept labels for training the pre-trained language model;

[0018] Inputting the training concept label and the sample article into the pre-trained language model to be trained, and having the pre-trained language model to be trained output a predicted word corresponding to the placeholder in the training concept label;

[0019] Based on the predicted word corresponding to the placeholder and other words replaced by the placeholder, a training loss value is determined, and the pre-trained language model to be trained is trained based on the training loss value to obtain a trained pre-trained language model.

[0020] Optionally, obtaining sample articles and sample concept tags corresponding to the sample articles includes:

[0021] Obtain the user's search terms and the articles clicked in the corresponding search results after the user searches for the search terms;

[0022] The article clicked by the user is determined as a sample article, and the search term is determined as a sample concept tag corresponding to the sample article.

[0023] In a second aspect, a device for determining an article concept tag is provided, the device comprising:

[0024] A determination module, configured to determine at least one keyword corresponding to a target article;

[0025] a composition module, configured to compose input data of a pre-trained language model based on the at least one keyword, the placeholder, and the target article;

[0026] a processing module, configured to input the input data into the pre-trained language model, and have the pre-trained language model output a predicted word corresponding to the placeholder;

[0027] The composition module is configured to combine the predicted word and the at least one keyword to form a concept tag for the target article.

[0028] Optionally, the component module is used to:

[0029] Add a preset number of placeholders before and after each keyword;

[0030] The keywords with the placeholders added are concatenated with the title and body of the target article to obtain input data for the pre-trained language model.

[0031] Optionally, the device further includes a training module for:

[0032] Obtaining sample articles and sample concept tags corresponding to the sample articles;

[0033] Replacing other words in the sample concept labels except the keywords in the sample article with the placeholders to obtain training concept labels for training the pre-trained language model;

[0034] Inputting the training concept label and the sample article into the pre-trained language model to be trained, and having the pre-trained language model to be trained output a predicted word corresponding to the placeholder in the training concept label;

[0035] Based on the predicted word corresponding to the placeholder and other words replaced by the placeholder, a training loss value is determined, and the pre-trained language model to be trained is trained based on the training loss value to obtain a trained pre-trained language model.

[0036] Optionally, the training module is used to:

[0037] Obtain the user's search terms and the articles clicked in the corresponding search results after the user searches for the search terms;

[0038] The article clicked by the user is determined as a sample article, and the search term is determined as a sample concept tag corresponding to the sample article.

[0039] In a third aspect, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the operations performed by the method for determining article concept tags as described in the first aspect.

[0040] In a fourth aspect, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement the operations performed by the method for determining article concept tags as described in any one of claims 1 to 4.

[0041] The beneficial effects of the technical solution provided by the embodiments of the present application are:

[0042] In the embodiment of the present application, by determining the keywords of the target article, the keywords, placeholders, and the target article are combined into input data and fed into a pre-trained language model. This allows the pre-trained language model to use its understanding of the article content to generate predicted words corresponding to the placeholders, and the predicted words and keywords are combined to form a concept label for the target article. This allows for greater flexibility in determining the concept label for the target article, rather than requiring the technician to select a concept label from among multiple pre-set concept labels. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0044] Figure 1 This is a flow chart of a method for determining article concept tags provided by an embodiment of the present application;

[0045] Figure 2 This is a schematic diagram of a method for determining article concept tags provided by an embodiment of the present application;

[0046] Figure 3 This is a schematic diagram of a method for determining article concept tags provided by an embodiment of the present application;

[0047] Figure 4 This is a flow chart of a method for determining article concept tags provided by an embodiment of the present application;

[0048] Figure 5 This is a schematic diagram of the structure of a device for determining article concept tags provided by an embodiment of the present application;

[0049] Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0051] The present application provides a method for determining article concept tags, which relates to the technical field of artificial intelligence.

[0052] Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or digital computer-controlled machine simulations to extend and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results. In other words, AI is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI is also the study of the design principles and implementation methods of various intelligent machines, giving them the capabilities of perception, reasoning, and decision-making. For example, in the embodiments provided in this application, the concept tags of an article can be obtained based on AI technology. In fact, it can be considered that the content of the article is understood through AI technology, and then based on the understanding of the article content by AI technology, concept tags that can summarize the article content are generated.

[0053] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0054] Computer vision (CV) technology is the study of how machines can "see." Specifically, it refers to the use of cameras and computers to replace the human eye in identifying, tracking, and measuring objects. Further image processing is performed to transform the computer-generated images into images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems that can extract information from images or multidimensional data. Computer vision technologies generally include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / action recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0055] Key technologies in speech technology include automatic speech recognition (ASR), text-to-speech (TTS), and voiceprint recognition. Enabling computers to hear, see, speak, and feel is the future direction of human-computer interaction, with speech becoming one of the most promising methods of human-computer interaction.

[0056] Natural language processing (NLP) is a key area of ​​research in computer science and artificial intelligence. It studies the theories and methods that enable effective communication between humans and computers using natural language. Natural language processing integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language we use in everyday life—and is closely linked to the study of linguistics. Natural language processing technologies typically include text processing, semantic understanding, machine translation, robotic question answering, and knowledge graphs.

[0057] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning through demonstration.

[0058] Autonomous driving technology usually includes high-precision maps, environmental perception, behavioral decision-making, path planning, motion control and other technologies. Autonomous driving technology has broad application prospects.

[0059] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, autonomous driving, drones, robots, smart medical care, smart customer service, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0060] The solutions provided in the embodiments of this application may involve technologies such as speech, natural language processing, and machine learning in artificial intelligence software technology, which can be specifically illustrated through the embodiments provided in this application.

[0061] The method for determining an article concept tag provided in this application can be implemented by a computer device.

[0062] The computer device may be a terminal or a server, etc. When the computer device is a terminal, the terminal may be, but is not limited to, a mobile phone, a tablet computer, a smart wearable device, a desktop computer, a laptop computer, etc. When the computer device is a server, the server may be a single server or a server group. If it is a single server, the server may be responsible for all processing in the following scheme. If it is a server group, different servers in the server group may be responsible for different processing in the following scheme. The specific processing distribution can be arbitrarily set by technical personnel according to actual needs and will not be detailed here.

[0063] The computer device may include a memory and a processor. The memory may store data and instructions corresponding to the method for determining article concept tags provided herein. The data may be different types of articles, and the instructions may be program code corresponding to the method for determining article concept tags. The processor may process the data stored in the memory according to the instructions stored in the memory to implement the method for determining article concept tags provided herein.

[0064] To facilitate understanding of the embodiments of the present application, some terms and concepts involved in the present application are first explained:

[0065] A concept tag is a label used to represent key information in an article. It can generally be a word or phrase. For example, an article about actor XXXX's movies might have the concept tag "actor XXXX movies." Concept tags can be used for article classification and recommendation.

[0066] Entity tags are key nouns that appear in an article, such as names of people, places, and organizations that appear in the article.

[0067] A pretrained language model is a neural network language model that is not designed for a specific function and can be used for various downstream NLP tasks, such as language question answering and translation. Examples of pretrained language models include BERT (Bidirectional Encoder Representation from Transformers), RoBERTa (A Robustly Optimized BERT Pretraining Approach), and BERT-WWW (BERT-Whole Word Masking).

[0068] MLM (Mask Language Model) is a commonly used pre-trained language model training method. It can mask some words in a text, that is, replace some words in a text with mask placeholders. The masked text is input into the pre-trained language model. The pre-trained language model can predict the original word corresponding to the mask placeholder based on the context of the mask placeholder in the text. The loss value of the pre-trained language model training can then be determined based on the predicted word output by the pre-trained language model and the original word corresponding to the mask placeholder. The pre-trained language model is then trained based on the loss value. For a completed pre-trained language model, it can accurately predict the original word corresponding to the mask placeholder, or the output predicted word is close to the meaning of the original word.

[0069] The method for determining the concept label of an article provided in the embodiment of the present application can obtain the concept label of an article based on a pre-trained language model, thereby improving the flexibility of obtaining the concept label of the article.

[0070] Figure 1 This is a flow chart of a method for determining article concept tags provided by an embodiment of the present application. Figure 1 , the embodiment includes:

[0071] Step 101: Determine at least one keyword corresponding to a target article.

[0072] The target article can be any article for which a concept tag is to be generated, and the keywords corresponding to the target article can be the entity tags of the target article, that is, they can be key nouns appearing in the target article, such as names of people, places, and institutions.

[0073] This application also provides a method for obtaining keywords for a specific target article: a technician can pre-construct a keyword library that can include a large number of keywords. A classification model can be trained based on the keyword library and a large number of articles. By inputting an article into the trained classification model, the classification model can obtain one or more keywords from the keyword library that have the highest confidence level corresponding to the article. This classification model can be similar to the classification model used in the prior art to obtain concept labels, and its specific implementation details are not further described here.

[0074] After obtaining a trained classification model, the target article can be input into the trained classification model, and the classification model will obtain keywords corresponding to the target article. The obtained keywords can be one or multiple, and the specific number can be pre-set by the technician and is not limited here. Alternatively, the technician can set a confidence threshold and determine keywords obtained by the classification model that exceed the confidence threshold as keywords corresponding to the target article.

[0075] Step 102: Based on at least one keyword, a placeholder, and a target article, form input data for a pre-trained language model.

[0076] The placeholder can be a mask placeholder. After obtaining at least one keyword corresponding to the target article, the at least one keyword, the placeholder, and the target article can be combined into the input data for the pre-trained language model. The corresponding processing can be as follows: a preset number of placeholders are added before and after each keyword; the keywords with the placeholders added are concatenated with the title and body of the target article to obtain the input data for the pre-trained language model.

[0077] In implementation, a preset number of placeholders can be added before and after the keyword, and the preset number can be set by technical personnel and is not limited here. For example, the keyword is "Jiangsu" and the preset number is 2. The keyword after adding the placeholder is "[Mask] [Mask] Jiangsu [Mask] [Mask]". After adding the placeholder to the keyword, the keyword with the placeholder added can be spliced ​​with the title and text of the target article to obtain the input data of the pre-trained language model. Among them, the keyword with the placeholder added and the title and text of the target article can be spliced ​​into the input data of the following structure: "[CLS][Mask][Mask]Keyword 1[Mask][Mask]... [Mask][Mask]Keyword n[Mask][Mask] [SEP]Title content body content". Among them, [CLS] and [SEP] are used to distinguish the keyword part from the title and text of the article. For example, for an article with the keyword "Jiangsu", the corresponding input data can be, "[CLS][Mask][Mask]Jiangsu[Mask][Mask] [SEP]Jiangsu, a "century-old school", is expected to enter... some 211. The strength of Jiangsu Province... is the most balanced in the country", among which "Jiangsu, a "century-old school", is expected to enter... some 211" is the title of the article, and "Jiangsu Province's strength... is the most balanced in the country" is the main content of the article.

[0078] Step 103: Input the input data into the pre-trained language model, and the pre-trained language model outputs the predicted word corresponding to the placeholder.

[0079] like Figure 2 As shown, Figure 2 This diagram illustrates how a pre-trained language model generates predicted words for each placeholder in the output data. After obtaining the input data for the target article, it can be fed into the trained pre-trained language model. The pre-trained language model then outputs predicted words for each placeholder based on its understanding of the target article's title, content, and keywords.

[0080] Among them, in this application, it is not necessary to mask the target article or keywords, but to add additional Mask placeholders before and after the keywords of the target article. The pre-trained language model "thinks" that there was originally a word at the position corresponding to the Mask placeholder, and generates predicted words corresponding to the Mask placeholder through the pre-trained language model's understanding of the keywords, article titles, and text content. Since the Mask placeholder is before and after the keywords, the generated predicted words and the phrases composed of keywords are logically coherent and semantically consistent with the content of the target article. The predicted words corresponding to the Mask placeholders obtained by the pre-trained language model in this application can be considered as modifiers used to modify keywords, such as nouns, adjectives, etc.

[0081] Step 104: The predicted words and keywords are combined into a concept tag for the target article.

[0082] In implementation, after obtaining the predicted words corresponding to each keyword, the predicted words corresponding to each keyword pair can be combined into the concept label of the target article. Figure 2 As shown, the pre-trained language model can be used for each keyword K1-K n The added Mask placeholder generates the corresponding W1, W2-W 2n-1 、W 2n , then for each keyword, the corresponding generated predicted words can be used to form the concept label of the target article. Figure 2 W1K1W2, W 2n-1 K n W 2n In this application, if there are multiple keywords corresponding to the target article, there may also be multiple concept tags generated.

[0083] As a supplementary explanation, in this application, the Mask placeholder does not replace the keywords in the input data. The pre-trained language model does not need to generate corresponding prediction words for each Mask placeholder. The pre-trained language model can generate modifiers for modifying keywords based on the understanding of keywords, article titles, and text content. The modifiers can be generated for the Mask placeholder in front of the keyword, or for the Mask placeholder after the keyword, or for the Mask placeholders before and after the keyword. For Mask placeholders that do not need to generate corresponding prediction words, the pre-trained language model can output a specific symbol, which can be pre-set by technicians and applied in the training process of the pre-trained language model, such as "T". For example, the keyword with the Mask placeholder added in the input data is "[Mask][Mask]Jiangsu[Mask] [Mask]", and the corresponding output of the pre-trained language model can be T, T, high, and school. The concept label generated in this way is "Jiangsu Colleges and Universities". In addition, the training process of the pre-trained language model will not be introduced in detail here. For details, please see below. Figure 4 Corresponding embodiments.

[0084] like Figure 3 As shown, in this application, the keywords of the target article are first determined. The keywords can be the entity tags of the target article, which can be used to reflect the key information of the article. After that, Mask placeholders can be set before and after the entity tag of the target article, and the text and title corresponding to the target article form the input data and input it into the pre-trained language model. Through the understanding of the target article by the pre-trained language model, modifiers for modifying the entity tags are generated, and finally the modifiers and the entity tags are combined to form the concept tag of the target article. In this way, in this application, the key information of the article can be obtained by determining the entity tag of the article, and then the modifiers obtained by the pre-trained language model can be used to further focus on the entity tag. For example, for an article reviewing the movie "××", the entity tag can be actor ×××, but actor ××× is not enough to fully summarize the article, and the generated modifier can be "movie". The concept tag obtained in this way is "actor ××× movie", which can summarize the content of the article and can more clearly express the content of the article. It can be seen that the use of this application can use the pre-trained language model to understand the article to obtain concept labels, which can break through the limitation in related technologies that only the corresponding concept labels of the article can be selected from the concept labels set by technicians, and can improve the accuracy and flexibility of the concept labels obtained for the article.

[0085] Moreover, in the present application, the corresponding concept labels are not generated directly according to the content of the target article, but the keywords of the target article are first determined, and then the modifiers for limiting the keywords are generated according to the target article. This has higher controllability than the solution of directly generating the concept labels of the article. This is because the solution of directly generating the concept labels of the article is an end-to-end generation solution, and the final generation result is uncontrollable. There may be a problem that the final generated concept labels have a low correlation with the corresponding article content, and the solution of directly generating the concept labels of the article cannot determine the content, structure, etc. of the final generated concept labels. In the present application, by first determining the keywords of the target article, the key information of the target article can be determined first, and then the modifiers for modifying the keywords are obtained based on the understanding of the target article using the pre-trained language model, thereby further limiting the keywords and clarifying the content of the target article. The concept labels obtained in this way are a structure of keywords plus modifiers, and since the obtained concept labels include the keywords of the article, the obtained concept labels will not be irrelevant to the content of the article. It can be seen that the concept labels generated by the present application have higher controllability in structure and content compared to the solution of directly generating the concept labels of the article.

[0086] In the embodiment of the present application, by determining the keywords of the target article, the keywords, placeholders, and the target article are combined into input data and fed into a pre-trained language model. This allows the pre-trained language model to use its understanding of the article content to generate predicted words corresponding to the placeholders, and the predicted words and keywords are combined to form a concept label for the target article. This allows for greater flexibility in determining the concept label for the target article, rather than requiring the technician to select a concept label from among multiple pre-set concept labels.

[0087] Figure 4 This is a flow chart of the method for training a pre-trained language model provided in an embodiment of the present application. Figure 4 , the embodiment includes:

[0088] Step 401: Obtain sample articles and sample concept tags corresponding to the sample articles.

[0089] During implementation, technicians can obtain a large number of articles of different types as sample articles, and can set corresponding sample concept tags according to the content in the sample articles.

[0090] This application also provides a method for obtaining sample articles and sample concept tags corresponding to the sample articles. The corresponding processing is as follows: obtaining the user's search terms and the articles clicked by the user in the corresponding search results after searching the search terms; determining the article clicked by the user as a sample article, and determining the search terms as the sample concept tags corresponding to the sample article.

[0091] In practice, we can obtain records of keywords (hereinafter referred to as search terms) that users have searched for in applications such as browsers, as well as the articles that users clicked on in the corresponding search results, to determine sample articles and sample concept tags. This is because the user's search terms and the articles they clicked on are generally highly correlated, and the user's search terms generally include keywords from the articles they clicked on in the search results. Therefore, the user's search terms can be used as sample concept tags, and the articles that the user clicked on can be determined as sample articles.

[0092] Technicians can obtain the user's search terms from the backend management system corresponding to an application such as a browser. After obtaining the user's search terms, they can first be filtered. For example, a search term structure can be pre-set, such as "adjective + noun," "noun + noun," etc. Based on this pre-set search term structure, the search terms can be filtered, discarding search terms that do not meet the pre-set search term structure. For example, if the search term structure is "noun + noun," the search term "Shenzhen weather" can be considered a search term that meets the search term structure, while the search term "What are some good restaurants in Shenzhen?" can be considered a search term that does not meet the search term structure. After filtering the search terms, the search results corresponding to each search term can be determined. If the user clicks on an article-type search result, the corresponding article can be identified as a sample article, and the corresponding search term can be identified as a sample concept tag corresponding to the sample article. If the user clicks on a search result that is not an article-type search result, such as a video or music search result, no processing is performed.

[0093] In this application, by utilizing the high correlation between the user's search terms and the search results clicked by the user, when the search result clicked by the user is an article, the user's search terms can be regarded as the concept tags of the article. In this way, a large number of sample articles and corresponding concept tags can be obtained in a short time, which can improve the efficiency of obtaining sample data for training the pre-trained language model.

[0094] Step 402: Replace the words in the sample concept labels except for the keywords in the sample article with placeholders to obtain training concept labels for training the pre-trained language model.

[0095] The keywords in the sample article can be obtained by the method of determining the keywords of the target sample article in step 102 above, which will not be repeated here. After determining the keywords of the target sample article, the words other than the keywords in the corresponding sample concept label can be replaced with mask placeholders to obtain the training concept label for training the pre-trained language model.

[0096] In addition, since Mask placeholders are added before and after each keyword in the corresponding input data during the application of the pre-trained language model, after replacing the words other than the keywords in the corresponding sample concept labels with Mask placeholders, the number of Mask placeholders corresponding to the before and after positions of each keyword and the preset number can be determined. If the number of Mask placeholders corresponding to any keyword is less than the preset number, the Mask placeholders can be supplemented to increase the number of Mask placeholders corresponding to the keyword to the preset number, where the preset number here can be consistent with the preset number of placeholders added before and after the keyword in the above step 102.

[0097] Step 403: Input the training concept labels and the sample articles into the pre-trained language model to be trained, and the pre-trained language model to be trained outputs the predicted words corresponding to the placeholders in the training concept labels.

[0098] After obtaining the training concept labels, the training concept labels and the title and body of the sample article can be combined to form training data. For example, the training data may consist of the following structure: "[CLS] [Mask][Mask]Keyword 1[Mask][Mask]…[Mask][Mask]Keyword n[Mask][Mask] [SEP]Title Content Body Content." Among them, "[Mask][Mask]Keyword 1[Mask][Mask]…[Mask][Mask]Keyword n[Mask][Mask]" are the training concept labels. After obtaining the training data, it can be input into the pre-trained language model to be trained. The pre-trained language model will then output the predicted word corresponding to each mask placeholder in the training data.

[0099] Step 404: Determine a training loss value based on the predicted word corresponding to the placeholder and other words replaced by the placeholder, and train the pre-trained language model to be trained based on the training loss value to obtain a trained pre-trained language model.

[0100] After obtaining the predicted words corresponding to each Mask placeholder output by the pre-trained language model to be trained, the corresponding training loss value is determined according to the replaced words corresponding to each Mask placeholder and the corresponding predicted words, according to the pre-set loss function. For example, the corresponding training loss value can be determined by the word feature vectors corresponding to the replaced words and the corresponding predicted words. Then, the pre-trained language model to be trained is trained according to the training loss value, and the parameters in the pre-trained language model are adjusted. After the pre-trained language model is trained with a large amount of training data, and the accuracy of the generated predicted words reaches the preset accuracy threshold, the training of the pre-trained language model can be terminated to obtain a pre-trained language model. Among them, the training method of the pre-trained language model belongs to the prior art and will not be introduced in detail here.

[0101] In addition, there may be Mask placeholders in the training data that are supplemented in step 402. For the supplemented Mask placeholders, technicians can set corresponding specific symbols to represent the replaced words. For example, it can be "T", and the corresponding word feature vector is set. It is not necessary to generate corresponding predicted words for the supplemented Mask placeholders, so the pre-trained language model is trained through the specific symbols that can be set, so that the pre-trained language model outputs the corresponding specific symbols for the supplemented Mask placeholders. In this way, in the process of using the pre-trained language model, when the pre-trained language model outputs a specific symbol, it can be considered that the corresponding Mask placeholder does not need to generate a predicted word.

[0102] In the embodiment of the present application, by determining the keywords of the target article, the keywords, placeholders, and the target article are combined into input data and fed into a pre-trained language model. This allows the pre-trained language model to use its understanding of the article content to generate predicted words corresponding to the placeholders, and the predicted words and keywords are combined to form a concept label for the target article. This allows for greater flexibility in determining the concept label for the target article, rather than requiring the technician to select a concept label from among multiple pre-set concept labels.

[0103] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present disclosure, and will not be described in detail here.

[0104] Figure 5 An embodiment of the present application provides a device for determining a concept tag of an article. The device may be the computer device in the above embodiment, and the device includes:

[0105] A determination module 510 is configured to determine at least one keyword corresponding to a target article;

[0106] A composition module 520 is configured to compose input data of a pre-trained language model based on the at least one keyword, the placeholder, and the target article;

[0107] A processing module 530 is configured to input the input data into the pre-trained language model, and have the pre-trained language model output a predicted word corresponding to the placeholder;

[0108] The composition module 520 is configured to combine the predicted word and the at least one keyword into a concept tag for the target article.

[0109] Optionally, the component module 520 is used to:

[0110] Add a preset number of placeholders before and after each keyword;

[0111] The keywords with the placeholders added are concatenated with the title and body of the target article to obtain input data for the pre-trained language model.

[0112] Optionally, the device further includes a training module for:

[0113] Obtaining sample articles and sample concept tags corresponding to the sample articles;

[0114] Replacing other words in the sample concept labels except the keywords in the sample article with the placeholders to obtain training concept labels for training the pre-trained language model;

[0115] Inputting the training concept label and the sample article into the pre-trained language model to be trained, and having the pre-trained language model to be trained output a predicted word corresponding to the placeholder in the training concept label;

[0116] Based on the predicted word corresponding to the placeholder and other words replaced by the placeholder, a training loss value is determined, and the pre-trained language model to be trained is trained based on the training loss value to obtain a trained pre-trained language model.

[0117] Optionally, the training module is used to:

[0118] Obtain the user's search terms and the articles clicked in the corresponding search results after the user searches for the search terms;

[0119] The article clicked by the user is determined as a sample article, and the search term is determined as a sample concept tag corresponding to the sample article.

[0120] It should be noted that the device for determining article concept tags provided in the above embodiment only uses the division of the above functional modules as an example when determining the article concept tags. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device for determining article concept tags provided in the above embodiment and the method embodiment for determining article concept tags belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0121] Figure 6 The following is a block diagram of a computer device 600 according to an exemplary embodiment of the present application. This computer device 600 can be a portable mobile terminal, such as a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. Computer device 600 may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other similar names.

[0122] Typically, the computer device 600 includes a processor 601 and a memory 602 .

[0123] Processor 601 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 601 may be implemented in hardware using at least one of the following: a DSP (digital signal processing), an FPGA (field-programmable gate array), or a PLA (programmable logic array). Processor 601 may also include a main processor and a coprocessor. The main processor is used to process data in the awake state, also known as a CPU (central processing unit); the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 601 may integrate a GPU (graphics processing unit), which is responsible for rendering and drawing content required to be displayed on the display screen. In some embodiments, processor 601 may also include an AI (artificial intelligence) processor, which is used to handle computational operations related to machine learning.

[0124] The memory 602 may include one or more computer-readable storage media, which may be non-transitory. The memory 602 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 602 is used to store at least one instruction, which is used to be executed by the processor 601 to implement the method for determining article concept tags provided in the method embodiment of the present application.

[0125] In some embodiments, computer device 600 may optionally include a peripheral device interface 603 and at least one peripheral device. Processor 601, memory 602, and peripheral device interface 603 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 603 via a bus, signal lines, or circuit boards. Specifically, the peripheral device may include at least one of a radio frequency circuit 604, a display screen 605, a camera assembly 606, an audio circuit 607, a positioning assembly 608, and a power supply 609.

[0126] The peripheral device interface 603 can be used to connect at least one I / O (input / output)-related peripheral device to the processor 601 and the memory 602. In some embodiments, the processor 601, the memory 602, and the peripheral device interface 603 are integrated on the same chip or circuit board. In other embodiments, any one or two of the processor 601, the memory 602, and the peripheral device interface 603 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0127] The RF circuit 604 is used to receive and transmit RF (radio frequency) signals, also known as electromagnetic signals. The RF circuit 604 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 604 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the RF circuit 604 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The RF circuit 604 can communicate with other terminals via at least one wireless communication protocol. Such wireless communication protocols include, but are not limited to, the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (wireless fidelity) networks. In some embodiments, the RF circuit 604 may also include circuitry related to near field communication (NFC), although this application does not limit this.

[0128] Display screen 605 is used to display a user interface (UI). This UI may include graphics, text, icons, videos, or any combination thereof. If display screen 605 is a touchscreen display, it is also capable of detecting touch signals on or above the surface of display screen 605. These touch signals can be input as control signals to processor 601 for processing. Display screen 605 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be a single display screen 605, located on the front panel of computer device 600. In other embodiments, there may be at least two display screens 605, located on different surfaces of computer device 600 or in a foldable design. In still other embodiments, display screen 605 may be a flexible display, located on a curved or foldable surface of computer device 600. Display screen 605 can also be configured as a non-rectangular, irregular shape, also known as a special-shaped screen. Display screen 605 can be made of materials such as LCD (liquid crystal display) and OLED (organic light-emitting diode).

[0129] The camera assembly 606 is used to capture images or videos. Optionally, the camera assembly 606 includes a front camera and a rear camera. Typically, the front camera is provided on the front panel of the terminal, and the rear camera is provided on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (virtual reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 606 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0130] The audio circuit 607 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input into the processor 601 for processing, or input into the radio frequency circuit 604 to achieve voice communication. For the purpose of stereo sound collection or noise reduction, there can be multiple microphones, each located in different parts of the computer device 600. The microphone can also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert electrical signals from the processor 601 or the radio frequency circuit 604 into sound waves. The speaker can be a traditional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert electrical signals into sound waves audible to humans, but also convert electrical signals into sound waves inaudible to humans for purposes such as distance measurement. In some embodiments, the audio circuit 607 may also include a headphone jack.

[0131] Positioning component 608 is used to locate the current geographic location of computer device 600 to implement navigation or location-based services (LBS). Positioning component 608 can be based on the US GPS (Global Positioning System), China's BeiDou system, or Russia's Galileo system.

[0132] Power supply 609 is used to power various components in computer device 600. Power supply 609 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 609 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0133] In some embodiments, the computer device 600 further includes one or more sensors 610 , including but not limited to: an acceleration sensor 611 , a gyroscope sensor 612 , a pressure sensor 613 , a fingerprint sensor 614 , an optical sensor 615 , and a proximity sensor 616 .

[0134] The accelerometer 611 can detect the magnitude of acceleration along the three coordinate axes of the coordinate system established by the computer device 600. For example, the accelerometer 611 can be used to detect the components of gravity acceleration along the three coordinate axes. The processor 601 can control the display screen 605 to display the user interface in a landscape or portrait view based on the gravity acceleration signal collected by the accelerometer 611. The accelerometer 611 can also be used to collect game or user motion data.

[0135] The gyroscope sensor 612 can detect the orientation and rotation angle of the computer device 600. It can also work with the accelerometer 611 to collect the user's 3D movements of the computer device 600. Based on the data collected by the gyroscope sensor 612, the processor 601 can implement the following functions: motion sensing (for example, changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0136] The pressure sensor 613 can be installed on the side frame of the computer device 600 and / or below the display screen 605. When the pressure sensor 613 is installed on the side frame of the computer device 600, it can detect the user's grip signal of the computer device 600. The processor 601 can perform left and right hand recognition or shortcut operations based on the grip signal collected by the pressure sensor 613. When the pressure sensor 613 is installed below the display screen 605, the processor 601 controls the operational controls on the UI interface based on the user's pressure operation on the display screen 605. The operational controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0137] The fingerprint sensor 614 is used to collect the user's fingerprint. The processor 601 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 614, or the fingerprint sensor 614 identifies the user's identity based on the collected fingerprint. When the user's identity is recognized as a trusted identity, the processor 601 authorizes the user to perform relevant sensitive operations, such as unlocking the screen, viewing encrypted information, downloading software, making payments, and changing settings. The fingerprint sensor 614 can be set on the front, back, or side of the computer device 600. When a physical button or manufacturer logo is provided on the computer device 600, the fingerprint sensor 614 can be integrated with the physical button or manufacturer logo.

[0138] Optical sensor 615 is used to detect ambient light intensity. In one embodiment, processor 601 can control the display brightness of display screen 605 based on the ambient light intensity detected by optical sensor 615. Specifically, when the ambient light intensity is high, the display brightness of display screen 605 is increased; when the ambient light intensity is low, the display brightness of display screen 605 is decreased. In another embodiment, processor 601 can also dynamically adjust the shooting parameters of camera assembly 606 based on the ambient light intensity detected by optical sensor 615.

[0139] Proximity sensor 616, also known as a distance sensor, is typically located on the front panel of computer device 600. Proximity sensor 616 is used to detect the distance between the user and the front of computer device 600. In one embodiment, when proximity sensor 616 detects that the distance between the user and the front of computer device 600 is gradually decreasing, processor 601 controls display screen 605 to switch from the screen-on state to the screen-off state. When proximity sensor 616 detects that the distance between the user and the front of computer device 600 is gradually increasing, processor 601 controls display screen 605 to switch from the screen-off state to the screen-on state.

[0140] Those skilled in the art will understand that Figure 6 The structure shown in the figure does not constitute a limitation on the computer device 600, and the computer device 600 may include more or fewer components than shown in the figure, or combine some components, or adopt a different component arrangement.

[0141] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions. The instructions are executable by a processor in a terminal to implement the method for determining article concept tags in the above embodiment. The computer-readable storage medium can be non-transitory. For example, the computer-readable storage medium can be a ROM (read-only memory), RAM (random access memory), magnetic tape, floppy disk, or optical data storage device.

[0142] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0143] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A method for determining article concept tags, characterized in that: The method comprises: Determine at least one keyword corresponding to the target article; Add a preset number of placeholders before and after each keyword; Concatenating the keywords with the placeholders added, the title and the body of the target article to obtain input data for a pre-trained language model; The input data is input into the pre-trained language model, and the pre-trained language model outputs a predicted word corresponding to the placeholder; the predicted word is a modifier used to modify the keyword; and the training method of the pre-trained language model includes: Obtain a sample article and a sample concept label corresponding to the sample article; replace other words in the sample concept label except the keywords in the sample article with the placeholder to obtain a training concept label for training the pre-trained language model; input the training concept label and the sample article into the pre-trained language model to be trained, and the pre-trained language model to be trained outputs the predicted word corresponding to the placeholder in the training concept label; determine a training loss value based on the predicted word corresponding to the placeholder and other words replaced by the placeholder, and train the pre-trained language model to be trained based on the training loss value to obtain a trained pre-trained language model; The predicted word and the at least one keyword are combined into a concept tag for the target article; the concept tag is a structure of the keyword plus the modifier.

2. The method according to claim 1, characterized in that The step of obtaining sample articles and sample concept tags corresponding to the sample articles includes: Obtain the user's search terms and the articles clicked in the corresponding search results after the user searches for the search terms; The article clicked by the user is determined as a sample article, and the search term is determined as a sample concept tag corresponding to the sample article.

3. A device for determining article concept tags, characterized in that: The device comprises: A training module is configured to obtain a sample article and a sample concept tag corresponding to the sample article; replace words in the sample concept tag except for keywords in the sample article with placeholders to obtain a training concept tag for training a pre-trained language model; input the training concept tag and the sample article into the pre-trained language model to be trained, and the pre-trained language model to be trained outputs a predicted word corresponding to the placeholder in the training concept tag; determine a training loss value based on the predicted word corresponding to the placeholder and other words replaced by the placeholder, and train the pre-trained language model to be trained based on the training loss value to obtain a trained pre-trained language model; A determination module, configured to determine at least one keyword corresponding to a target article; A composition module is used to add a preset number of placeholders before and after each keyword; the keywords with the placeholders added, the title and text of the target article are spliced ​​together to obtain input data for the pre-trained language model; a processing module, configured to input the input data into the pre-trained language model, and have the pre-trained language model output the predicted word corresponding to the placeholder; the predicted word is a modifier used to modify the keyword; The composition module is used to combine the predicted word and the at least one keyword to form a concept tag of the target article; the concept tag is a structure of the keyword plus the modifier.

4. The device according to claim 3, characterized in that The training module is used to: Obtain the user's search terms and the articles clicked in the corresponding search results after the user searches for the search terms; The article clicked by the user is determined as a sample article, and the search term is determined as a sample concept tag corresponding to the sample article.

5. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the operations performed by the method for determining article concept tags according to any one of claims 1 to 2.

6. A computer-readable storage medium, characterized in that The storage medium stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the operations performed by the method for determining article concept tags according to any one of claims 1 to 2.

Images