Text recommendation method, device and electronic device based on artificial intelligence

Abstract

The present invention provides an artificial intelligence-based text recommendation method, device, electronic device, and computer-readable storage medium. The method includes: splitting a text trigger sequence into multiple subsequences based on the attributes of the text in the text trigger sequence of a user to be recommended; combining any two subsequences into a sequence sample, and predicting the text at the predicted position in the sequence sample and the association relationship between the subsequences in the sequence sample based on the sequence sample to form a prediction result; updating the vector representation of the text in the sequence sample based on the difference between the predicted result and the actual result; determining the similarity between the text in the text trigger sequence and text in a database based on the vector representation of the text in the sequence sample, and filtering the text to be recommended corresponding to the user to be recommended from multiple texts in the database based on the similarity. The present invention can broaden the scope of text recommendation and improve the accuracy of text recommendation.

Description

Technical Field

[0001] The present invention relates to artificial intelligence technology, and in particular to an artificial intelligence-based text recommendation method, device, electronic device, and computer-readable storage medium. Background Art

[0002] Artificial Intelligence (AI) is the theory, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. Natural Language Processing (NLP) is a key area within AI, focusing on theories and methods that enable effective communication between humans and computers using natural language.

[0003] Text recommendation is a key application of natural language processing, specifically news and advertising recommendations. Related solutions typically train a user's historically triggered text by inputting it into a word2vec model or other training model to generate a vector representation of the text. Based on this vector representation, text that is similar to the previously triggered text is then calculated and recommended to the user. However, this approach often results in recommended text that is extremely similar to the previously triggered text, which can easily cause visual fatigue and result in low text recommendation accuracy. Summary of the Invention

[0004] Embodiments of the present invention provide an artificial intelligence-based text recommendation method, device, electronic device, and computer-readable storage medium, which can broaden the scope of text recommendations and increase the probability of users triggering recommended texts.

[0005] The technical solution of the embodiment of the present invention is achieved as follows:

[0006] An embodiment of the present invention provides an artificial intelligence-based text recommendation method, comprising:

[0007] Splitting the text trigger sequence into multiple subsequences based on the attributes of the text in the text trigger sequence of the user to be recommended; wherein the attributes of the text included in each subsequence are the same;

[0008] Combining any two of the subsequences into a sequence sample, and predicting the text located at the prediction position in the sequence sample and the association relationship between the subsequences in the sequence sample based on the sequence sample to form a prediction result;

[0009] According to the difference between the predicted result and the actual result, updating the vector representation of the text in the sequence sample;

[0010] Determine the similarity between the text in the text trigger sequence and the text in the database based on the vector representation of the text in the sequence sample, and

[0011] According to the similarity, a text to be recommended corresponding to the user to be recommended is screened out from a plurality of texts in the database.

[0012] An embodiment of the present invention provides an artificial intelligence-based text recommendation device, comprising:

[0013] a splitting module, configured to split the text trigger sequence into a plurality of subsequences according to the attributes of the text in the text trigger sequence of the user to be recommended; wherein the attributes of the text included in each subsequence are the same;

[0014] a prediction module, configured to combine any two of the subsequences into a sequence sample, and predict, based on the sequence sample, the text at the prediction position in the sequence sample and the association relationship between the subsequences in the sequence sample, to form a prediction result;

[0015] An updating module, configured to update the vector representation of the text in the sequence sample according to the difference between the predicted result and the actual result;

[0016] A screening module is used to determine the similarity between the text in the text trigger sequence and the text in the database based on the vector representation of the text in the sequence sample, and

[0017] According to the similarity, a text to be recommended corresponding to the user to be recommended is screened out from a plurality of texts in the database.

[0018] An embodiment of the present invention provides an electronic device, including:

[0019] a memory for storing executable instructions;

[0020] The processor is configured to implement the artificial intelligence-based text recommendation method provided by an embodiment of the present invention when executing the executable instructions stored in the memory.

[0021] An embodiment of the present invention provides a computer-readable storage medium storing executable instructions for causing a processor to execute the instructions to implement an artificial intelligence-based text recommendation method provided by an embodiment of the present invention.

[0022] The embodiments of the present invention have the following beneficial effects:

[0023] The embodiment of the present invention splits a text trigger sequence into multiple subsequences according to the attributes of the text, combines any two subsequences into a sequence sample, and performs prediction based on the sequence sample. It further updates the vector representation of the text based on the difference between the predicted result and the actual result, and finally uses the updated vector representation to perform text recommendation. In this way, it is possible to foresee the development of user interests, broaden the scope of text recommendation, and improve the accuracy of text recommendation. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a schematic diagram of an optional architecture of a text recommendation system provided by an embodiment of the present invention;

[0025] Figure 2 This is a schematic diagram of an optional architecture of a server provided in an embodiment of the present invention;

[0026] Figure 3 This is a schematic diagram of an optional architecture of an artificial intelligence-based text recommendation device provided in an embodiment of the present invention;

[0027] Figure 4A This is an optional flowchart of the artificial intelligence-based text recommendation method provided by an embodiment of the present invention;

[0028] Figure 4B This is an optional flowchart of the artificial intelligence-based text recommendation method provided by an embodiment of the present invention;

[0029] Figure 4C This is an optional flowchart of the artificial intelligence-based text recommendation method provided by an embodiment of the present invention;

[0030] Figure 5 This is an optional schematic diagram of a recommendation system provided by an embodiment of the present invention;

[0031] Figure 6 is an optional schematic diagram of constructing a sequence sample provided by an embodiment of the present invention;

[0032] Figure 7 is an optional schematic diagram of a model input vector provided by an embodiment of the present invention;

[0033] Figure 8 This is an optional comparison diagram of the recommendation results provided by the embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting the present invention. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0035] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0036] In the following description, the terms "first\second" are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present invention described herein can be implemented in an order other than that illustrated or described herein.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention pertains. The terms used herein are for the purpose of describing embodiments of the present invention only and are not intended to limit the present invention.

[0038] Before further explaining the embodiments of the present invention in detail, the nouns and terms involved in the embodiments of the present invention are explained. The nouns and terms involved in the embodiments of the present invention are subject to the following interpretations.

[0039] 1) Text recommendation system: A tool that connects users with text, can help users discover interesting information in an information-overloaded environment, and can also push information to users who are interested in it.

[0040] 2) Recall: Roughly select a batch of texts for the user, which is equivalent to rough sorting.

[0041] 3) Click-Through-Rate (CTR) estimation: The CTR of a text is estimated based on a feature list and a specific model. The estimated CTR can be used for fine-tuning the text ranking.

[0042] 4) Re-ranking: Based on specific strategies or rules, the refined ranking results are re-updated to obtain the final recommendation sequence.

[0043] 5) Language representation model: used to map text into vector representation. In an embodiment of the present invention, the language representation model can be a Bidirectional Encoder Representation from Transformers (BERT) model. The BERT model is a translation model based on a neural network and an attention mechanism.

[0044] 6) Text trigger sequence: a sequence obtained by counting the texts triggered by users within a set time period in chronological order. The embodiment of the present invention does not limit the specific operation of the trigger, for example, the trigger can be a click or a favorite.

[0045] 7) Attributes: These are abstract properties of text that distinguish it from other text. For example, if the text is news, attributes can be first-level categories of news, such as entertainment and sports; or second-level categories of news, such as basketball and football under sports.

[0046] 8) Sample User: For users different from the user to be recommended, the sample user's text trigger sequences typically differ from those of the user to be recommended. For text in the database that does not appear in the sequence samples of the user to be recommended, the sample user's sequence samples containing the text can be processed accordingly to obtain an updated vector representation of the text.

[0047] Personalized text recommendation refers to recommending texts that may be of interest to users based on their interests and triggering behaviors. A user's text trigger sequence is crucial information for mining user interests, reflecting both the distribution of their interests and the implicit relationships between texts. In related art solutions, the text trigger sequence is typically intercepted using a specific window size. Multiple texts that fit the window size are used as training samples and fed into a word2vec model or other model for training, resulting in a vector representation of the text. The word2vec model is a natural language processing (NLP) tool for vectorizing text. The similarity between texts is then calculated based on the resulting vector representations, and other texts that are similar to a text that was previously triggered by the user are selected and recommended to the user. However, this approach generally results in recommended texts that are extremely similar to the text that triggered the user. Repeating this approach repeatedly results in a narrowing of the recommended texts, causing visual fatigue and negatively impacting the accuracy of text recommendations.

[0048] Embodiments of the present invention provide an artificial intelligence-based text recommendation method, device, electronic device, and computer-readable storage medium, which can broaden the scope of text recommendations and improve the accuracy of recommendations.

[0049] The following describes an exemplary application of an electronic device provided by an embodiment of the present invention. The electronic device provided by an embodiment of the present invention can be a server, such as a server deployed in the cloud, which filters out texts that may be of interest to the user to be recommended in a database based on the text trigger sequence of the user to be recommended, and performs a recommendation operation; it can also be a terminal device, such as a news recommendation device or an advertising recommendation device in a shopping mall; it can even be a handheld terminal or other device. By running the text recommendation solution provided by an embodiment of the present invention, the electronic device can broaden the scope of text recommendation and improve the accuracy of text recommendation, that is, improve the text recommendation performance of the electronic device itself, and is suitable for multiple application scenarios, such as news recommendation, advertising recommendation, product recommendation, and movie recommendation.

[0050] See also Figure 1 , Figure 1 This is an optional architectural diagram of a text recommendation system 100 provided in an embodiment of the present invention. To support a text recommendation application, a terminal device 400 (terminal device 400-1 and terminal device 400-2 are shown as examples) is connected to a server 200 via a network 300, and the server 200 is connected to a database 500. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0051] In some embodiments, the terminal device 400 may locally execute the artificial intelligence-based text recommendation method provided by the embodiment of the present invention, construct a sequence sample based on the text trigger sequence of multiple users of the terminal device 400, and update the vector representation of each text in the sequence sample, and save the vector representation of the text locally on the terminal device 400 (while storing the mapping relationship between the text identifier and the vector representation) or upload it to the database 500 for storage via the network 300. It is worth noting that the "multiple" in the embodiment of the present invention refers to at least two, and the multiple users here include the user to be recommended and the sample user. The terminal device 400 calculates the similarity between the text in the text trigger sequence of the user to be recommended and the text in the database 500 in terms of vector representation, and filters out the text to be recommended corresponding to the user to be recommended in the database 500 based on the similarity, so that in the subsequent process, direct recommendation or further screening can be performed based on the text to be recommended.

[0052] In addition, the server 200 may also execute the artificial intelligence-based text recommendation method provided in an embodiment of the present invention, specifically obtaining text trigger sequences of multiple users from the terminal device 400 and / or the database 500, constructing sequence samples, and then updating the vector representation of the text in the database 500 (the text is also located in the sequence samples). The server 200 then calculates the similarity between the text in the text trigger sequence of the user to be recommended and the text in the database 500 in terms of vector representation, and filters the text to be recommended from the database 500 based on the similarity. The server may send the text to be recommended directly to the terminal device 400 for recommendation, or may further filter the text to be recommended.

[0053] The terminal device 400 can display various results of the text recommendation process in the graphical interface 410 (graphic interface 410-1 and graphical interface 410-2 are shown as examples), such as the text to be recommended and the corresponding similarity. Figure 1 Only the text to be recommended is taken as an example.

[0054] The following describes exemplary applications of electronic devices provided by embodiments of the present invention. The electronic devices can be implemented as various types of terminal devices, such as laptop computers, tablet computers, desktop computers, set-top boxes, mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), and can also be implemented as servers.

[0055] The following description will be made by taking the electronic device as the server as an example. Figure 2 , Figure 2 The server 200 provided by the embodiment of the present invention (for example, it can be Figure 1 The server 200 shown in FIG. Figure 2 The server 200 shown includes: at least one processor 210, a memory 240 and at least one network interface 220. The various components in the server 200 are coupled together via a bus system 230. It is understood that the bus system 230 is used to achieve connection and communication between these components. In addition to including a data bus, the bus system 230 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 230 is not described in detail. Figure 2 Various buses are labeled as bus system 230 .

[0056] The processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0057] The memory 240 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, etc. The memory 240 may optionally include one or more storage devices that are physically remote from the processor 210.

[0058] The memory 240 includes volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 240 described in the embodiments of the present invention is intended to include any suitable type of memory.

[0059] In some embodiments, memory 240 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplified below.

[0060] Operating system 241, including system programs for processing various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, and driver layer, which are used to implement various basic services and process hardware-based tasks;

[0061] The network communication module 242 is used to reach other computing devices via one or more (wired or wireless) network interfaces 220. Exemplary network interfaces 220 include Bluetooth, Wireless LAN (WiFi), and Universal Serial Bus (USB).

[0062] In some embodiments, the artificial intelligence-based text recommendation device provided by the embodiments of the present invention can be implemented in software. Figure 2 The AI-based text recommendation device 243 stored in memory 240 is shown. This device can be software in the form of a program or plug-in, and includes the following software modules: a splitting module 2431, a prediction module 2432, an update module 2433, and a screening module 2434. These modules are logical and can be arbitrarily combined or further split according to the functions they implement. The functions of each module will be described below.

[0063] In other embodiments, the artificial intelligence-based text recommendation device provided by the embodiments of the present invention can be implemented in hardware. As an example, the artificial intelligence-based text recommendation device provided by the embodiments of the present invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the artificial intelligence-based text recommendation method provided by the embodiments of the present invention. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0064] The text recommendation method based on artificial intelligence provided by the embodiment of the present invention can be executed by the above-mentioned server, or by a terminal device (for example, Figure 1 The process is executed by the terminal device 400-1 and the terminal device 400-2 shown in the figure, or is executed jointly by the server and the terminal device.

[0065] The following will describe the process of implementing the artificial intelligence-based text recommendation method by embedding an artificial intelligence-based text recommendation device in the electronic device, in combination with the exemplary applications and structures of the electronic device described above.

[0066] See also Figure 3 and Figure 4A , Figure 3 2 is a schematic diagram of the architecture of the artificial intelligence-based text recommendation device 243 provided in an embodiment of the present invention, illustrating a process of implementing text recommendation through a series of modules. Figure 4A This is a flowchart of the text recommendation method based on artificial intelligence provided by an embodiment of the present invention, which will be combined with Figure 3 right Figure 4A The steps shown are explained.

[0067] In step 101, the text trigger sequence is split into multiple subsequences according to the attributes of the text in the text trigger sequence of the user to be recommended; wherein the attributes of the text included in each subsequence are the same.

[0068] For example, see Figure 3In the splitting module 2431, the text in the database that is triggered by the user to be recommended within the set time period is obtained and sorted in chronological order to obtain a text trigger sequence. The chronological order can be the order of trigger time from earliest to latest, and the set time period is set according to the actual application scenario, such as March 1st to March 7th. It is worth noting that the trigger in this embodiment of the present invention can be a click, favorite, or other operation, which can also be determined according to the actual application scenario.

[0069] After obtaining the text trigger sequence, the text trigger sequence is split into multiple subsequences according to the attributes of the text therein, wherein each subsequence includes the same attribute of the text, for example Figure 3 Subsequence 1, subsequence 2, and subsequence 3 in the text are shown in Figure 1. For example, if the text is news and the attribute is a primary classification of news, and the text trigger sequence is "News 1 -> News 2 -> News 3 -> News 4 -> News 5," and the attributes of News 1 to News 5 are, in order, entertainment, sports, entertainment, entertainment, and sports, then after splitting, one subsequence is "News 1 -> News 3 -> News 4," and the other is "News 2 -> News 5." It's worth noting that the order of the text in the subsequences matches the order of the text in the text trigger sequence.

[0070] In step 102, any two subsequences are combined into a sequence sample, and the text at the prediction position in the sequence sample and the association relationship between the subsequences in the sequence sample are predicted based on the sequence sample to form a prediction result.

[0071] For example, see Figure 3 In prediction module 2432, the obtained multiple subsequences are combined in pairs to obtain sequence samples. Here, all possible pairwise combinations can be traversed until all possible sequence samples are obtained. For each obtained sequence sample, the text at the predicted position in the sequence sample and the association between subsequences in the sequence sample are predicted based on the sequence sample to form a prediction result. The predicted position can be determined by random selection.

[0072] In step 103, the vector representation of the text in the sequence sample is updated according to the difference between the predicted result and the actual result.

[0073] Here, based on the difference between the predicted and actual results, the vector representation of the text in the sequence sample is updated so that the difference is smaller when the next prediction is made based on the updated vector representation. In this way, not only can the associations between text within an attribute be learned, but also the associations between different attributes can be learned.

[0074] In step 104, the similarity between the text in the text trigger sequence and the text in the database is determined based on the vector representation of the text in the sequence sample, and based on the similarity, the text to be recommended corresponding to the user to be recommended is screened out from the multiple texts in the database.

[0075] For example, see Figure 3 In the screening module 2434, after completing the update of the vector representation of the text in the sequence sample, the vector representation can be stored in the database and a mapping relationship can be established with the text in the database. When making text recommendations to the user to be recommended, one or more texts can be selected from the text trigger sequence, and the vector similarity between the vector representation of the selected text and the vector representation of the text in the database can be used as the similarity between the selected text and the text in the database. The vector similarity can be cosine similarity or the absolute value of the Pearson correlation coefficient, etc.

[0076] Based on the obtained similarity, the texts in the database are screened for the recommended user. For example, the texts with the largest R similarities are selected as the recommended texts, where R is an integer greater than 0. The selected recommended texts can be directly recommended to the user or further screened to improve the recommendation effect.

[0077] Embodiments of the present invention can be applied to a variety of scenarios. For example, in a news recommendation scenario, a text trigger sequence formed by a user clicking on news in a news client can be collected, and then the vector representation of each news in the database can be updated. The news that the user is more likely to be interested in is determined based on the updated vector representation, and a recommendation operation is performed; for example, in a product recommendation scenario, a record of a user clicking on a product on an e-commerce platform can be obtained, and the description text corresponding to the clicked product can be searched in the database to form a text trigger sequence. After a series of processing, the product that the user is more likely to be interested in is finally selected for recommendation, thereby increasing the probability of the user consuming on the e-commerce platform; for example, in a movie recommendation scenario, the movie name can be identified from the user's web browsing history and download history (for example, matching according to the movie name in the database), and the movie content description text corresponding to the movie name can be searched in the database to form a text trigger sequence. After a series of processing based on the text trigger sequence, the movie that the user is more likely to be interested in is finally selected for recommendation, thereby improving the user experience.

[0078] In some embodiments, the above-mentioned determination of the similarity between the text in the text trigger sequence and the text in the database based on the vector representation of the text in the sequence sample can be achieved in the following manner: when the text in the database is different from the text in the sequence sample, the vector representation of the text in the database is determined based on the sequence sample of the sample user; wherein the sequence sample of the sample user includes the text in the database; the text in the text trigger sequence that meets the trigger condition is determined as the text to be compared; when there are multiple texts to be compared, the vector representations of all the texts to be compared are summed and averaged to obtain an average vector representation; and the similarity between the average vector representation and the vector representation of the text in the database is determined.

[0079] Since only the vector representation of the text in the sequence sample of the user to be recommended is updated in step 103, and the database usually does not only include the text in the sequence sample of the user to be recommended, in an embodiment of the present invention, when the text in the database is different from the text in the sequence sample, the vector representation of the text in the database is also updated based on the sequence sample of the sample user. There may be one or more sample users, and the sequence sample of the sample user includes the text in the database. The process of updating the vector representation is similar to steps 101 to 103 and will not be repeated here. After completing the update of the vector representation of the text in the database, the text that meets the trigger condition in the text trigger sequence of the user to be recommended is determined as the text to be compared. For example, the N texts with the closest trigger time in the text trigger sequence are determined as the text to be compared, where N is an integer greater than 0.

[0080] When there is only one text to be compared, the vector similarity between the vector representation of the text to be compared and the vector representation of the text in the database is directly used to determine the similarity between the text to be compared and the text in the database. In this case, the efficiency of calculating similarity and screening the recommended texts is higher. When there are multiple texts to be compared, the vector representations of all the texts to be compared are summed and averaged to obtain an average vector representation. The similarity between the average vector representation and the vector representation of the text in the database (vector similarity) is then determined. In this case, the calculated average vector representation can more comprehensively reflect the interests of the user to be recommended.

[0081] In some embodiments, before step 104, it also includes: performing portrait analysis on the text trigger sequence to obtain user portrait attributes; screening texts in the database whose attributes meet the user portrait attributes to determine the similarity between the text in the text trigger sequence and the text screened out from the database.

[0082] Before calculating similarity, the text in the database can be preliminarily screened. Specifically, a profile analysis is performed on the text trigger sequence of the recommended user to obtain user profile attributes. The embodiment of the present invention does not limit the method of portrait analysis and processing. For example, all attributes of the text appearing in the text trigger sequence can be determined as user profile attributes. For example, attributes that appear in the text trigger sequence with a frequency higher than a frequency threshold (such as 20%) can be determined as user profile attributes.

[0083] The texts in the database are screened based on the user profile attributes. Specifically, texts in the database whose attributes match the user profile attributes are selected as the texts for which similarity is calculated in step 104. For example, if the user profile attributes include entertainment and fashion, and text A in the database has the attribute of entertainment, and text B has the attribute of sports, then text A is screened out to calculate the similarity between the text in the text trigger sequence and text A in step 104. This method performs a preliminary screening of the texts in the database, reducing the processing load for subsequent similarity calculations.

[0084] Through the embodiment of the invention Figure 4A From the above exemplary implementation, it can be seen that the embodiment of the present invention constructs sequence samples based on two subsequences of different attributes, and performs prediction tasks based on the sequence samples, thereby learning the association relationship between texts within an attribute and the association relationship between different attributes. When determining the text to be recommended based on the updated vector representation, it is possible to broaden the scope of recommended texts, bring freshness to users while satisfying their interests, and improve the accuracy of text recommendations.

[0085] In some embodiments, see Figure 4B , Figure 4B This is an optional flowchart of the artificial intelligence-based text recommendation method provided by an embodiment of the present invention. Figure 4A The illustrated step 102 can be implemented through steps 201 to 202 , which will be described in conjunction with each step.

[0086] In step 201, any two subsequences are combined into a sequence sample, and mapping processing is performed on each text in the sequence sample to obtain a vector representation of the text.

[0087] In the embodiment of the present invention, the prediction task based on the sequence sample can be performed by the language representation model. Since the language representation model can only accept vector input, each text in the sequence sample is first mapped to obtain the vector representation of the text.

[0088] In some embodiments, the above-mentioned mapping processing of each text in the sequence sample to obtain the vector representation of the text can be achieved in the following ways: vector initialization processing is performed on each text in the sequence sample to obtain the text vector representation of the text; vector initialization processing is performed on the attributes of each text in the sequence sample to obtain the attribute vector representation of the text; vector initialization processing is performed on the position of each text in the sequence sample to obtain the position vector representation of the text; the text vector representation, attribute vector representation and position vector representation of the text are fused to obtain the vector representation of the text.

[0089] For example, see Figure 3 In the prediction module 2432, for each text in the sequence sample, its vector representation can be composed of three parts. During initialization, the text itself, the attributes of the text, and the position of the text are respectively initialized to obtain a text vector representation, an attribute vector representation, and a position vector representation. Among them, the text vector representation corresponds to semantic information, the attribute vector representation corresponds to attribute information, and the position vector representation corresponds to syntactic information. Specifically, for the text vector representation and the attribute vector representation, they can be obtained by random initialization in a set vector space; for the position vector representation, it can be initialized by an open source construction formula, which uses sine and cosine functions to map the position of the text in the text trigger sequence to a position vector representation. After initialization, the text vector representation, attribute vector representation, and position vector representation of the text are fused to obtain a vector representation of the text. The vector representation of the text is used to input into the language representation model, and the fusion process is such as summation. In this way, the text is mapped to a vector representation that includes semantic information, syntactic information, and attribute information, which improves the comprehensiveness of the vector representation and the effect of subsequent prediction tasks.

[0090] In some embodiments, the above-mentioned determination of the similarity between the text in the text trigger sequence and the text in the database based on the vector representation of the text in the sequence sample can be achieved in the following manner: performing any one of the following processing to obtain the similarity: determining the similarity between the text in the text trigger sequence and the text in the database based on the text vector representation of the text in the sequence sample; determining the first similarity between the text in the text trigger sequence and the text in the database based on the attribute vector representation of the text in the sequence sample, and determining the second similarity between the text in the text trigger sequence and the text in the database based on the text vector representation of the text in the sequence sample, and taking the result of weighted averaging of the first similarity and the second similarity as the similarity between the text in the text trigger sequence and the text in the database.

[0091] When calculating the similarity between the text in the text trigger sequence and the text in the database, either of the following two methods can be used: The first method is to use the vector similarity between the text vector representation of the text in the text trigger sequence and the text vector representation of the text in the database as the similarity between the text in the text trigger sequence and the text in the database.

[0092] The second method involves determining a first similarity between the attribute vector representation of the text in the text trigger sequence and the attribute vector representation of the text in the database, and simultaneously determining a second similarity between the text vector representation of the text in the text trigger sequence and the text vector representation of the text in the database. The first and second similarities are weighted averages, representing the similarity between the text in the text trigger sequence and the text in the database. Both the first and second similarities are vector similarities. The weights corresponding to the first and second similarities can be set based on the actual application scenario. For example, if the first similarity is 80% and the weight is 20%, and the second similarity is 70% and the weight is 80%, the final similarity is 80% * 20% + 70% * 80% = 72%. This method facilitates attribute expansion. For example, if a user has triggered entertainment news, and the attribute vector representation of entertainment is similar to the attribute vector representation of fashion, the user is likely to be interested in fashion news. Therefore, fashion news can be recommended to the user during the text recommendation process, broadening the scope of recommendations.

[0093] In step 202, the vector representation of the text in the sequence sample is predicted using the language representation model to form a prediction result corresponding to the text at the prediction position in the sequence sample and the association relationship between subsequences in the sequence sample.

[0094] For example, see Figure 3 In the prediction module 2432, the vector representation of the text in the sequence sample is input into the language representation model, and two prediction tasks are performed by the language representation model. The first prediction task is to form a prediction result corresponding to the text at the prediction position in the sequence sample, such as Figure 3 The second prediction task is to form a prediction result corresponding to the association relationship between subsequences in the sequence sample, such as Figure 3 The prediction result in 2. It is worth noting that the language representation model can be a BERT model.

[0095] In some embodiments, before step 202, the method further includes: randomly determining the position of the text in the sequence sample as the predicted position until the proportion of the text corresponding to the predicted position reaches the predicted proportion; performing one of masking, replacement, and retention processing on the text corresponding to the predicted position; and adding a classification identifier before the sequence sample; wherein the classification identifier is used to enable the language representation model to learn the association relationship between subsequences in the sequence sample.

[0096] Before performing a prediction task, the sequence samples are preprocessed. Specifically, for the first prediction task, the position of the text in the sequence sample is randomly determined as the prediction position until the proportion of the text corresponding to the predicted position in the total text of the sequence sample reaches the predicted proportion, for example, 15%. Then, the text corresponding to the predicted position is subjected to one of the following processing steps: masking, replacement, or retention. Specifically, the probability of performing each processing step can be set, for example, setting the probability of performing masking, replacement, and retention to 80%, 10%, and 10%, respectively. Masking refers to replacing the text corresponding to the predicted position with a mask; replacing refers to replacing the text corresponding to the predicted position with any text in the database; and retaining refers to leaving the text corresponding to the predicted position unchanged. The purpose of this setting is to prevent the language representation model from knowing whether the text corresponding to the predicted position is correct, forcing the language representation model to rely more on contextual information to predict text and giving the model a certain degree of error correction capability.

[0097] For the second prediction task, a classification identifier is added before the sequence sample. This identifier prompts the language representation model to learn the associations between subsequences in the sequence sample. The specific form of the classification identifier varies depending on the language representation model. For example, if the language representation model is the BERT model, the classification identifier is the [CLS] identifier.

[0098] In some embodiments, the above-mentioned prediction processing of the vector representation of the text in the sequence sample through the language representation model can be achieved in the following manner: the vector representation of the text in the sequence sample is predicted through the language representation model to obtain an output vector corresponding to the prediction position and an output vector corresponding to the classification identifier; the output vector corresponding to the prediction position is classified to obtain a prediction result corresponding to the text at the prediction position in the sequence sample; the output vector corresponding to the classification identifier is classified to obtain a prediction result corresponding to the association relationship between the subsequences in the sequence sample.

[0099] The two prediction tasks described above can be performed simultaneously. After the vector representation corresponding to the preprocessed sequence sample is input into the language representation model, the language representation model undergoes forward propagation to obtain an output vector corresponding to the predicted position and an output vector corresponding to the classification identifier. The output vector corresponding to the predicted position is then classified to obtain a prediction result corresponding to the text at the predicted position in the sequence sample; the output vector corresponding to the classification identifier is also classified to obtain a prediction result corresponding to the association between subsequences in the sequence sample. The classification process can be performed by applying a classification function, such as a softmax function, to the fully connected output vector. By combining the two prediction tasks in this manner, the efficiency and accuracy of updating the vector representation are improved.

[0100] exist Figure 4B middle, Figure 4A The illustrated step 103 may be updated to step 203, in which back propagation is performed in the language representation model based on the difference between the predicted result and the actual result, and in the process of back propagation, the vector representation of the text in the sequence sample is updated.

[0101] After obtaining the predicted results, the difference between the predicted and actual results is determined. This difference is equivalent to the loss value of the language representation model. Based on this difference, backpropagation is performed in the language representation model. During the backpropagation process, the weight parameters of the language representation model and the vector representation of the text in the sequence sample are updated until the set convergence conditions, such as the set number of training rounds, are met. When the language representation model training is completed, the vector representation of the text in the sequence sample is also updated.

[0102] In some embodiments, the above-mentioned back propagation in the language representation model based on the difference between the predicted result and the actual result can be achieved in the following manner: according to the first loss function of the language representation model, a first difference between the predicted result corresponding to the text at the predicted position in the sequence sample and the actual result is determined; according to the second loss function of the language representation model, a second difference between the predicted result corresponding to the association relationship between subsequences in the sequence sample and the actual result is determined; the first difference and the second difference are summed, and back propagation is performed in the language representation model based on the result of the summation.

[0103] For example, see Figure 3In the update module 2433, according to the first loss function of the language representation model corresponding to the first prediction task, the first difference between the predicted result corresponding to the text at the prediction position in the sequence sample and the actual result is determined; according to the second loss function corresponding to the second prediction task, the second difference between the predicted result corresponding to the association relationship between subsequences in the sequence sample and the actual result is determined, wherein the embodiment of the present invention does not limit the type of the first loss function and the second loss function. Then, the first difference and the second difference are fused, and back propagation is performed in the language representation model based on the result of the fusion process. The fusion process here can be a summation process. In the above manner, the loss is calculated for different prediction results respectively, thereby improving the accuracy of the updated vector representation.

[0104] Through the embodiment of the invention Figure 4B As can be seen from the above exemplary implementation, the embodiment of the present invention jointly performs two prediction tasks and updates the vector representation according to the back-propagation mechanism, thereby improving the efficiency and accuracy of the update process.

[0105] In some embodiments, see Figure 4C , Figure 4C This is an optional flow chart of the text recommendation method based on artificial intelligence provided by an embodiment of the present invention. Figure 4A After step 104, in step 301, user characteristics of the user to be recommended may be obtained.

[0106] After determining the text to be recommended, the text to be recommended can be further screened. First, user characteristics of the user to be recommended are obtained. The specific content of the user characteristics is not limited here, and can include, for example, the gender, age, location, and occupation of the user to be recommended.

[0107] In step 302, the user features and the vector representation of the text in the text trigger sequence are combined into a trigger sample, and the recommendation model is updated according to the trigger sample.

[0108] Here, the user features are combined with the vector representations of the texts in the text trigger sequence to obtain trigger samples, wherein the user features can be combined with the vector representations of each text in the text trigger sequence separately, or some texts can be selected from the text trigger sequence first, such as the T texts with the most recent trigger time, and then the user features are combined with the vector representations of the T texts separately, where T is an integer greater than 0. Then, the weight parameters of the recommendation model are updated based on the obtained trigger samples. It is worth noting that the vector representation of the text in step 302 can include the text vector representation and the attribute vector representation, or can include only the text vector representation.

[0109] In some embodiments, the above-mentioned updating of the recommendation model based on the triggering sample can be achieved in the following manner: the triggering sample is predicted and processed by the recommendation model to obtain the predicted label to be compared; based on the difference between the label of the triggering sample and the predicted label to be compared, backpropagation is performed in the recommendation model, and the weight parameters of the recommendation model are updated during the backpropagation process; wherein, the label of the triggering sample is used to indicate that the user to be recommended has performed a triggering operation on the text corresponding to the triggering sample.

[0110] Here, in addition to trigger samples, untriggered samples can also be constructed. For example, user features and vector representations of untriggered text can be combined to form untriggered samples, where untriggered text refers to text that has been recommended to the user to be recommended but has not been triggered by the user to be recommended. For ease of explanation, taking the case where the label corresponding to the trigger sample is 1 and the label corresponding to the untriggered sample is 0, for the trigger sample, the trigger sample is used as the input of the recommendation model, and the predicted label to be compared is obtained after the recommendation model performs forward propagation. The difference between the label of the trigger sample and the predicted label to be compared is determined based on the loss function of the recommendation model. The forward propagation is equivalent to the prediction process, and the predicted label reflects the estimated trigger rate (click-through rate). The embodiment of the present invention does not limit the type of loss function here; for example, it can be a cross-entropy loss function. For untriggered samples, the process of determining the difference is similar and will not be elaborated here. Based on the obtained difference, backpropagation is performed in the recommendation model, and during the backpropagation process, the weight parameters of the recommendation model are updated until the recommendation model converges. In this way, the recommendation model is trained based on the historical triggering situation of the user to be recommended, improving the targeting of the recommendation to the user to be recommended.

[0111] In step 303, the user features and the vector representation of the text to be recommended are combined into a sample to be recommended.

[0112] For the text to be recommended determined in step 104 , the user features of the user to be recommended and the vector representation of the text to be recommended are combined into a sample to be recommended.

[0113] In step 304, prediction processing is performed on the sample to be recommended based on the updated recommendation model to obtain a predicted label.

[0114] Based on the updated recommendation model, forward propagation is performed on the recommended samples, i.e., prediction processing is performed to obtain a predicted label. This predicted label reflects the probability that the recommended user will trigger the recommended text. In this way, a predicted label corresponding to each recommended text can be obtained.

[0115] In step 305 , the texts to be recommended are filtered according to the predicted labels, and the filtered texts to be recommended are sorted to obtain a sequence of texts to be recommended.

[0116] For example, if the predicted tag is the trigger rate, the recommended texts corresponding to the S largest predicted tags can be retained, and the rest can be filtered out. Then, the retained recommended texts are sorted in descending order of predicted tags to obtain a recommended text sequence.

[0117] In step 306, a recommendation operation corresponding to the user to be recommended is performed according to the text sequence to be recommended.

[0118] Based on the obtained sequence of texts to be recommended, a recommendation operation is performed on the user to be recommended. For example, a text list is constructed based on the sequence of texts to be recommended and the text list is presented on the terminal device of the user to be recommended. Another example is that, based on the order of the texts in the sequence of texts to be recommended, one text in the sequence of texts to be recommended is recommended to the user in sequence, with each recommendation being made at a fixed time interval (e.g., 1 hour).

[0119] In some embodiments, before step 306, it also includes: determining the intersection text between the text sequence to be recommended and the text trigger sequence, and deleting the intersection text in the text sequence to be recommended; when the number of texts to be recommended belonging to the same attribute in the text sequence to be recommended exceeds a quantity threshold, obtaining the heat value of the texts to be recommended belonging to the same attribute, and deleting the text to be recommended with the smallest heat value until the number of texts to be recommended belonging to the same attribute is equal to the quantity threshold.

[0120] Before making recommendations based on the sequence of texts to be recommended, the sequence of texts to be recommended can also be rearranged. Specifically, first determine the intersection text between the sequence of texts to be recommended and the text trigger sequence, and delete the intersection text in the sequence of texts to be recommended to ensure that the user's visual fatigue will not be caused by the appearance of repeated texts. In addition, when the number of texts to be recommended belonging to the same attribute in the sequence of texts to be recommended exceeds the quantity threshold, obtain the heat value of the texts to be recommended belonging to the same attribute, and delete the texts to be recommended with the smallest heat value until the number of texts to be recommended belonging to the same attribute is equal to the quantity threshold, wherein the heat value can be a weighted result of the number of triggers and the number of comments of the texts to be recommended recorded in the background, and the quantity threshold can be set according to the actual application scenario, such as 2. In the above manner, it is possible to avoid recommending triggered texts to users, and it is possible to balance the number of texts belonging to each attribute in the sequence of texts to be recommended, thereby improving the diversity of recommendations.

[0121] Through the embodiment of the invention Figure 4C As can be seen from the above exemplary implementation, the embodiment of the present invention updates the recommendation model to fine-tune the recommended texts, thereby improving the targeting of the recommended users and further improving the accuracy of the text recommendation.

[0122] The following describes an exemplary application of an embodiment of the present invention in a practical application scenario. For ease of understanding, an example is given in which an electronic device is deployed in the cloud and the text is news.

[0123] The embodiment of the present invention can be applied to a news recommendation system, such as Figure 5 As shown, the recommendation system can be deployed in the background and includes a user profile module, a recall module, a click-through rate estimation module, and a re-ranking module, which are described in turn below. After receiving a request from the access layer, the recommendation system first calls the user profile module to obtain the profile of the user to be recommended (corresponding to the user profile attributes described above), and then sends the profile of the user to be recommended to the recall module. The access layer request can be issued by the user to be recommended in the news client, for example, by opening the news client on a terminal device or performing a refresh operation in the news client. The recall module performs a rough sorting of the news in the content pool and sends the sorted news to the click-through rate estimation module, where the content pool corresponds to the database described above. The click-through rate estimation module estimates the click-through rate of the recalled news and performs a fine sorting based on the estimated click-through rate, such as taking the k news with the highest click-through rate and sending them to the re-ranking module, where k is an integer greater than 0. The re-ranking module filters and sorts the k news for diversity, and recommends the resulting news sequence (corresponding to the text sequence to be recommended described above) to the user to be recommended, such as by presenting it in the news client.

[0124] Personalized recommendation is the goal of a recommendation system. Specifically, it refers to recommending news that may be of interest to the user based on their interests and click behavior. The click sequence of the user (corresponding to the text trigger sequence mentioned above) is crucial information for mining user interests. It not only reflects the interest distribution of the user but also implies the relationships between news items. Based on the BERT model, this embodiment of the present invention conducts intra-granular and inter-granular interest jump mining on click sequences. Granularity corresponds to the attributes mentioned above. Granularity can be a first-level news category, such as entertainment / sports, or a second-level news category, such as basketball / football under sports, or a news topic, etc., though this embodiment of the present invention does not limit this. During the mining process, a click sequence is first constructed based on the news items clicked by the user within a set time period. Based on the granularity of each news item in the click sequence, the click sequence is split into multiple subsequences. The clicked news items are obtained by processing the click records of the user on the news client. Furthermore, each subsequence contains news items of the same granularity. Then, any two subsequences are combined into a sequence sample.

[0125] As an example, in Figure 6In the figure, the original click sequence "News 1 -> News 2 -> ... -> News n+2" is shown, where "News 1" is the earliest news clicked by the user to be recommended within a set time period, and "News n+2" is the latest news clicked by the user to be recommended within a set time period, with n being an integer greater than 0. After splitting according to granularity, the subsequences corresponding to granularity 1 are shown as "News 1 -> News 4 -> News 6", the subsequence corresponding to granularity 2 is shown as "News 2 -> News 3 -> News 5", and the subsequence corresponding to granularity n is shown as "News n-3 -> News n -> News n+2". In the sequence sample construction step, sequence samples are shown, which are obtained by combining the subsequences corresponding to granularity 1 with the subsequences corresponding to granularity 2, and the sequence sample is shown by combining the subsequences corresponding to granularity 1 with the subsequences corresponding to granularity n.

[0126] The obtained sequence samples are input into the BERT model for training. The training process of the embodiment of the present invention can be divided into the pre-training (Pre-Train) stage and the fine-tuning (Fine-Tune) stage. In the Pre-Train stage, two prediction tasks are performed based on the BERT model. One prediction task is to predict the text at the prediction position in the sequence sample, that is, text fill-in-the-blank prediction; the other prediction task is to predict the association relationship between subsequences in the sequence sample, that is, next granularity prediction. The embodiment of the present invention provides such Figure 7 The schematic diagram of the input vector is shown in Figure 7 In the example, for the sequence sample including the subsequences "news 3->news 4->news 6" and "news 1->news 2->news 5", the input vector consists of three parts:

[0127] 1) Position vector representation (Position Embedding), such as Figure 7 The subscripts E0 and E3 shown in the figure represent the position of the corresponding news in the click sequence.

[0128] 2) Granularity vector representation (corresponding to the attribute vector representation above), such as Figure 7 E shown in 娱乐 and E 体育 It is worth noting that E in different positions 娱乐 Same, E 体育 Same thing.

[0129] 3) Text vector representation, such as Figure 7 E shown in 新闻3 and E 新闻4 wait.

[0130] The result of adding the above three vector representations is used as the final vector representation input to the BERT model. In addition, Figure 7The [CLS] symbol in the figure is a special symbol used for classification output, corresponding to the classification identifier mentioned above. [SEP] is a special symbol used to separate two subsequences within a sequence sample. During the Pre-Train phase, the BERT model continuously updates the vector representation of each news item in the sequence sample through backpropagation. As the BERT model completes training, the news item vector representation is also updated.

[0131] After the Pre-Train phase, an updated vector representation is obtained, and the Fine-Tune phase begins. The Fine-Tune phase is used to explore associations between granularities and between news items within a granularity. Essentially, the vector representations obtained in the Pre-Train phase are applied to specific business operations. For example, in the recall module, the similarity between news items is calculated based on the vector representations, and news items are recalled from the content pool based on this similarity. Another example is the CTR prediction module, which trains a CTR prediction model (corresponding to the recommendation model described above) based on user characteristics and the vector representations of the news items. The recalled news items are then ranked based on the estimated CTRs.

[0132] Through the above method, when recommending a news item that the user to be recommended has recently clicked on (corresponding to the text to be compared above), the news in the final news sequence is a comprehensive consideration of the user's interest development and potential interest prediction, rather than news that is extremely similar to the news that has been clicked on recently. In this way, the scope of recommended content is broadened, potential interests are expanded on the basis of satisfying user interests, and the predictability of recommendations is enhanced.

[0133] For ease of understanding, the present invention provides the following Figure 8 The recommended result diagram is shown in Figure 2. Figure 8 In the solution provided by the related technology, when performing text recommendation based on a certain news that the user has historically clicked, the results obtained are news 1, news 2, and news 3, etc. that are very similar to the news that the user has historically clicked; while with the text recommendation method provided by the embodiment of the present invention, when performing text recommendation based on the news that the user has historically clicked, the results obtained include interest jump news 4 and interest jump news 6 that have different granularity from the news that the user has historically clicked, and interest jump news 5 that has the same granularity as the news that the user has historically clicked, that is, the recommendation results are not limited to the granularity of the news that the user has historically clicked.

[0134] The news recommendation method described above is also applicable to recommendation systems in other scenarios, such as advertising, product, and movie recommendations. By mining interests within and across click sequences, we can foresee user interests, uncovering both peripheral and potential interests. This improves user satisfaction with text recommendations and enriches the ecosystem of recommendation systems.

[0135] Next, we will continue to describe the exemplary structure of the artificial intelligence-based text recommendation device 243 provided in the embodiment of the present invention as a software module. In some embodiments, for example, Figure 2 As shown, the software modules stored in the artificial intelligence-based text recommendation device 243 of the memory 240 may include: a splitting module 2431, which is used to split the text trigger sequence into multiple subsequences according to the attributes of the text in the text trigger sequence of the user to be recommended; wherein the attributes of the text included in each subsequence are the same; a prediction module 2432, which is used to combine any two subsequences into a sequence sample, and predict the text located at the predicted position in the sequence sample and the association relationship between the subsequences in the sequence sample based on the sequence sample to form a prediction result; an updating module 2433, which is used to update the vector representation of the text in the sequence sample according to the difference between the prediction result and the actual result; a screening module 2434, which is used to determine the similarity between the text in the text trigger sequence and the text in the database according to the vector representation of the text in the sequence sample, and based on the similarity, screen out the text to be recommended corresponding to the user to be recommended from the multiple texts in the database.

[0136] In some embodiments, the prediction module 2432 is further used to: perform mapping processing on each text in the sequence sample to obtain a vector representation of the text; and perform prediction processing on the vector representation of the text in the sequence sample through a language representation model to form a prediction result corresponding to the text located at the prediction position in the sequence sample and the association relationship between subsequences in the sequence sample.

[0137] In some embodiments, the updating module 2433 is further used to: perform back propagation in the language representation model according to the difference between the predicted result and the actual result, and update the vector representation of the text in the sequence sample during the back propagation process.

[0138] In some embodiments, the prediction module 2432 is further used to: perform vector initialization processing on each text in the sequence sample to obtain a text vector representation of the text; perform vector initialization processing on the attributes of each text in the sequence sample to obtain an attribute vector representation of the text; perform vector initialization processing on the position of each text in the sequence sample to obtain a position vector representation of the text; and fuse the text vector representation, attribute vector representation and position vector representation of the text to obtain a vector representation of the text.

[0139] In some embodiments, the screening module 2434 is also used to: perform any one of the following processing to obtain similarity: determine the similarity between the text in the text trigger sequence and the text in the database based on the text vector representation of the text in the sequence sample; determine the first similarity between the text in the text trigger sequence and the text in the database based on the attribute vector representation of the text in the sequence sample, and determine the second similarity between the text in the text trigger sequence and the text in the database based on the text vector representation of the text in the sequence sample, and take the result of weighted averaging of the first similarity and the second similarity as the similarity between the text in the text trigger sequence and the text in the database.

[0140] In some embodiments, the prediction module 2432 is further used to: randomly determine the position of the text in the sequence sample as the predicted position, until the proportion of the text corresponding to the predicted position reaches the predicted proportion; perform one of the following processing on the text corresponding to the predicted position: shielding processing, replacement processing, and retention processing; and add a classification identifier before the sequence sample; wherein the classification identifier is used to enable the language representation model to learn the association relationship between subsequences in the sequence sample.

[0141] In some embodiments, the prediction module 2432 is further used to: perform prediction processing on the vector representation of the text in the sequence sample through the language representation model to obtain an output vector corresponding to the prediction position and an output vector corresponding to the classification identifier; perform classification processing on the output vector corresponding to the prediction position to obtain a prediction result corresponding to the text located at the prediction position in the sequence sample; perform classification processing on the output vector corresponding to the classification identifier to obtain a prediction result corresponding to the association relationship between subsequences in the sequence sample.

[0142] In some embodiments, the update module 2433 is further used to: determine a first difference between a predicted result corresponding to a text at a predicted position in a sequence sample and an actual result based on a first loss function of the language representation model; determine a second difference between a predicted result corresponding to an association relationship between subsequences in the sequence sample and an actual result based on a second loss function of the language representation model; fuse the first difference and the second difference, and perform backpropagation in the language representation model based on the result of the summation process.

[0143] In some embodiments, the screening module 2434 is further used to: when the text in the database is different from the text in the sequence sample, determine the vector representation of the text in the database based on the sequence sample of the sample user; wherein the sequence sample of the sample user includes the text in the database; determine the text that meets the trigger condition in the text trigger sequence as the text to be compared; when there are multiple texts to be compared, add and average the vector representations of all texts to be compared to obtain an average vector representation; and determine the similarity between the average vector representation and the vector representation of the text in the database.

[0144] In some embodiments, the artificial intelligence-based text recommendation device 243 also includes: a portrait analysis module, which is used to perform portrait analysis on the text trigger sequence to obtain user portrait attributes; a preliminary screening module, which is used to screen texts in the database whose attributes meet the user portrait attributes to determine the similarity between the text in the text trigger sequence and the text screened out from the database.

[0145] In some embodiments, the artificial intelligence-based text recommendation device 243 also includes: a user feature acquisition module for acquiring user features of the user to be recommended; a first combination module for combining the user features with the vector representation of the text in the text trigger sequence into a trigger sample, and updating the recommendation model according to the trigger sample; a second combination module for combining the user features with the vector representation of the text to be recommended into a sample to be recommended; a label prediction module for predicting the sample to be recommended based on the updated recommendation model to obtain a predicted label; a sorting module for filtering the text to be recommended based on the predicted label, and sorting the filtered text to be recommended to obtain a sequence of texts to be recommended; and a refined recommendation module for executing recommendation operations corresponding to the user to be recommended based on the sequence of texts to be recommended.

[0146] In some embodiments, the first combination module is further used to: perform predictive processing on the trigger sample through the recommendation model to obtain a predicted label to be compared; perform backpropagation in the recommendation model based on the difference between the label of the trigger sample and the predicted label to be compared, and update the weight parameters of the recommendation model during the backpropagation process; wherein the label of the trigger sample is used to indicate that the user to be recommended has performed a trigger operation on the text corresponding to the trigger sample.

[0147] In some embodiments, the artificial intelligence-based text recommendation device 243 also includes: an intersection determination module, which is used to determine the intersection text between the text sequence to be recommended and the text trigger sequence, and delete the intersection text in the text sequence to be recommended; a rearrangement module, which is used to obtain the heat value of the text to be recommended belonging to the same attribute when the number of texts to be recommended belonging to the same attribute in the text sequence to be recommended exceeds a quantity threshold, and delete the text to be recommended with the smallest heat value until the number of texts to be recommended belonging to the same attribute is equal to the quantity threshold.

[0148] An embodiment of the present invention provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the artificial intelligence-based text recommendation method provided by the embodiment of the present invention, for example, Figure 4A 、 Figure 4B or Figure 4C It is worth noting that computers include various computing devices including terminal devices and servers.

[0149] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories.

[0150] In some embodiments, executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0151] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinating files (for example, files storing one or more modules, subroutines, or code portions).

[0152] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0153] In summary, the following technical effects can be achieved through the embodiments of the present invention:

[0154] 1) The text trigger sequence is split according to the text attributes, and sequence samples are constructed based on the subsequences obtained from the split, so as to mine the interests within and between attributes of the text trigger sequence. This enables the foresight of user interest development, broadens the scope of text recommendations, brings freshness to users while satisfying their interests, and improves the accuracy of text recommendations.

[0155] 2) The text recommendation method of the embodiment of the present invention can be applied to the recall module of the recommendation system, and can also be applied to the click rate prediction module. It is highly flexible and can be adjusted according to actual business scenarios.

[0156] 3) Before calculating the similarity between texts, the texts in the database can be preliminarily screened according to the user portrait attributes to reduce the processing pressure when calculating the similarity and save computing resources.

[0157] 4) When initializing the text in the sequence sample, the text is mapped into a vector representation that includes semantic information, syntactic information, and attribute information, which improves the comprehensiveness of the vector representation and also improves the effect of subsequent prediction tasks.

[0158] 5) The language representation model is used to simultaneously perform two prediction tasks: predicting text and predicting associations. The vector representation is updated based on the back-propagation mechanism, which improves the efficiency and accuracy of updating the vector representation.

[0159] 6) After obtaining the sequence of texts to be recommended, the sequence of texts to be recommended can be rearranged to further improve the user experience and increase the probability of users clicking on the recommended texts. The rearrangement strategy can be adjusted according to the actual application scenario.

[0160] The above are merely embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present invention are included in the scope of protection of the present invention.

Claims

1. A text recommendation method based on artificial intelligence, characterized in that: include: Obtain the texts in the database that are triggered by the user to be recommended within a set time period, and sort the texts in chronological order to obtain a text trigger sequence; According to the attributes of the text in the text trigger sequence of the user to be recommended, split the text trigger sequence into multiple subsequences; wherein the attributes of the text included in each subsequence are the same; Combining any two of the subsequences into a sequence sample, and performing mapping processing on each text in the sequence sample to obtain a vector representation of the text; Performing prediction processing on the vector representation of the text in the sequence sample using a language representation model to form a prediction result corresponding to the text at the prediction position in the sequence sample and the association relationship between subsequences in the sequence sample; Performing back propagation in the language representation model according to the difference between the predicted result and the actual result, and updating the vector representation of the text in the sequence sample during the back propagation process; Determine the similarity between the text in the text trigger sequence and the text in the database based on the vector representation of the text in the sequence sample, and According to the similarity, a text to be recommended corresponding to the user to be recommended is screened out from a plurality of texts in the database.

2. The text recommendation method according to claim 1, characterized in that The mapping process is performed on each text in the sequence sample to obtain a vector representation of the text, including: Performing vector initialization processing on each text in the sequence sample to obtain a text vector representation of the text; Performing vector initialization processing on the attributes of each text in the sequence sample to obtain an attribute vector representation of the text; Performing vector initialization processing on the position of each text in the sequence sample to obtain a position vector representation of the text; The text vector representation, attribute vector representation and position vector representation of the text are fused to obtain the vector representation of the text.

3. The text recommendation method according to claim 2, characterized in that Determining the similarity between the text in the text trigger sequence and the text in the database based on the vector representation of the text in the sequence sample includes: Perform any of the following processing to obtain the similarity: Determining the similarity between the text in the text trigger sequence and the text in the database based on the text vector representation of the text in the sequence sample; Determine a first similarity between the text in the text trigger sequence and the text in the database based on the attribute vector representation of the text in the sequence sample, and Based on the text vector representation of the text in the sequence sample, a second similarity between the text in the text trigger sequence and the text in the database is determined, and a result of weighted averaging of the first similarity and the second similarity is used as the similarity between the text in the text trigger sequence and the text in the database.

4. The text recommendation method according to claim 1, wherein: Before performing prediction processing on the vector representation of the text in the sequence sample by the language representation model, the method further includes: Randomly determining the position of the text in the sequence sample as the predicted position until the proportion of the text corresponding to the predicted position reaches the predicted proportion; performing one of masking, replacing, and retaining processing on the text corresponding to the predicted position; Adding a classification identifier before the sequence sample; The classification identifier is used to enable the language representation model to learn the association relationship between subsequences in the sequence sample.

5. The text recommendation method according to claim 4, characterized in that The predicting process of the vector representation of the text in the sequence sample by using the language representation model to form a prediction result corresponding to the text located at the prediction position in the sequence sample and the association relationship between subsequences in the sequence sample includes: Performing prediction processing on the vector representation of the text in the sequence sample using the language representation model to obtain an output vector corresponding to the predicted position and an output vector corresponding to the classification identifier; Performing classification processing on the output vector corresponding to the predicted position to obtain a prediction result corresponding to the text located at the predicted position in the sequence sample; Classification processing is performed on the output vector corresponding to the classification identifier to obtain a prediction result corresponding to the association relationship between subsequences in the sequence sample.

6. The text recommendation method according to claim 1, characterized in that The performing back propagation in the language representation model according to the difference between the predicted result and the actual result includes: determining, according to a first loss function of the language representation model, a first difference between a predicted result corresponding to the text at the predicted position in the sequence sample and an actual result; determining, according to a second loss function of the language representation model, a second difference between a predicted result corresponding to the association relationship between subsequences in the sequence sample and an actual result; The first difference and the second difference are merged, and Back propagation is performed in the language representation model based on the result of the summation process.

7. The text recommendation method according to any one of claims 1 to 6, characterized in that: Determining the similarity between the text in the text trigger sequence and the text in the database based on the vector representation of the text in the sequence sample includes: When the text in the database is different from the text in the sequence sample, determining a vector representation of the text in the database based on the sequence sample of the sample user; wherein the sequence sample of the sample user includes the text in the database; Determining the text that meets the trigger condition in the text trigger sequence as the text to be compared; When there are multiple texts to be compared, performing summing and averaging processing on the vector representations of all the texts to be compared to obtain an average vector representation; A similarity between the average vector representation and vector representations of text in the database is determined.

8. The text recommendation method according to any one of claims 1 to 6, characterized in that: Before determining the similarity between the text in the text trigger sequence and the text in the database based on the vector representation of the text in the sequence sample, the method further includes: Performing portrait analysis on the text trigger sequence to obtain user portrait attributes; Filter the text in the database whose attributes match the user portrait attributes to Determine the similarity between the text in the text trigger sequence and the text selected from the database.

9. The text recommendation method according to any one of claims 1 to 6, characterized in that: Also includes: Obtaining user characteristics of the user to be recommended; Combining the user features with the vector representation of the text in the text trigger sequence into a trigger sample, and updating the recommendation model according to the trigger sample; Combining the user features and the vector representation of the text to be recommended into a sample to be recommended; According to the updated recommendation model, prediction processing is performed on the sample to be recommended to obtain a prediction label; Filtering the texts to be recommended according to the predicted labels, and sorting the filtered texts to be recommended to obtain a sequence of texts to be recommended; Execute a recommendation operation corresponding to the user to be recommended according to the text sequence to be recommended.

10. The text recommendation method according to claim 9, characterized in that The updating of the recommendation model according to the triggering sample includes: Performing prediction processing on the trigger sample using the recommendation model to obtain a prediction label to be compared; According to the difference between the label of the triggering sample and the predicted label to be compared, back propagation is performed in the recommendation model, and During the back propagation process, the weight parameters of the recommendation model are updated; The label of the triggering sample is used to indicate that the user to be recommended has performed a triggering operation on the text corresponding to the triggering sample.

11. The text recommendation method according to claim 9, characterized in that Before performing the recommendation operation corresponding to the user to be recommended according to the text sequence to be recommended, the method further includes: Determining an intersection text between the to-be-recommended text sequence and the text trigger sequence, and deleting the intersection text from the to-be-recommended text sequence; When the number of recommended texts with the same attribute in the recommended text sequence exceeds a threshold, the popularity value of the recommended texts with the same attribute is obtained, and The text to be recommended with the smallest heat value is deleted until the number of the text to be recommended belonging to the same attribute is equal to the number threshold.

12. A text recommendation device based on artificial intelligence, characterized in that: include: A splitting module is used to obtain texts in the database triggered by the user to be recommended within a set time period, and sort the texts in chronological order to obtain a text trigger sequence; according to the attributes of the texts in the text trigger sequence of the user to be recommended, the text trigger sequence is split into multiple subsequences; wherein the attributes of the texts included in each subsequence are the same; A prediction module is configured to combine any two of the subsequences into a sequence sample, and perform mapping processing on each text in the sequence sample to obtain a vector representation of the text; and perform prediction processing on the vector representation of the text in the sequence sample using a language representation model to form a prediction result corresponding to the text at the prediction position in the sequence sample and the association relationship between the subsequences in the sequence sample; An updating module, configured to perform back propagation in the language representation model according to the difference between the predicted result and the actual result, and update the vector representation of the text in the sequence sample during the back propagation process; A screening module is used to determine the similarity between the text in the text trigger sequence and the text in the database based on the vector representation of the text in the sequence sample, and According to the similarity, a text to be recommended corresponding to the user to be recommended is screened out from a plurality of texts in the database.

13. The device according to claim 12, characterized in that The prediction module is further used to: perform vector initialization processing on each text in the sequence sample to obtain a text vector representation of the text; perform vector initialization processing on the attributes of each text in the sequence sample to obtain an attribute vector representation of the text; perform vector initialization processing on the position of each text in the sequence sample to obtain a position vector representation of the text; and fuse the text vector representation, attribute vector representation and position vector representation of the text to obtain a vector representation of the text.

14. The device according to claim 13, characterized in that The screening module is also used to: perform any one of the following processing to obtain similarity: determine the similarity between the text in the text trigger sequence and the text in the database based on the text vector representation of the text in the sequence sample; determine the first similarity between the text in the text trigger sequence and the text in the database based on the attribute vector representation of the text in the sequence sample, and determine the second similarity between the text in the text trigger sequence and the text in the database based on the text vector representation of the text in the sequence sample, and take the result of weighted averaging of the first similarity and the second similarity as the similarity between the text in the text trigger sequence and the text in the database.

15. The device according to claim 12, characterized in that The prediction module is further configured to: randomly determine the position of the text in the sequence sample as the predicted position until the proportion of the text corresponding to the predicted position reaches the predicted proportion; perform one of a masking process, a replacement process, and a retention process on the text corresponding to the predicted position; and add a classification identifier before the sequence sample; wherein the classification identifier is used to enable the language representation model to learn the association relationship between subsequences in the sequence sample.

16. The device according to claim 15, characterized in that The prediction module is further configured to: perform prediction processing on the vector representation of the text in the sequence sample using the language representation model to obtain an output vector corresponding to the prediction position and an output vector corresponding to the classification identifier; perform classification processing on the output vector corresponding to the prediction position to obtain a prediction result corresponding to the text located at the prediction position in the sequence sample; Classification processing is performed on the output vector corresponding to the classification identifier to obtain a prediction result corresponding to the association relationship between subsequences in the sequence sample.

17. The device according to claim 12, characterized in that The updating module is further configured to: determine, based on a first loss function of the language representation model, a first difference between a predicted result corresponding to a text at a predicted position in the sequence sample and an actual result; determine, based on a second loss function of the language representation model, a second difference between a predicted result corresponding to an association relationship between subsequences in the sequence sample and an actual result; perform a fusion process on the first difference and the second difference, and perform backpropagation in the language representation model based on a result of the summation process.

18. The device according to any one of claims 12 to 17, characterized in that The screening module is further configured to: when the text in the database is different from the text in the sequence sample, determine the vector representation of the text in the database based on the sequence sample of the sample user; wherein the sequence sample of the sample user includes the text in the database; determine the text in the text trigger sequence that meets the trigger condition as the text to be compared; when there are multiple texts to be compared, sum and average the vector representations of all the texts to be compared to obtain an average vector representation; and determine the similarity between the average vector representation and the vector representation of the text in the database.

19. The device according to any one of claims 12 to 17, characterized in that The device also includes: a portrait analysis module, which is used to perform portrait analysis on the text trigger sequence to obtain user portrait attributes; and a preliminary screening module, which is used to screen text in the database whose attributes meet the user portrait attributes to determine the similarity between the text in the text trigger sequence and the text screened out from the database.

20. The device according to any one of claims 12 to 17, characterized in that The device also includes: a user feature acquisition module for acquiring user features of the user to be recommended; a first combination module for combining the user features with the vector representation of the text in the text trigger sequence into a trigger sample, and updating the recommendation model according to the trigger sample; a second combination module for combining the user features with the vector representation of the text to be recommended into a sample to be recommended; a label prediction module for performing prediction processing on the sample to be recommended based on the updated recommendation model to obtain a predicted label; a sorting module for filtering the text to be recommended based on the predicted label, and sorting the filtered text to be recommended to obtain a sequence of texts to be recommended; and a refined recommendation module for executing a recommendation operation corresponding to the user to be recommended based on the sequence of texts to be recommended.

21. The device according to claim 20, characterized in that The first combination module is further used to: perform predictive processing on the trigger sample through the recommendation model to obtain a predicted label to be compared; perform backpropagation in the recommendation model based on the difference between the label of the trigger sample and the predicted label to be compared, and update the weight parameters of the recommendation model during the backpropagation process; wherein the label of the trigger sample is used to indicate that the user to be recommended has performed a trigger operation on the text corresponding to the trigger sample.

22. The device according to claim 20, characterized in that The device also includes: an intersection determination module, which is used to determine the intersection text between the to-be-recommended text sequence and the text trigger sequence, and delete the intersection text in the to-be-recommended text sequence; a rearrangement module, which is used to obtain the heat value of the to-be-recommended text belonging to the same attribute when the number of to-be-recommended texts belonging to the same attribute in the to-be-recommended text sequence exceeds a threshold, and delete the to-be-recommended text with the smallest heat value until the number of to-be-recommended texts belonging to the same attribute equals the threshold.

23. An electronic device, characterized in that: include: a memory for storing executable instructions; The processor is configured to implement the artificial intelligence-based text recommendation method according to any one of claims 1 to 11 when executing the executable instructions stored in the memory.

24. A computer-readable storage medium, characterized in that Executable instructions are stored, which are used to cause a processor to execute and implement the artificial intelligence-based text recommendation method according to any one of claims 1 to 11.

Images