Literature recommendation method and device based on double-tower model, equipment and medium

Abstract

The invention relates to the technical field of literature retrieval. The invention discloses a literature recommendation method, device and equipment based on a double-tower model and a medium, and the method comprises the steps: obtaining behavior data of a target user; according to the behavior data of the target user, extracting a behavior preference feature vector of the target user, and calculating a deep representation vector of the target user based on a double-tower model; obtaining a deep representation vector of the candidate literature based on a preset vector database, and calculating the similarity between the deep representation vector of the target user and the deep representation vector of the candidate literature; and generating a literature recommendation list of the target user according to the similarity. According to the method, the semantic features and the interest preference features of the user are obtained by deeply fusing the historical behavior data of the user, a literal matching category is exceeded, user requirements can be more accurately matched, and personalized recommendation is realized. And meanwhile, by utilizing the parallel processing capability of the double-tower model, the response speed of the recommendation system can be increased, and the recommendation efficiency is improved.

Description

Technical Field

[0001] The present application relates to the technical field of document retrieval, and in particular to a document recommendation method, device, equipment and medium based on a double-tower model. Background Art

[0002] In recent years, with the rapid increase in the amount of information, the number of academic documents has grown rapidly, bringing unprecedented challenges to users in finding and obtaining documents related to their research interests. Traditional document retrieval methods, such as keyword-based search, can often only return results based on literal matching, which is difficult to meet users' needs for personalized and highly relevant document recommendations. Therefore, personalized document recommendation systems have emerged.

[0003] However, existing personalized document recommendation methods, such as content-based recommendation and collaborative filtering recommendation, can achieve personalized recommendation to a certain extent, but they have many limitations. For example, the content-based recommendation method mainly relies on the text information such as the title and abstract of the document, which is still limited to the literal matching category; while the collaborative filtering recommendation method relies on the user-document interaction matrix. When the matrix is ​​very sparse, the recommendation effect cannot meet the user's personalized recommendation needs. Summary of the invention

[0004] In view of the above situation, the embodiments of the present application provide a document recommendation method, device, equipment and medium based on the double-tower model, aiming to solve the above problem or at least partially solve the above problem.

[0005] In a first aspect, the present application provides a document recommendation method based on a double-tower model, comprising:

[0006] Obtain the behavior data of target users;

[0007] Extracting a behavior preference feature vector of the target user according to the behavior data of the target user;

[0008] Based on the user tower of the pre-trained dual-tower model, a deep representation vector of the target user is calculated according to the behavioral preference feature vector of the target user;

[0009] Obtaining a deep-level representation vector of a candidate document based on a pre-set vector database, and calculating the similarity between the deep-level representation vector of the target user and the deep-level representation vector of the candidate document, wherein the deep-level representation vector of the candidate document is obtained by calculating the document tower of the pre-trained dual-tower model;

[0010] A document recommendation list for a target user is generated based on the similarity.

[0011] Preferably, extracting the target user's behavior preference feature vector according to the target user's behavior data includes:

[0012] Constructing a target user behavior document list according to the target user behavior data;

[0013] According to the target user behavior document list, obtain text data, subject code data, and publication time data of all documents;

[0014] According to the text data of all documents, a semantic feature vector of target user behavior documents is obtained;

[0015] According to the subject code data of all the documents, the subject preference feature vector of the target user behavior is obtained;

[0016] According to the publication time data of all documents, the publication time preference feature vector of the target user behavior is obtained;

[0017] A behavior preference feature vector of the target user is obtained according to the target user behavior document semantic feature vector, the target user behavior subject preference feature vector, and the target user behavior publication time preference feature vector.

[0018] Preferably, the target user's behavior data includes browsing data, downloading data, favorite data and attention data;

[0019] A target user behavior literature list is constructed based on the target user behavior data, including:

[0020] According to the behavior data of the target user, an initial list of target user behavior documents is established, wherein the initial list of target user behavior documents includes document items and time items;

[0021] Searching for multiple times corresponding to the target document in the initial list of target user behavior documents, and determining the latest time corresponding to the target document;

[0022] For the target document, the document item and time item corresponding to the latest time are retained, and the document items and time items corresponding to other times are deleted to obtain the target user behavior document list.

[0023] Preferably, obtaining a target user behavior document semantic feature vector based on the text data of all documents includes:

[0024] According to the text data of all documents, a document semantic vector of each document is obtained;

[0025] According to the document semantic vector of each document, the product of the document semantic vector of each document and a preset time decay function is calculated, that is, the document semantic decay vector of each document;

[0026] According to the document semantic decay vector of each document, the sum of the document semantic decay vectors of all documents is calculated;

[0027] According to the number of all documents in the target user behavior document list, the ratio of the sum of the document semantic attenuation vectors of all documents to the number of all documents is calculated to obtain the target user behavior document semantic feature vector.

[0028] Preferably, based on the pre-trained dual-tower model, the deep representation vector of the target user is calculated according to the behavior preference feature vector of the target user, including:

[0029] The behavior preference feature vector of the target user is input into the user tower of the dual-tower model, and a deep representation vector of the target user is obtained based on the multi-layer fully connected network in the user tower, wherein the deep representation vector of the target user is a vector normalized by L2 norm.

[0030] Preferably, the candidate document deep representation vector is obtained by calculating the document tower of the pre-trained dual-tower model, including:

[0031] Obtain candidate document data;

[0032] Extracting a candidate document feature vector according to the candidate document data;

[0033] The candidate document feature vector is input into the document tower of the pre-trained dual-tower model, and based on the multi-layer fully connected network in the document, the candidate document deep-level representation vector is obtained, wherein the candidate document deep-level representation vector is a vector normalized by L2 norm.

[0034] Preferably, extracting a candidate document feature vector according to the candidate document data includes:

[0035] According to the candidate document data, obtaining text data, subject code data, publication time data, and whether it is a core journal data of each candidate document in the document database;

[0036] Obtaining a semantic feature vector of each candidate document according to the text data of each candidate document;

[0037] According to the subject code data of each candidate document, a subject feature vector of each candidate document is obtained;

[0038] According to the publication time data of each candidate document, a publication time feature vector of each candidate document is obtained;

[0039] According to the core journal data of each candidate document, a core journal feature vector of each candidate document is obtained;

[0040] Obtaining a candidate document feature vector for each candidate document according to the semantic feature vector, the subject feature vector, the publication time feature vector, and the whether-it-is-a-core-journal feature vector of each candidate document;

[0041] According to the candidate document feature vector of each candidate document, the candidate document feature vectors of all candidate documents are obtained.

[0042] In a second aspect, a document recommendation device based on a double-tower model is provided, comprising:

[0043] An acquisition module is used to obtain the behavior data of target users;

[0044] An extraction module, used to extract a behavior preference feature vector of the target user based on the behavior data of the target user;

[0045] A first calculation module is used to calculate a deep representation vector of the target user based on a pre-trained dual-tower model and according to the behavior preference feature vector of the target user;

[0046] A second calculation module is used to obtain a deep-level representation vector of a candidate document based on a preset vector database, and calculate the similarity between the deep-level representation vector of the target user and the deep-level representation vector of the candidate document;

[0047] A generation module is used to generate a document recommendation list for a target user based on the similarity.

[0048] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the document recommendation method based on the double-tower model as described in the first aspect are implemented.

[0049] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the document recommendation method based on the double-tower model as described in the first aspect are implemented.

[0050] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:

[0051] This application deeply integrates user historical behavior data to obtain user historical semantic features and interest preference features, which goes beyond the scope of literal matching and can more accurately match user needs and achieve personalized recommendations. At the same time, the parallel processing capabilities of the dual-tower model can speed up the response speed of the recommendation system and improve the recommendation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0053] Figure 1 is a schematic diagram of an application environment of a document recommendation method based on a double-tower model in one embodiment of the present invention;

[0054] Figure 2 is a flow chart of a document recommendation method based on a double-tower model in one embodiment of the present invention;

[0055] Figure 3 yes Figure 2 A schematic flow chart of a specific implementation of step S50;

[0056] Figure 4 It is a flow chart of a dual-tower model training process in one embodiment of the present invention;

[0057] Figure 5 is a structural schematic diagram of a document recommendation device based on a double-tower model in one embodiment of the present invention;

[0058] Figure 6 is a schematic diagram of a structure of a computer device in one embodiment of the present invention;

[0059] Figure 7 It is another structural schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0061] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that such use is interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the term "including" and its variants are to be interpreted as open-ended terms meaning "including but not limited to".

[0062] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0063] As mentioned above, current personalized recommendations are still limited to literal matching, and it is difficult to maintain recommendation accuracy in a sparse data environment, and cannot meet personalized recommendation needs. In order to solve this technical problem, the embodiment of the present application provides a document recommendation method based on a dual-tower model.

[0064] The document recommendation method based on the double tower model provided by the embodiment of the present invention can be applied to Figure 1 In the application environment, the device communicates with the server through the network. The server can obtain the behavior data of the target user through the device; extract the behavior preference feature vector of the target user according to the behavior data of the target user; based on the user tower of the pre-trained dual-tower model, calculate the deep representation vector of the target user according to the behavior preference feature vector of the target user; obtain the deep representation vector of the candidate document based on the pre-set vector database, calculate the similarity between the deep representation vector of the target user and the deep representation vector of the candidate document, wherein the deep representation vector of the candidate document is obtained by calculating the document tower of the pre-trained dual-tower model; generate a document recommendation list for the target user according to the similarity. The present application obtains the user's historical semantic features and interest preference features by deeply integrating the user's historical behavior data, which goes beyond the scope of literal matching, can more accurately match user needs, and realize personalized recommendations. At the same time, the parallel processing capability of the dual-tower model can be used to speed up the response speed of the recommendation system and improve the recommendation efficiency. Among them, the device end can be but not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The server can be implemented with an independent server or a server cluster composed of multiple servers. The present invention is described in detail below through specific embodiments.

[0065] See also Figure 2 As shown, Figure 2 A schematic flow chart of a document recommendation method based on a double-tower model provided in an embodiment of the present invention includes the following steps:

[0066] S10: Obtaining behavior data of target users.

[0067] Specifically, the target user's behavior data includes browsing data, downloading data, favorite data, and attention data.

[0068] Specifically, in the process of daily document acquisition, users will generate various behavioral traces, such as browsing, downloading, collecting, and following. This embodiment focuses on collecting the document activity records of users in the four behaviors of browsing, downloading, collecting, and following in the past N months from detailed behavioral logs. Through these behavioral data, users' interest preferences and their interest migration trends that evolve over time can be deeply explored and revealed.

[0069] S20: Extracting a behavior preference feature vector of the target user based on the behavior data of the target user.

[0070] Behavioral preferences mainly include preferences in three aspects: document semantics, subject code, and publication time.

[0071] Specifically, they include:

[0072] S21: Construct a target user behavior document list according to the target user's behavior data.

[0073] Specifically, the browsing data, downloading data, favorite data and attention data are obtained. The browsing data includes the browsed documents and the browsing time, the downloading data includes the downloaded documents and the downloading time, the favorite data includes the favorite documents and the favorite time, and the attention data includes the attention documents and the attention time. In this embodiment, the documents in the browsing data, downloading data, favorite data and attention data are counted when the documents are counted, but the behavior types are not distinguished.

[0074] Specifically, step S21 constructs a target user behavior document list according to the target user's behavior data, including:

[0075] S211: Creating an initial list of target user behavior documents based on the target user's behavior data, wherein the initial list of target user behavior documents includes document items and time items.

[0076] Specifically, the document item is used to store the document identifier, which can be the document storage ID or the document code. It can be understood that a document has a document identifier. The time item is used to store the time when the behavior occurred. For example, Table 1 is the initial list of behavior documents. The Arabic numerals in the document item are the identification code of the document, and the time item is the time when the behavior occurred, which can be the browsing time, download time, collection time, or attention time.

[0077] Table 1 Initial list of behavioral literature

[0078]

[0079] S212: Searching for multiple times corresponding to the target document in the initial list of target user behavior documents, and determining the latest time corresponding to the target document.

[0080] Specifically, in one implementation, since a document has a document identifier, the document item is traversed, and multiple document identifiers in the document item are grouped, each group contains a document identifier, and the time corresponding to each group is further obtained to find out multiple times corresponding to the target document. The multiple times corresponding to the target document are sorted to determine the latest time corresponding to the target document. For example, in Table 1, the target document is a document with a document identifier code of "1", and the corresponding behavior occurrence times are found to be time 1, time 2, and time 6. If they are sorted in chronological order as time 1, time 2, and time 6, then the latest time corresponding to the target document is time 6.

[0081] S213: For the target document, the document item and time item corresponding to the latest time are retained, and the document items and time items corresponding to other times are deleted to obtain a target user behavior document list.

[0082] Specifically, for example, in Table 1, the document items and time items corresponding to time 6 are retained, and the document items and time items corresponding to time 1 and time 2 are deleted. The target user behavior document list obtained is shown in Table 2.

[0083] Table 2 List of behavioral literature

[0084]

[0085] S22: According to the target user behavior document list, the text data, subject code data, and publication time data of all documents are obtained.

[0086] Specifically, the text data includes document title, keywords, abstract, etc. The subject code data is the document subject code. The publication time data is the document publication time. According to the document item, the document title, keywords, abstract, document subject code, and document publication time of each document are obtained through the preset document database.

[0087] S23: Obtaining a semantic feature vector of target user behavior documents based on the text data of all documents;

[0088] According to the subject code data of all the documents, the subject preference feature vector of the target user behavior is obtained;

[0089] According to the publication time data of all documents, the publication time preference feature vector of the target user behavior is obtained.

[0090] Specifically, in step S23, the target user behavior document semantic feature vector is obtained according to the text data of all documents, including:

[0091] S231: Obtain the document semantic vector of each document based on the text data of all documents. Specifically, the process of obtaining the document semantic vector includes: for the text data of each document, the text data is spliced ​​and then the pre-processed document text is obtained by using text processing techniques such as word segmentation and stop words, and the pre-processed document text is input into the pre-trained text embedding model doc2vec to obtain the document semantic vector p of each document. i The examples of preprocessed literature text are shown in Table 3:

[0092] Table 3 Examples of preprocessed literature text

[0093]

[0094] S232: According to the document semantic vector of each document, the product of the document semantic vector of each document and a preset time decay function is calculated, that is, the document semantic decay vector of each document. Specifically, in order to reflect the user's interest migration change, a time decay function is preset based on the user's operation time:

[0095] λ(T)=β |T-t| (1)

[0096] Among them, β represents the time decay coefficient, and its value range is 0<β<1; T represents the current time; t represents the user behavior time; λ represents the time decay weight.

[0097] S233: Calculate the sum of the document semantic decay vectors of all documents according to the document semantic decay vector of each document;

[0098] S234: According to the number of all documents in the target user behavior document list, the ratio of the sum of the document semantic attenuation vectors of all documents to the number of all documents is calculated to obtain the target user behavior document semantic feature vector.

[0099] Specifically, the semantic feature vector of the target user behavior document is calculated according to the following formula:

[0100]

[0101] Where n represents the number of documents in the target user behavior document list; p i represents the document semantic vector of the i-th document; i Represents the time decay weight corresponding to the i-th document.

[0102] Through the above analysis, we can obtain the semantic feature vector of the target user behavior document and understand the semantic preference of the target user behavior document.

[0103] The target user behavior subject preference feature vector can be obtained by the following method: according to the document subject code, the subject preference of the user's historical behavior document is obtained through statistical analysis technology, and the subject preference of the user's historical behavior document is converted into the target user behavior subject preference feature vector u by using discretization technology, Lookup and embedding technology. subj .

[0104] The target user behavior publication time preference feature vector can be obtained by the following method: according to the publication time of the document, the publication time preference of the user's historical behavior document is obtained through statistical analysis technology, and the publication time preference of the user's historical behavior document is converted into the target user behavior publication time preference feature vector u by using discretization technology, Lookup and embedding technology. time .

[0105] S24: Obtaining a behavior preference feature vector of the target user according to the target user behavior document semantic feature vector, the target user behavior subject preference feature vector, and the target user behavior publication time preference feature vector.

[0106] Specifically, the target user behavior document semantic feature vector, the target user behavior subject preference feature vector, and the target user behavior publication time preference feature vector are concatenated to obtain the target user behavior preference feature vector u=[u hist ,u subj ,u time ].

[0107] S30: Based on the pre-trained dual-tower model, a deep representation vector of the target user is calculated according to the behavior preference feature vector of the target user.

[0108] Specifically, the dual-tower model consists of a user tower and a document tower, such as Figure 3 As shown. The user tower is composed of a multi-layer fully connected network. The behavior preference feature vector u of the target user is u=[u hist ,u subj ,u time ] is input into the user tower of the dual-tower model. Based on the multi-layer fully connected network in the user tower, the MLP fully connected network formula is as follows:

[0109] h (l) =α(W (l) h (l-1) +b (l) ) (3)

[0110] Among them, h (l) is the output of layer l; W (l) is the weight matrix of the lth layer; b (l)is the bias vector of the lth layer; α is the activation function (such as ReLU).

[0111] Get the deep representation vector h of the target user u , where the deep representation vector h of the target user u is a vector normalized by L2 norm, and the formula is as follows:

[0112]

[0113] Among them, L u is the number of floors of the user tower; is a vector The L2 norm of .

[0114] S40: Obtaining a deep representation vector of a candidate document based on a preset vector database, and calculating the similarity between the deep representation vector of the target user and the deep representation vector of the candidate document, wherein the deep representation vector of the candidate document is obtained by calculating the document tower of the pre-trained dual-tower model.

[0115] Specifically, the deep representation vector h of the candidate document is obtained from the pre-set vector database v The pre-set vector database contains a plurality of candidate document deep-level representation vectors. It can be understood that one document corresponds to one candidate document deep-level representation vector.

[0116] The similarity between the deep representation vector of the target user and the deep representation vector of the candidate document is calculated based on cosine similarity. In this embodiment, the calculation formula is as follows:

[0117] sim(h u ,h v )=h u ·h v (5)

[0118] This embodiment uses cosine similarity to measure the similarity between the user's deep-level representation vector and the candidate document's deep-level representation vector. The larger the cosine similarity, the more the user prefers the candidate document.

[0119] Specifically, the candidate document deep representation vector is obtained by calculating the document tower of the pre-trained dual-tower model, including:

[0120] S01: Obtain candidate document data.

[0121] Specifically, the candidate document data is the document data in the preset academic document database, which may include the currently commonly used academic document database, covering various types such as journals, conferences, patents, doctoral dissertations and master's theses, and the publication time span of these documents is wide. In order to meet specific business needs, this embodiment focuses on academic documents published in the past N years and selects them as candidate document data to be screened. Afterwards, the text information of the candidate documents is obtained by searching the academic document database, specifically including the core content such as the title, keywords and abstract of the document, and the additional attribute data of each document is collected, such as the publication time, the subject code to which it belongs, and whether it is a core journal and other key information.

[0122] S02: Extracting a candidate document feature vector based on the candidate document data.

[0123] Specifically, the candidate document feature vector is used to characterize the inherent attributes of the candidate document.

[0124] Step S02 extracts a candidate document feature vector based on the candidate document data, including:

[0125] S021: Based on the candidate document data, obtain text data, subject code data, publication time data, and whether it is a core journal data of each candidate document in the document database.

[0126] Specifically, the document database is a preset academic document database. The text data includes document titles, keywords, abstracts, etc. The subject code data is the document subject code. The publication time data is the document publication time. Whether it is a core journal data is whether the journal that publishes the document is a core journal.

[0127] S022: Obtaining a semantic feature vector of each candidate document based on the text data of each candidate document.

[0128] Specifically, referring to step S231, for the text data of each candidate document, the text data is concatenated and then processed by word segmentation, stop words and other text processing techniques to obtain a pre-processed document text, and the pre-processed document text is input into the pre-trained text embedding model doc2vec to obtain the document semantic vector v of each candidate document. text .

[0129] S023: Obtaining a subject feature vector of each candidate document based on the subject code data of each candidate document.

[0130] Specifically, according to the subject code of the candidate document, the subject characteristics of the candidate document are obtained through statistical analysis technology, and the discretization technology, Lookup and embedding technology are used to convert the subject characteristics of the candidate document into the subject feature vector v of the candidate document subj .

[0131] S024: According to the publication time data of each candidate document, obtain the publication time feature vector of each candidate document. Specifically, according to the publication time of the candidate document, obtain the publication time feature of the candidate document through statistical analysis technology, and use discretization technology, Lookup and embedding technology to convert the publication time feature of the candidate document into the publication time feature vector v of the candidate document. time .

[0132] S025: Obtain a core journal feature vector for each candidate document based on the core journal data for each candidate document.

[0133] Specifically, according to whether the candidate document is published in a core journal, the core journal characteristics of the candidate document are obtained through statistical analysis technology, and the discretization technology, Lookup and embedding technology are used to convert the core journal characteristics of the candidate document into the core journal feature vector v of the candidate document. jour .

[0134] S026: Obtain a candidate document feature vector for each candidate document based on the semantic feature vector, the subject feature vector, the publication time feature vector, and the whether-it-is-a-core-journal feature vector of each candidate document.

[0135] Specifically, the semantic feature vector v of each candidate document is text , the subject feature vector v subj , the publication time feature vector v time , whether it is a core journal feature vector v jour The candidate document feature vector v = [v text ,v subj ,v time ,v jour ]. A candidate document has a candidate document feature vector v.

[0136] S027: Obtain the candidate document feature vectors of all candidate documents based on the candidate document feature vector of each candidate document.

[0137] Specifically, according to the feature vector v of each candidate document, the candidate document feature vectors of all candidate documents are obtained.

[0138] S03: Input the candidate document feature vector into the document tower of the pre-trained dual-tower model, and obtain the candidate document deep representation vector h based on the multi-layer fully connected network of the document tower. v , where the candidate document deep level representation vector h v is a vector normalized by L2 norm.

[0139] Specifically, the literature tower is also composed of a multi-layer fully connected network. The MLP fully connected network formula is shown in (3).

[0140] Specifically, the candidate document deep representation vector h v is a vector normalized by L2 norm, and the formula is as follows:

[0141]

[0142] Among them, L v is the number of floors of the Literature Tower; is a vector The L2 norm of .

[0143] Specifically, we obtain the deep representation vector h of the candidate document v Stored in a vector database.

[0144] S50: Generate a document recommendation list for the target user according to the similarity.

[0145] Specifically, Figure 3 As shown, step S50 includes:

[0146] S51: According to the similarity, obtaining recommended documents corresponding to a positive similarity from the candidate documents;

[0147] S52: Sort the obtained recommended documents in descending order of similarity;

[0148] S53: Generate a document recommendation list for the target user according to preset recommendation rules.

[0149] Specifically, the preset recommendation rule can be set to recommend the top k documents based on the ranking results, where the k value is flexibly set according to the needs of the actual application scenario and resource allocation to balance the diversity and accuracy of the recommendation results.

[0150] Furthermore, in step S53, generating a document recommendation list for a target user according to a preset recommendation rule also includes:

[0151] S531: Generate an initial recommendation list of target user documents according to preset recommendation rules;

[0152] S532: Screening the initial document recommendation list for the target user according to the behavior data of the target user to obtain a document recommendation list for the target user.

[0153] Specifically, to avoid redundant recommendations, this embodiment obtains the documents corresponding to the browsing data and / or download data from the behavior data of the target user, performs a secondary screening on the target user's initial document recommendation list, and obtains the target user's document recommendation list:

[0154] R u =R init -F u (7)

[0155] Among them, R u Represents the document set in the target user's document recommendation list; R init represents the set of documents in the target user's initial recommendation list, F u Represents the set of documents in the target user's behavior data.

[0156] Through this filtering process, the degradation of user experience caused by repeated recommendations can be effectively avoided, and the novelty of the recommended content can be further improved, providing users with more valuable and attractive reading options. The recommendation method of this embodiment can adapt to different types of document recommendation scenarios and has good versatility.

[0157] The pre-training process of the dual-tower model in this embodiment is as follows: Figure 4 As shown, the training data in the training process comes from user behavior data of multiple users and a predetermined proportion of candidate document data obtained from the candidate document data through non-uniform sampling.

[0158] The specific process of extracting the user behavior preference feature vector through the user behavior data of multiple users during the training process is the same as the specific process of step S20, which will not be repeated here.

[0159] The specific process of extracting the candidate document feature vector using a predetermined proportion of the candidate document data during the training process is the same as the specific process of step S02.

[0160] like Figure 4 As shown, during the training process, the user behavior preference feature vector obtained from the training data is input into the user tower to obtain the deep representation vector of the user of the training data, as shown in formula (4); the candidate document feature vector obtained from the training data is input into the document tower to obtain the deep representation vector of the candidate document of the training data, as shown in formula (6). Then, the similarity between the deep representation vector of the user of the training data and the deep representation vector of the candidate document is calculated by dot product, and the cross entropy loss function is used as the optimization target, and the weights and biases of the user tower and the document tower are updated using the back propagation algorithm to obtain the pre-trained dual-tower model.

[0161] Specifically, the user behavior data in the training data can be collected from the behavior logs of multiple users, including browsing data, downloading data, collection data, and attention data of multiple users in the past N months.

[0162] Specifically, in the present embodiment, cosine similarity is used during the training process to measure the similarity between the user's deep-level representation vector and the candidate document's deep-level representation vector. The greater the cosine similarity, the more the user prefers the candidate document.

[0163] Specifically, the cross entropy loss function formula is:

[0164]

[0165] Where N is the number of samples; y i is the true label of the i-th sample (1 means the user is interested in the document, 0 means not interested), is the predicted probability of the ith sample (obtained by passing the similarity score through the sigmoid function):

[0166]

[0167] Where σ is the sigmoid function.

[0168] This implementation uses the back propagation algorithm to update the weights and biases of the user tower and the document tower to minimize the cross entropy loss function. The model in the training process is evaluated with indicators such as recall, precision, and F1 score to determine the final weights and biases, and the model with the best indicators is selected to be launched in the production environment.

[0169] To improve computing performance, this embodiment makes full use of the parallel computing capability of the dual-tower model, inputs the user feature vector into the trained user tower to obtain the user deep-level representation vector, and simultaneously inputs the candidate document feature vector into the trained document tower to obtain the candidate document deep-level representation vector.

[0170] This embodiment introduces a variety of user and document auxiliary information through the dual-tower model, which can better cope with data changes and noise interference, and enhance the robustness of the recommendation system. At the same time, the parallel processing capability of the dual-tower model can speed up the response speed of the recommendation system and improve the recommendation efficiency.

[0171] Specifically, in this embodiment, the process of acquiring the behavior data of the target user in the application process and the user behavior data of multiple users in the training process are both authorized by the user.

[0172] It can be seen that in the above scheme, this application obtains the user's historical semantic features and interest preference features by deeply integrating the user's historical behavior data, which goes beyond the scope of literal matching, can more accurately match user needs, and realize personalized recommendations. This application constructs a more accurate user and document matching mechanism by deeply integrating the user's historical behavior document semantic features and interest preference features, combined with the multi-dimensional attribute characteristics of candidate documents. The dual-tower model can better cope with data changes and noise interference by introducing a variety of user and document auxiliary information, thereby enhancing the robustness of the recommendation system. By utilizing the parallel processing capabilities of the dual-tower model, the response speed of the recommendation system can be accelerated and the recommendation efficiency can be improved. It can adapt to different types of document recommendation scenarios and has good versatility.

[0173] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0174] In one embodiment, a document recommendation device based on a double-tower model is provided, and the document recommendation device based on a double-tower model corresponds one-to-one to the document recommendation method based on a double-tower model in the above embodiment. Figure 5 As shown, the document recommendation device based on the double tower model includes an acquisition module 101, an extraction module 102, a first calculation module 103, a second calculation module 104, and a generation module 105. The detailed description of each functional module is as follows:

[0175] An acquisition module 101 is used to acquire the behavior data of a target user;

[0176] An extraction module 102, configured to extract a behavior preference feature vector of a target user based on the behavior data of the target user;

[0177] A first calculation module 103 is used to calculate a deep representation vector of a target user based on a pre-trained dual-tower model and according to the behavior preference feature vector of the target user;

[0178] A second calculation module 104 is used to obtain a deep-level representation vector of a candidate document based on a preset vector database, and calculate the similarity between the deep-level representation vector of the target user and the deep-level representation vector of the candidate document;

[0179] The generating module 105 is used to generate a document recommendation list for a target user according to the similarity.

[0180] Specifically, the acquisition module 101 is further used to:

[0181] Get the user's browsing data, download data, favorite data and follow data.

[0182] Specifically, the extraction module 102 is further used to:

[0183] Constructing a target user behavior document list according to the target user behavior data;

[0184] According to the target user behavior document list, obtain text data, subject code data, and publication time data of all documents;

[0185] According to the text data of all documents, a semantic feature vector of target user behavior documents is obtained;

[0186] According to the subject code data of all the documents, the subject preference feature vector of the target user behavior is obtained;

[0187] According to the publication time data of all documents, the publication time preference feature vector of the target user behavior is obtained;

[0188] A behavior preference feature vector of the target user is obtained according to the target user behavior document semantic feature vector, the target user behavior subject preference feature vector, and the target user behavior publication time preference feature vector.

[0189] Specifically, the extraction module 102 is further used to:

[0190] According to the behavior data of the target user, an initial list of target user behavior documents is established, wherein the initial list of target user behavior documents includes document items and time items;

[0191] Searching for multiple times corresponding to the target document in the initial list of target user behavior documents, and determining the latest time corresponding to the target document;

[0192] For the target document, the document item and time item corresponding to the latest time are retained, and the document items and time items corresponding to other times are deleted to obtain the target user behavior document list.

[0193] Specifically, the extraction module 102 is further used to:

[0194] According to the text data of all documents, a document semantic vector of each document is obtained;

[0195] According to the document semantic vector of each document, the product of the document semantic vector of each document and a preset time decay function is calculated, that is, the document semantic decay vector of each document;

[0196] According to the document semantic decay vector of each document, the sum of the document semantic decay vectors of all documents is calculated;

[0197] According to the number of all documents in the target user behavior document list, the ratio of the sum of the document semantic attenuation vectors of all documents to the number of all documents is calculated to obtain the target user behavior document semantic feature vector.

[0198] Specifically, the first calculation module 103 is further used to:

[0199] The behavior preference feature vector of the target user is input into the user tower of the dual-tower model, and a deep representation vector of the target user is obtained based on the multi-layer fully connected network in the user tower, wherein the deep representation vector of the target user is a vector normalized by L2 norm.

[0200] Specifically, the generating module 105 is further used to:

[0201] According to the similarity, obtaining recommended documents corresponding to when the similarity is a positive value from the candidate documents;

[0202] Sorting the obtained recommended documents in order of the similarity from high to low;

[0203] Generate a document recommendation list for the target user based on preset recommendation rules.

[0204] Specifically, the generating module 105 is further used to:

[0205] Generate an initial recommendation list of target user documents based on preset recommendation rules;

[0206] The target user's initial document recommendation list is screened according to the target user's behavior data to generate a document recommendation list for the target user.

[0207] Specifically, the acquisition module 101 is further used to:

[0208] Obtain candidate document data.

[0209] Specifically, the extraction module 102 is further used to:

[0210] A candidate document feature vector is extracted based on the candidate document data.

[0211] Specifically, the extraction module 102 is further used to:

[0212] According to the candidate document data, obtaining text data, subject code data, publication time data, and whether it is a core journal data of each candidate document in the document database;

[0213] Obtaining a semantic feature vector of each candidate document according to the text data of each candidate document;

[0214] According to the subject code data of each candidate document, a subject feature vector of each candidate document is obtained;

[0215] According to the publication time data of each candidate document, a publication time feature vector of each candidate document is obtained;

[0216] According to the core journal data of each candidate document, a core journal feature vector of each candidate document is obtained;

[0217] Obtaining a candidate document feature vector for each candidate document according to the semantic feature vector, the subject feature vector, the publication time feature vector, and the whether-it-is-a-core-journal feature vector of each candidate document;

[0218] According to the candidate document feature vector of each candidate document, the candidate document feature vectors of all candidate documents are obtained.

[0219] During the training process of the pre-trained dual-tower model,

[0220] Specifically, the acquisition module 101 is further used to:

[0221] A predetermined proportion of candidate document data is obtained from the candidate document data, and user behavior data of multiple users is obtained as training data.

[0222] Specifically, the extraction module 102 is further used to:

[0223] Extracting a user behavior preference feature vector of the training data based on the user behavior data of the training data;

[0224] According to the candidate document data of the training data, a candidate document feature vector of the training data is extracted.

[0225] Specifically, the first calculation module 103 is further used to:

[0226] Inputting the user behavior preference feature vector of the training data into the user tower of the dual-tower model, and calculating the user deep level representation vector of the training data, wherein the user tower includes a multi-layer fully connected network;

[0227] The candidate document feature vectors of the training data are input into the document tower of the dual-tower model to calculate the candidate document deep representation vectors of the training data, wherein the document tower includes a multi-layer fully connected network.

[0228] Specifically, the second calculation module 104 is further used to:

[0229] Calculate the similarity between the user deep representation vector of the training data and the candidate document deep representation vector.

[0230] Specifically, it also includes an optimization module 106, which is used to:

[0231] According to the similarity between the user deep representation vector of the training data and the candidate document deep representation vector, the cross entropy loss function is used as the optimization target, and the back propagation algorithm is used to update the weights and biases of the user tower and the document tower to obtain the pre-trained dual-tower model.

[0232] The present invention provides a document recommendation device based on a dual-tower model. This application deeply integrates user historical behavior data to obtain user historical semantic features and interest preference features, which goes beyond the scope of literal matching and can more accurately match user needs to achieve personalized recommendations. This application deeply integrates user historical behavior document semantic features and interest preference features, combined with the multi-dimensional attribute features of candidate documents, to construct a more accurate user and document matching mechanism. The dual-tower model can better cope with data changes and noise interference by introducing a variety of user and document auxiliary information, thereby enhancing the robustness of the recommendation system. By utilizing the parallel processing capabilities of the dual-tower model, the response speed of the recommendation system can be accelerated and the recommendation efficiency can be improved. It can adapt to different types of document recommendation scenarios and has good versatility.

[0233] For the specific definition of the document recommendation device based on the twin-tower model, please refer to the definition of the document recommendation method based on the twin-tower model in the above text, which will not be repeated here. Each module in the above-mentioned document recommendation device based on the twin-tower model can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0234] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external device via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a service-side document recommendation method based on a dual-tower model.

[0235] In one embodiment, a computer device is provided. The computer device may be a device end, and its internal structure diagram may be as follows: Figure 7As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, the functions or steps of a device side of a document recommendation method based on a dual-tower model are implemented.

[0236] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0237] Obtain the behavior data of target users;

[0238] Extracting a behavior preference feature vector of the target user according to the behavior data of the target user;

[0239] Based on the pre-trained dual-tower model, a deep representation vector of the target user is calculated according to the behavioral preference feature vector of the target user;

[0240] Obtaining a deep representation vector of a candidate document based on a preset vector database, and calculating the similarity between the deep representation vector of the target user and the deep representation vector of the candidate document;

[0241] A document recommendation list for a target user is generated based on the similarity.

[0242] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0243] Obtain the behavior data of target users;

[0244] Extracting a behavior preference feature vector of the target user according to the behavior data of the target user;

[0245] Based on the pre-trained dual-tower model, a deep representation vector of the target user is calculated according to the behavioral preference feature vector of the target user;

[0246] Obtaining a deep representation vector of a candidate document based on a preset vector database, and calculating the similarity between the deep representation vector of the target user and the deep representation vector of the candidate document;

[0247] A document recommendation list for a target user is generated based on the similarity.

[0248] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the device side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0249] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0250] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0251] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A document recommendation method based on a double tower model, characterized in that: include: Obtain the behavior data of target users; Extracting a behavior preference feature vector of the target user according to the behavior data of the target user; Based on the user tower of the pre-trained dual-tower model, a deep representation vector of the target user is calculated according to the behavioral preference feature vector of the target user; Obtaining a deep-level representation vector of a candidate document based on a pre-set vector database, and calculating the similarity between the deep-level representation vector of the target user and the deep-level representation vector of the candidate document, wherein the deep-level representation vector of the candidate document is obtained by calculating the document tower of the pre-trained dual-tower model; A document recommendation list for a target user is generated based on the similarity.

2. The method according to claim 1, characterized in that Extracting the target user's behavior preference feature vector according to the target user's behavior data includes: Constructing a target user behavior document list according to the target user behavior data; According to the target user behavior document list, obtain text data, subject code data, and publication time data of all documents; According to the text data of all documents, a semantic feature vector of target user behavior documents is obtained; According to the subject code data of all the documents, the subject preference feature vector of the target user behavior is obtained; According to the publication time data of all documents, the publication time preference feature vector of the target user behavior is obtained; A behavior preference feature vector of the target user is obtained according to the target user behavior document semantic feature vector, the target user behavior subject preference feature vector, and the target user behavior publication time preference feature vector.

3. The method according to claim 2, characterized in that The target user's behavior data includes browsing data, downloading data, favorite data and attention data; A target user behavior literature list is constructed based on the target user behavior data, including: According to the behavior data of the target user, an initial list of target user behavior documents is established, wherein the initial list of target user behavior documents includes document items and time items; Searching for multiple times corresponding to the target document in the initial list of target user behavior documents, and determining the latest time corresponding to the target document; For the target document, the document item and time item corresponding to the latest time are retained, and the document items and time items corresponding to other times are deleted to obtain the target user behavior document list.

4. The method according to claim 2, characterized in that: According to the text data of all documents, the semantic feature vector of the target user behavior document is obtained, including: According to the text data of all documents, a document semantic vector of each document is obtained; According to the document semantic vector of each document, the product of the document semantic vector of each document and a preset time decay function is calculated, that is, the document semantic decay vector of each document; According to the document semantic decay vector of each document, the sum of the document semantic decay vectors of all documents is calculated; According to the number of all documents in the target user behavior document list, the ratio of the sum of the document semantic attenuation vectors of all documents to the number of all documents is calculated to obtain the target user behavior document semantic feature vector.

5. The method according to claim 1, characterized in that Based on the user tower of the pre-trained dual-tower model, the deep representation vector of the target user is calculated according to the behavior preference feature vector of the target user, including: The behavior preference feature vector of the target user is input into the user tower of the dual-tower model, and a deep representation vector of the target user is obtained based on the multi-layer fully connected network in the user tower, wherein the deep representation vector of the target user is a vector normalized by L2 norm.

6. The method according to claim 1, characterized in that The candidate document deep representation vector is obtained by calculating the document tower of the pre-trained dual-tower model, including: Obtain candidate document data; Extracting a candidate document feature vector according to the candidate document data; The candidate document feature vector is input into the document tower of the pre-trained dual-tower model, and based on the multi-layer fully connected network in the document, the candidate document deep-level representation vector is obtained, wherein the candidate document deep-level representation vector is a vector normalized by L2 norm.

7. The method according to claim 6, characterized in that Extracting a candidate document feature vector according to the candidate document data includes: According to the candidate document data, obtaining text data, subject code data, publication time data, and whether it is a core journal data of each candidate document in the document database; Obtaining a semantic feature vector of each candidate document according to the text data of each candidate document; According to the subject code data of each candidate document, a subject feature vector of each candidate document is obtained; According to the publication time data of each candidate document, a publication time feature vector of each candidate document is obtained; According to the core journal data of each candidate document, a core journal feature vector of each candidate document is obtained; Obtaining a candidate document feature vector for each candidate document according to the semantic feature vector, the subject feature vector, the publication time feature vector, and the whether-it-is-a-core-journal feature vector of each candidate document; According to the candidate document feature vector of each candidate document, the candidate document feature vectors of all candidate documents are obtained.

8. A document recommendation device based on a double tower model, characterized in that: include: An acquisition module is used to obtain the behavior data of target users; An extraction module, used to extract a behavior preference feature vector of the target user based on the behavior data of the target user; A first calculation module is used to calculate a deep representation vector of the target user based on a pre-trained dual-tower model and according to the behavior preference feature vector of the target user; A second calculation module is used to obtain a deep-level representation vector of a candidate document based on a preset vector database, and calculate the similarity between the deep-level representation vector of the target user and the deep-level representation vector of the candidate document; A generation module is used to generate a document recommendation list for a target user based on the similarity.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the document recommendation method based on the double-tower model as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the document recommendation method based on the double-tower model as described in any one of claims 1 to 7 are implemented.

Images