Literature retrieval method and system, computer equipment and storage medium

Abstract

The invention relates to a literature retrieval method and system, computer equipment and a storage medium. The method comprises the following steps: extracting feature data of training data; constructing at least one optimization loss function: constructing a first loss function according to the distance between the feature data with the same feature domain; constructing a second loss function according to the distance between the feature data of different feature domains; inputting the feature data and the disturbed feature data into an encoder to obtain a first compression feature and a second compression feature, and constructing a third loss function according to the distance between the first compression feature and the second compression feature; constructing a basic loss function according to a predicted value of the click rate obtained by prediction of the prediction model and a corresponding true value; adjusting parameters of the prediction model according to the at least one optimization loss function and the basic loss function; and predicting the click rate of the candidate literature associated with the target user request based on the prediction model after parameter adjustment. By adopting the method, the accuracy of predicting the document click rate by the user can be improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a document retrieval method, system, computer device and storage medium. Background Art

[0002] The rapid development of astronomical research has brought about an explosive growth in the number of astronomical research papers. According to statistics, the number of papers published each year in mainstream astronomical journals and preprint platforms has exceeded 100,000, and is increasing at an average annual rate of 15%. These documents not only contain traditional text content, but also a large amount of multimodal scientific data such as observational images, spectral data, and radio data. For astronomers, how to quickly and accurately find literature related to their research direction from a large amount of literature, and to be able to implement effective literature recommendations based on user personalized data, has become a difficult problem that needs to be solved urgently.

[0003] Current click-through rate prediction methods and corresponding astronomical search recommendation systems face many problems, such as data sparsity, feature sparsity, and uneven feature distribution. These problems make it impossible to fully optimize feature representation using only supervised learning training methods, limiting most models from achieving better prediction results. The effect of astronomical recommendation search systems that use click-through rate prediction models as ranking modules is also limited.

[0004] There is currently no effective solution to the problem of inaccurate results in predicting document click-through rates in related technologies. Summary of the invention

[0005] Based on this, it is necessary to provide a document retrieval method, system, computer device and storage medium that can solve the problem of inaccurate document click-through rate prediction results in response to the above technical problems.

[0006] In a first aspect, a document retrieval method is provided in this embodiment, the method comprising: Extracting feature data from training data, wherein the training data includes user data and document data; Constructing at least one optimization loss function, the optimization loss function comprising: a first loss function constructed according to the distance between the feature data with the same feature domain; a second loss function constructed according to the distance between the feature data with different feature domains; inputting the feature data into an encoder to obtain a first compressed feature, inputting the feature data after perturbation processing into the encoder to obtain a second compressed feature, and constructing a third loss function according to the distance between the first compressed feature and the second compressed feature; Constructing a basic loss function for training the prediction model according to a predicted value of the click rate predicted by the prediction model based on the feature data and a true value of the feature data corresponding to the predicted value; Adjusting parameters of the prediction model according to at least one of the optimization loss function and the basic loss function; The click rate of the candidate document associated with the target user's request is predicted based on the prediction model after adjusting the parameters.

[0007] In some of the embodiments, constructing at least one optimization loss function comprises: Acquire a feature domain corresponding to the feature data; Calculating a virtual central node in the feature domain; The first loss function is constructed according to the distance between the feature data in the same feature domain and the virtual central node of the feature domain; wherein the first loss function is used to minimize the distance between the feature data corresponding to the same feature domain.

[0008] In some of the embodiments, constructing at least one optimization loss function comprises: Acquire a feature domain corresponding to the feature data; Calculating a virtual central node in the feature domain; The second loss function is constructed according to the distance between the virtual central nodes of multiple different feature domains; wherein the second loss function is used to maximize the distance between the feature data corresponding to different feature domains.

[0009] In some of the embodiments, calculating a virtual central node in a feature domain includes: Normalizing a plurality of feature data corresponding to the same feature domain; The virtual central node is obtained according to the average of the plurality of characteristic data.

[0010] In some of the embodiments, constructing at least one optimization loss function comprises: Obtain a first encoder and a second encoder; wherein the first encoder and the second encoder have the same structure but different parameters; Inputting the feature data into the first encoder to obtain the first compression feature; Inputting the feature data after disturbance processing into the second encoder to obtain the second compressed feature; calculating a third distance between the first compressed feature and the second compressed feature; The third loss function is constructed according to the third distance; wherein the third loss function is used to minimize the distance between the first compressed feature and the second compressed feature.

[0011] In some embodiments, a basic loss function for training a prediction model is constructed according to a predicted value of a click rate predicted by the prediction model based on the feature data and a true value of the feature data corresponding to the predicted value, including: Performing feature conversion processing and dimensionality reduction processing on the feature data; Inputting the feature data into a first encoder to obtain a first compressed feature; Inputting the first compression feature into the prediction model to obtain a predicted value of the click rate output by the prediction model based on the first compression feature; The basic loss function is constructed based on the predicted value and the true value of the click rate corresponding to the predicted value in the feature data.

[0012] In some embodiments, predicting the click rate of the candidate document associated with the target user's request based on the prediction model after adjusting the parameters includes: Acquiring the feature data optimized during the parameter adjustment process of the prediction model; Acquire candidate feature data corresponding to the candidate document from the optimized feature data; The prediction model predicts the click rate of the candidate document according to the candidate feature data and the target user request; A search list of the document is obtained according to the click rate of the candidate document.

[0013] In a second aspect, a document retrieval system is provided in this embodiment, the system comprising: A training module is used to extract feature data from training data, wherein the training data includes user data and document data; construct at least one optimization loss function, wherein the optimization loss function includes: a first loss function constructed according to the distance between the feature data with the same feature domain; a second loss function constructed according to the distance between the feature data with different feature domains; inputting the feature data into an encoder to obtain a first compressed feature, inputting the feature data after perturbation processing into the encoder to obtain a second compressed feature, and constructing a third loss function according to the distance between the first compressed feature and the second compressed feature; constructing a basic loss function for training a prediction model according to a predicted value of a click-through rate predicted by a prediction model based on the feature data and a true value of the feature data corresponding to the predicted value; and adjusting the parameters of the prediction model according to at least one of the optimization loss function and the basic loss function; A click rate prediction module, used to predict the click rate of the candidate document associated with the target user's request based on the prediction model after adjusting the parameters; The sorting and display module is used to obtain a document search list according to the click rate of the candidate documents.

[0014] In a third aspect, a computer device is provided in this embodiment, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the document retrieval method described in the first aspect is implemented.

[0015] In a fourth aspect, a computer-readable storage medium is provided in this embodiment, on which a computer program is stored. When the computer program is executed by a processor, the document retrieval method described in the first aspect is implemented.

[0016] The above-mentioned document retrieval method, system, computer device and storage medium construct a first loss function and a second loss function according to the distance between feature data and the feature domain corresponding to the feature data, construct a third loss function according to the first compressed feature and the second compressed feature obtained after compression by the encoder, optimize the feature representation through any one or more optimized loss functions among the first loss function, the second loss function and the third loss function, and train the prediction model based on more accurate feature representation, which can improve the accuracy of the trained prediction model in predicting click-through rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 An application environment diagram of a document retrieval method in an embodiment; Figure 2 A schematic diagram of a process of a document retrieval method in one embodiment; Figure 3 is a flow chart of a click rate prediction method based on feature representation in one embodiment; Figure 4 A schematic diagram of a document recommendation system in one embodiment; Figure 5 is a schematic diagram of an alignment-uniformity unit in one embodiment; Figure 6 A schematic diagram of offline training module training in one embodiment; Figure 7 A schematic diagram showing a sorting display module in an embodiment; Figure 8 is a structural block diagram of a document retrieval system in one embodiment; Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0019] In the related art, when training the click-through rate prediction model, there are many and sparse features, and the parameters of the feature representation will occupy a large proportion of the parameters of the click-through rate model. The related training method is to jointly train the feature representation parameters and the parameters of the feature interaction and prediction layers. Although this method can achieve end-to-end parameter optimization; however, it will cause the feature interaction parameters and prediction layer parameters, which have a smaller proportion, to be over-optimized during the training process, while the feature representation parameters are not fully optimized due to their large proportion, making it impossible for the performance of the click-through rate prediction model to reach the overall optimal.

[0020] In addition, the unevenness of feature distribution is an intuitive reason that affects the optimization of feature representation, which is specifically manifested as the "long-tail distribution" problem of feature frequency. In the process of related model training methods, the corresponding parameters of high-frequency features are optimized significantly more times than low-frequency features, which leads to sub-optimal feature representations corresponding to low-frequency features. In addition, because the quality of feature representation in deep learning models will directly affect the prediction effect of downstream structures, the feature representation corresponding to low-frequency features will make it impossible for the model to obtain accurate prediction results during the prediction process. This is also a common problem of current click-through rate prediction models based on feature interaction.

[0021] The document retrieval method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown in FIG. 1 , the terminal 102 communicates with the server 104 via a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed on a cloud or other network server. Figure 1 The application environment shown implements the training of the prediction model and predicts the click rate of the candidate document when receiving the target user's request. The terminal 102 can be, but is not limited to, various personal computers, laptops, tablet computers, etc. The server 104 can be implemented as an independent server or a server cluster composed of multiple servers.

[0022] In one embodiment, Figure 2 As shown, a literature retrieval method is provided, which is applied to Figure 1 The terminal 102 in the example is used as an example to illustrate, and the following steps are included: Step S202: extracting feature data from training data, where the training data includes user data and document data.

[0023] The user data includes but is not limited to the basic attributes of the user, user behavior data, such as user ID, user access time distribution, user frequently visited regions, and user historical search behavior. The document data can be multimodal document data, including but not limited to document identification, document content, and document statistical information. Optionally, the user data and document data are used as training data, and feature data in the training data is extracted through existing statistical methods or machine learning models.

[0024] Step S204, construct at least one optimization loss function, the optimization loss function includes: a first loss function constructed according to the distance between feature data with the same feature domain; a second loss function constructed according to the distance between feature data with different feature domains; inputting the feature data into the encoder to obtain a first compressed feature, inputting the feature data after disturbance processing into the encoder to obtain a second compressed feature, and constructing a third loss function according to the distance between the first compressed feature and the second compressed feature.

[0025] Among them, the feature domain is a set of feature data with common semantics or attributes; therefore, each feature data may correspond to a different feature domain, and the same feature domain may include multiple features.

[0026] Optionally, according to the feature domain of the feature data, the distance between each feature data is obtained, and a first loss function is constructed according to the distance between each feature data. The distance between features can be the distance between at least two features in the same feature domain. The prediction model is trained based on the first loss function obtained based on the distance between features in the same feature domain, which helps to align multiple features in the same feature domain in the vector space. Alternatively, the distance between features can also be the distance between at least two features corresponding to different feature domains. The prediction model is trained based on the second loss function obtained based on the distance between different feature domains, which helps to evenly distribute multiple features between different feature domains.

[0027] Among them, the feature data is a set of feature representations obtained based on the input instance (user data and document data); the feature data after perturbation processing is a set of perturbation feature representations. Optionally, a random mask is introduced into the feature data through a perturbation function to achieve the effect of perturbation processing on the feature data. The encoder is used to capture and process the interaction between the feature data, and generate corresponding compressed feature representations. The third loss function is constructed based on the distance between the first compressed feature and the second compressed feature, and the feature representation can be directly optimized by the dependency relationship between the features, reducing the difference between different features and optimizing the feature representation.

[0028] Step S206, constructing a basic loss function for training the prediction model according to the predicted value of the click rate predicted by the prediction model based on the feature data and the true value of the feature data corresponding to the predicted value.

[0029] The prediction model is used to predict the click-through rate of the document, and the prediction model can select the existing click-through rate prediction model. Optionally, after vectorizing the feature data, the interaction information between the features corresponding to the user data and the features corresponding to the document data is modeled, and the model such as a multi-layer perceptron or a linear regression model uses the interaction information to predict the click-through rate of the document, and the third loss function of the prediction model is calculated based on the predicted result (predicted value) of the click-through rate and the true value of the click-through rate in the document data.

[0030] Step S208: adjusting parameters of the prediction model according to at least one optimized loss function and a basic loss function.

[0031] Among them, one or more functions among the first loss function, the second loss function, the third loss function and the basic loss function can be integrated to obtain an integrated loss function. When integrating the loss functions, the weights of each loss function can be adjusted through hyperparameters.

[0032] For ease of understanding, take the integration of the first loss function, the second loss function, the third loss function and the basic loss function as an example: Optionally, obtain preset hyperparameters, adjust the weights of the first loss function, the second loss function, the third loss function and the basic loss function according to the hyperparameters, and integrate the adjusted first loss function, the second loss function, the third loss function and the basic loss function, and iteratively update the difference in the prediction model until the integrated loss converges.

[0033] Step S210, predicting the click rate of the candidate document associated with the target user request based on the prediction model after adjusting the parameters. Optionally, when receiving the target user request, the candidate document associated with the target user request is obtained, and the prediction model outputs the click rate of the corresponding candidate document based on the target user request.

[0034] In the above document retrieval method, full attention is paid to data attributes and model attributes, and one or more of the first loss function, the second loss function and the third loss function are constructed based on the inherent characteristics and feature interaction information between the data. In the process of training the prediction model, the feature representation in the prediction model can be optimized based on one or more of the first loss function, the second loss function and the third loss function. The prediction model for click-through rate is optimized from the bottom layer through high-quality basic feature representation, thereby improving the accuracy of document click-through rate prediction. In addition, the above document retrieval method can be applied to existing click-through rate prediction models, and has high practicality and high compatibility.

[0035] In one embodiment, constructing a first loss function includes: determining at least two feature data corresponding to the same feature domain, calculating the first distance between the feature data in the same feature domain, and obtaining a first loss function. The first loss function aims to reduce the first distance. By reducing the first distance, the feature data corresponding to the first distance can be closer in the implicit space (vector space) of the feature, which is conducive to achieving feature alignment. Constructing a second loss function includes: determining at least two feature data corresponding to different feature domains, and obtaining a second loss function by calculating the second distance between at least two feature data corresponding to different feature domains. The second loss function aims to expand the second distance. The larger the second distance, the farther the feature data corresponding to the second distance is in the implicit space (vector space) of the feature, which is conducive to achieving uniform distribution of features.

[0036] Considering that the amount of calculation for calculating the distance between two feature data is large, the virtual feature nodes in the feature domain are calculated, and the distance between the feature data is obtained by combining the virtual feature nodes. In one embodiment, at least one optimization loss function is constructed, including: obtaining the feature domain corresponding to the feature data; calculating the virtual center node in the feature domain; constructing a first loss function according to the distance between the feature data in the same feature domain and the virtual center node of the feature domain; wherein the second loss function is used to maximize the distance between the feature data corresponding to different feature domains.

[0037] Among them, the virtual center node is a node artificially defined in the feature domain. Optionally, multiple feature data corresponding to the same feature domain are determined, and the virtual center node of the feature domain is calculated, and then the distance between other nodes (multiple feature data) in the feature domain and the virtual center node is calculated. In this embodiment, after the mean of the feature data passing through the same feature domain is calculated and the virtual center node is obtained, the distance between the feature node in the feature domain and the virtual center node is calculated, which greatly reduces the amount of calculation and improves the training efficiency compared to directly calculating the first distance between the feature data in the same feature domain.

[0038] In one embodiment, constructing at least one optimization loss function includes: obtaining a feature domain corresponding to feature data; calculating a virtual central node in the feature domain; and constructing a first loss function according to a first distance between virtual central nodes of multiple different feature domains.

[0039] Optionally, at least two feature data corresponding to different feature domains are determined, and virtual central nodes in different feature domains are calculated respectively, and then the distances between virtual central nodes in different feature domains are calculated. Optionally, a second distance between two different virtual central nodes is calculated, and a second loss function is constructed according to the second distance.

[0040] In this embodiment, the second distance is obtained based on the distance calculation between different virtual central nodes, which has a smaller amount of calculation and higher calculation efficiency than directly calculating the distance between features corresponding to different feature domains.

[0041] Further, in one embodiment, calculating the virtual central node in the feature domain includes: normalizing a plurality of feature data corresponding to the same feature domain; and obtaining the virtual central node according to the mean of the plurality of feature data. The normalization process can be implemented based on an existing normalization method, which is not limited here.

[0042] In one embodiment, at least one optimization loss function is constructed, including: obtaining a first encoder and a second encoder; wherein the first encoder and the second encoder have the same structure and different parameters; inputting feature data into the first encoder to obtain a first compressed feature; inputting the feature data after perturbation processing into the second encoder to obtain a second compressed feature; calculating a third distance between the first compressed feature and the second compressed feature; constructing a third loss function according to the third distance; wherein the third loss function is used to minimize the distance between the first compressed feature and the second compressed feature.

[0043] The first encoder and the second encoder are feature interaction encoders for capturing the interactive information between features. Optionally, the feature data is perturbed by a feature representation perturbation function, for example, The function masks part of the feature data; or, the disturbance processing can also be achieved by adding noise, replacing features, etc. The third loss function is intended to reduce the third distance between the first compressed feature and the second compressed feature. Optionally, a feature alignment loss function, i.e., the third loss function, is constructed according to the third distance.

[0044] In this embodiment, the encoder captures the dependencies between features, which can enable the prediction model to learn a more stable and generalized feature representation; a second loss function is constructed based on the first compressed features and the second compressed features. The second loss function can enable the prediction model to better summarize unseen data, enhance the information content of a single element in the feature representation, and optimize the compatibility and information density of the feature representation.

[0045] In one embodiment, a basic loss function for training a prediction model is constructed based on a predicted value of a click-through rate predicted by a prediction model based on feature data and a true value of feature data corresponding to the predicted value, including: performing feature conversion and dimensionality reduction processing on the feature data; inputting the feature data into a first encoder to obtain a first compressed feature; inputting the first compressed feature into the prediction model to obtain a predicted value of the click-through rate output by the prediction model based on the first compressed feature; and constructing a basic loss function based on the predicted value and the true value of the click-through rate corresponding to the predicted value in the feature data.

[0046] Among them, the first encoder is a feature interaction encoder used to receive the first compressed feature when constructing the third loss function. The first encoder can use an attention module, a deep neural network, a convolutional neural network, etc. Optionally, by mapping the feature data into a low-dimensional, continuous vector space, feature conversion processing and dimensionality reduction processing are performed on the feature data to obtain a dense vector that can be stored in a database. The dense vector is input to the first encoder to obtain the first compressed feature. Use a model with predictive ability such as a multi-layer perceptron and a linear regression model to predict the click-through rate of the document based on the first compressed feature, and calculate the cross entropy loss function of the prediction model based on the predicted click-through rate, that is, the basic loss function.

[0047] Feature data is often high-dimensional and may be sparse. In this embodiment, by performing feature conversion and dimensionality reduction processing on the feature data, a low-dimensional dense vector can be output, which helps the processed feature data complete the machine learning task.

[0048] In one embodiment, the click rate of candidate documents associated with the target user's request is predicted based on the prediction model after adjusting parameters, including: obtaining feature data optimized during the parameter adjustment process of the prediction model; obtaining candidate feature data corresponding to the candidate document in the optimized feature data; predicting the click rate of the candidate document based on the candidate feature data and the target user's request by the prediction model; and obtaining a retrieval list of documents based on the click rate of the candidate document.

[0049] Among them, the candidate document is related to the target user request. Optionally, the corresponding candidate document is predicted based on the description of the document in the target user request. The document data in the training data includes at least part of the data of the candidate document; after training the prediction model according to one or more optimized loss functions and the basic loss function among the first loss function, the second loss function, and the third loss function, the optimized feature representation (feature data) is stored. After receiving the target user request, the candidate feature data corresponding to the candidate document is selected in the optimized feature representation according to the candidate document. Optionally, the documents are sorted according to the click-through rate predicted by the click-through rate of the candidate document, and a search list of the documents is obtained, which is displayed to the user in the user interaction interface.

[0050] In this embodiment, when predicting the click-through rate of a document, recalling candidate feature data can reduce the time required for optimizing the features corresponding to the candidate document data, thereby greatly improving the prediction efficiency while ensuring the accuracy of the prediction.

[0051] In one embodiment, Figure 3 A click rate prediction method based on feature representation is proposed, including: Step S301, acquiring feature data of user data and document data. The user data includes data related to user registration and user behavior. Optionally, the user data is tabular data, including user data and behavior data, and the user data includes multiple feature fields; the document data is also tabular data, and the document data includes multiple feature fields. Optionally, after cleaning the user data and document data, feature data is extracted to improve the quality of feature acquisition.

[0052] Step S302, the user feature data and the document feature data are subjected to feature conversion and dimension reduction processing through the feature embedding layer to obtain feature representation. Optionally, the user and document features are subjected to sparse feature data dimension reduction through the feature embedding layer to obtain dense vectors, and the dense vectors processed by the feature embedding layer are stored in the vector database.

[0053] Step S303, optimizing the feature representation based on the user feature data, the document feature data and the features of the model to obtain a high-quality feature representation. Optionally, using contrastive learning technology to construct an unsupervised loss to optimize the feature representation.

[0054] Optionally, three additional contrastive learning-based losses are constructed, namely the first, second and third loss functions, which are used to directly optimize the feature representation of the model during training. Among them, the constructed unsupervised losses include: intra-domain feature alignment loss (first loss function), inter-domain feature uniformity loss (second loss function), and interactive feature alignment loss (third loss function).

[0055] Among them, when constructing the intra-domain feature alignment loss, it is necessary to first calculate the virtual center node of the same feature, and then calculate the distance from other nodes to this virtual center node. Optionally, when constructing the inter-domain feature uniform loss, the calculated distance is the Euclidean distance between the virtual center nodes; based on the calculated Euclidean distance, the loss is constructed. Among them, the intra-domain feature alignment loss and the inter-domain feature uniform loss correspond to the first loss function and the second loss function in the above-mentioned embodiment, respectively.

[0056] Among them, when constructing the interactive feature alignment loss, a feature representation based on the input instance (user feature data and document feature data) is obtained, a set of perturbation feature representations corresponding to the feature data is obtained through a perturbation function, and then a feature interaction encoder is used to generate a first compressed feature corresponding to the feature representation and a second compressed feature corresponding to the perturbation. The third distance between the first compressed feature representation and the perturbation second compressed feature representation is calculated and used as the loss to obtain the interactive feature alignment loss.

[0057] Step S304, combining the unsupervised loss constructed by contrastive learning technology with the traditional click rate prediction loss, offline training the click rate prediction model, and storing the feature representation obtained by the training in a database.

[0058] Among them, combining the unsupervised loss with the traditional click rate prediction loss can optimize the feature representation and enhance the quality of the basic model feature representation. Most of the existing click rate prediction models can be selected for joint training without changing the model structure. In the prediction process, only the basic model needs to be used to predict the probability of users clicking on the candidate document.

[0059] When there is a need for click-through rate prediction, the probability of users clicking on candidate documents can be calculated based on high-quality feature representation and trained click-through rate prediction model, and then ranked and displayed.

[0060] In this embodiment, by combining contrastive learning and a method for click-through rate prediction based on feature interaction, the feature representation in the basic click-through rate prediction model is fully optimized; high-quality basic feature representation can optimize the quality of the click-through rate prediction model from the bottom layer, and is applicable to various click-through rate prediction models. In addition, this embodiment improves the prediction accuracy of the document click-through rate without increasing the basic offline training cost and the complexity of online reasoning, providing users with more accurate services.

[0061] In one embodiment, based on Figure 3 The method shown provides a document search and recommendation system to significantly improve the accuracy and user experience of document search and recommendation. Figure 4 Schematic diagram of the literature recommendation system. Figure 4 As shown in the figure, the literature search recommendation system includes an offline training module, an online prediction module and a ranking display module. Figure 3 The click rate prediction method based on feature representation is shown, which can be referred to as AU4CTR. AU4CTR includes three sub-units: a basic prediction unit, an alignment-uniform unit, and an interactive alignment unit.

[0062] The basic prediction unit mainly includes a feature embedding layer, a feature interaction layer, and a prediction layer which are connected in sequence. The feature data of users and papers are obtained, and the user feature vector and paper feature vector are extracted.

[0063] The feature embedding layer is used to perform feature conversion and dimensionality reduction on the feature data. Optionally, the data processed by the feature embedding layer is processed using E=[v 1 ;v2;…;v F ]∈R F×d To represent the F feature domains in a single input instance, use E=[E 1 ,E 2,…,E F ]∈R M×d represents the representation of all M features, and E f represents the representation of all features in the Fth feature domain, and d is the dimension of each feature vector.

[0064] The feature interaction layer includes a feature interaction encoder. The feature interaction encoder is used to model the interaction information between user feature data and document feature data and obtain compressed feature representation. ,in represents the feature interaction encoder, and θ is the training parameter in the corresponding feature interaction encoder. In the feature interaction layer, you can select an existing feature interaction encoder, such as an attention module, a deep neural network, a convolutional neural network, etc.

[0065] The prediction layer is used to predict the click probability using a multi-layer perceptron or linear regression model: ; The cross entropy loss function based on the predicted click rate calculation model: ; Where N represents the number of training data sets, y i Represents the true value of the instance.

[0066] The alignment-uniformity unit and the interactive alignment unit are used to optimize feature representation. In the alignment-uniformity unit, two additional training losses are constructed based on the user and candidate document features: intra-domain uniformity loss and inter-domain alignment loss. The interactive alignment unit constructs an additional training loss based on the model structure: interactive feature alignment loss.

[0067] Among them, the alignment-uniformity unit mainly optimizes the basic feature representation based on the potential relationship of the features. For ease of understanding, the feature domain is explained: To optimize the basic feature representation, the representation of features between the same feature domains in the implicit space (vector space) should be as close as possible. For example, the two features "City A" and "City B" in the feature domain of "City" have similar attributes. In this feature domain, the features they contain should be as close as possible; similarly, "Brand A" and "Brand B" in the feature domain of "Brand" have similar features, so the features "Brand A" and "Brand B" contained in the feature domain of "Brand" should be as close as possible. Secondly, to optimize the basic feature representation, the representation of features in different feature domains in the implicit space needs to be as far away as possible. Taking the above-mentioned "City A" and "Brand A" as examples, "City A" and "Brand A" represent different features respectively, and their implicit attributes are also very different. Therefore, the feature domains of "City A" and "Brand A" need to be as far away as possible.

[0068] like Figure 5A schematic diagram of an alignment-uniformity unit is provided, Figure 5 The circles of different colors represent different feature domains, including feature domain E 1 , E 2 , E 3 ,……,E f Different points in the circle represent different features. In order to improve the efficiency of calculation, a virtual central node is defined for each feature domain. , specifically expressed as:

[0069] in represents the representation of the i-th feature in the f-th feature domain.

[0070] Based on the virtual center node, obtain the feature alignment loss , the calculation formula is as follows:

[0071] Where F is the total number of feature domains.

[0072] Based on the virtual center node, obtain the uniform loss of features , the calculation formula is as follows:

[0073]

[0074] Among them, K represents the number of pairwise calculations when calculating the distance, is the virtual center node of feature domain f1, It is the virtual central node of feature domain f2.

[0075] The following is a code that can realize the calculation of feature alignment loss and feature uniformity loss in the alignment-uniformity unit: def compute_alignment_loss(self, a=2): # Calculate feature alignment loss L_align = 0.0 embedds=self.embedding.embedding.weight V_K_Es= [] for start, end in zip(self.field dims cum[:-1], self.fielddims cum[1:]): embed_f=embedds[int(start):int(end) , :] embed_f =F.normalize(embed f, p=2, dim=1) V_K_E = embed f.mean(dim=0) V_K_Es.append(V_K_E) L_align += torch.norm(embed f.sub( V_K_E), p=2).pow(a).sum() L_align =L_align / self.sum_field V_K_Es = torch.stack(V_K_Es, 0) L_uni = self.compute uniformity loss(V_K_Es) return L_align, L_uni def compute_uniformity_loss(self, V_K_Es, t = 1): # Calculate feature uniform loss sq_dist =torch.pdist(V_K_Es, p=2).pow(2) return sq_dist.mul(-t).exp().mean().log() In the feature interaction alignment unit, the interaction alignment loss of the model is calculated: First, a feature representation perturbation function is used Perturb the input feature representation. Specifically, Function, indicating that the probability of each element being masked is p, and a set of perturbation feature representations are obtained :

[0076] The perturbed feature representation and the original feature representation are input into two feature interaction encoders with the same structure but different parameters. and , we get a set of compressed feature representations h, a set of compressed feature representations with perturbations :

[0077] Based on two sets of compressed feature representations h and Calculate the interaction alignment loss .

[0078]

[0079] Where the letter B represents the number of instances in a training batch; I is a random matrix that conforms to the Bernoulli distribution; Represents the Stop Gradient function, indicating that this part stops back propagation, mainly to ensure that the parameters in the feature interaction unit are optimized during the training of the click rate prediction model , does not affect the parameters of the feature interaction encoder in the basic interaction model In this way, the constructed loss function still only optimizes the basic feature representation.

[0080] The following is a calculation code that can realize the interaction alignment loss (InteractionAlignment) in the feature interaction alignment unit: def compute_interaction_alignment_loss(self, x): #(1) Generate a set of perturbed embedding representations through the Perturbation function x_emb = self.embedding(x) dp_emb = self.dp(x_emb) #(2) Perform feature interaction operations based on the same interaction structure fi_vector = self.feature_interaction(x_emb) fi_vector_dp = self.encoder_copy(dp_emb) # Using the stop gradient method fi_vector.detach_() #(3) Calculate Interaction Alignment loss cl_loss = torch.norm(fi_vector_dp.sub(fi_vector), dim=1).pow(2).mean() return cl_loss By optimizing the feature alignment loss through the feature interaction alignment unit, the information content of a single element in the feature representation can be enhanced and the compatibility and information density of the feature representation can be optimized.

[0081] During the training process, the model is trained based on the basic click rate prediction model and the above three losses. The optimized feature representation is saved through the vector database; the corresponding click rate prediction model is saved in the inference node. Figure 6 A schematic diagram of offline training module training is provided. The loss function of offline training module training can be expressed as follows:

[0082] in, and Represents the hyperparameter used to balance the basic click rate loss and each contrast loss. The value of the hyperparameter can be obtained based on experiments or modified according to needs.

[0083] The offline training module uses the feature representation optimization method based on contrastive learning to train the feature representation and click-through rate prediction model, stores the trained feature vectors in the database, and deploys the model to the inference node. The offline training module uses the feature representation optimization method based on contrastive learning, fully focusing on two properties of data and model structure: alignment and uniformity; by constructing three auxiliary unsupervised losses to optimize the quality of the basic feature representation, the prediction accuracy of the click-through rate prediction model is enhanced, and it has strong applicability.

[0084] The online prediction module includes a click-through rate prediction model. The online prediction module can predict the probability of a user clicking on a document based on the user request, the document feature vector, and the deployed click-through rate prediction model to obtain a paper recommendation list. Optionally, the click-through rate prediction model is deployed according to the training of the offline training module, and the feature data optimized after training is stored in the paper candidate pool. The click-through rate prediction model recalls the candidate features of the paper candidate pool according to the target user's request, and predicts the user's click probability on the candidate document through the trained click-through rate prediction model.

[0085] The sorting and display module is used to sort and display candidate documents according to the predicted probability. The system can provide users with more intelligent and personalized document recommendations. Optionally, the document search recommendation system in this embodiment can be applied to the field of astronomical document search. Figure 7 A schematic diagram of a sorting display module is provided; wherein the target user requests that Figure 7 Keywords, paper titles, galaxy names, etc. entered in.

[0086] This embodiment obtains user feature data and document feature data, and performs feature conversion and dimensionality reduction on the user feature data and document feature data through the feature embedding layer; performs additional training on the feature representation through the alignment-uniform module and the feature interaction alignment module; extracts the interaction information of the user and document features through the feature interaction encoder of the feature interaction layer to realize the prediction of click probability. The quality of feature representation is optimized based on the contrastive learning method, that is, the intra-domain alignment loss and the inter-domain uniformity loss are constructed; the feature representation is directly optimized through the dependency relationship between features, which improves the accuracy of click-through rate prediction. By analyzing the structure of the click-through rate prediction model based on feature interaction, the interactive alignment loss based on the model structure is constructed, and the direct optimization of the feature representation is realized. On the basis of improving the accuracy of click-through rate prediction, the quality of feature representation is further optimized, and the accuracy of click-through rate prediction is improved. In addition, the feature representation optimization method in this embodiment can be used as a basic plug-in to promote the effect of the basic click-through rate prediction model.

[0087] Furthermore, the document search recommendation system in this embodiment can be applied to the field of astronomical document search to solve the problem of inaccurate click rate prediction in traditional astronomical document recommendation systems.

[0088] Based on the same inventive concept, the embodiment of the present application also provides a document retrieval system for implementing the document retrieval method involved above. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more document retrieval system embodiments provided below can refer to the limitations on the document retrieval method above, and will not be repeated here.

[0089] In one embodiment, Figure 8 As shown, a document retrieval system is provided, including: a training module, a click rate prediction module and a ranking display module, wherein: A training module is used to extract feature data from training data, the training data includes user data and literature data; construct at least one optimization loss function, the optimization loss function includes: a first loss function constructed according to the distance between feature data with the same feature domain; a second loss function constructed according to the distance between feature data with different feature domains; inputting feature data into an encoder to obtain a first compressed feature, inputting perturbation-processed feature data into an encoder to obtain a second compressed feature, and constructing a third loss function according to the distance between the first compressed feature and the second compressed feature; constructing a basic loss function for training a prediction model according to a predicted value of a click-through rate predicted by a prediction model based on feature data and a true value of feature data corresponding to the predicted value; and adjusting parameters of the prediction model according to at least one optimization loss function and the basic loss function.

[0090] The click rate prediction module is used to predict the click rate of the candidate document associated with the target user's request based on the prediction model with adjusted parameters.

[0091] The sorting and display module is used to obtain a document search list according to the click rate of the candidate documents.

[0092] In the device embodiment, in addition to the embodiment of the device alone, it is also necessary to write out the embodiments of the device items that correspond one-to-one to the method items.

[0093] In one embodiment, the training module constructs at least one optimization loss function, including: obtaining the feature domain corresponding to the feature data; calculating the virtual center node in the feature domain; constructing the first loss function based on the distance between the feature data in the same feature domain and the virtual center node of the feature domain; wherein the first loss function is used to minimize the distance between the feature data corresponding to the same feature domain.

[0094] In one embodiment, the training module constructs at least one optimization loss function and may also include: obtaining the feature domain corresponding to the feature data; calculating the virtual center node in the feature domain; constructing the second loss function based on the distance between the virtual center nodes of multiple different feature domains; wherein the second loss function is used to maximize the distance between the feature data corresponding to different feature domains.

[0095] Optionally, calculating the virtual central node in the feature domain includes: normalizing a plurality of the feature data corresponding to the same feature domain; and obtaining the virtual central node according to a mean value of the plurality of the feature data.

[0096] In one embodiment, the training module constructs at least one optimization loss function and may also include: obtaining a first encoder and a second encoder; wherein the first encoder and the second encoder have the same structure and different parameters; inputting the feature data into the first encoder to obtain the first compressed feature; inputting the feature data after disturbance processing into the second encoder to obtain the second compressed feature; calculating the third distance between the first compressed feature and the second compressed feature; constructing the third loss function according to the third distance; wherein the third loss function is used to minimize the distance between the first compressed feature and the second compressed feature.

[0097] In one embodiment, the training module constructs a basic loss function for training the prediction model based on the predicted value of the click-through rate predicted by the prediction model based on the feature data and the true value of the feature data corresponding to the predicted value, including: performing feature conversion and dimensionality reduction processing on the feature data; inputting the feature data into a first encoder to obtain a first compressed feature; inputting the first compressed feature into the prediction model to obtain the predicted value of the click-through rate output by the prediction model based on the first compressed feature; and constructing a basic loss function based on the predicted value and the true value of the click-through rate corresponding to the predicted value in the feature data.

[0098] In one embodiment, a click rate prediction module predicts the click rate of a candidate document associated with a target user request based on the prediction model after adjusting parameters, including: obtaining the feature data optimized during the parameter adjustment process of the prediction model; obtaining candidate feature data corresponding to the candidate document from the optimized feature data; predicting the click rate of the candidate document by the prediction model based on the candidate feature data and the target user request; and obtaining a search list of the document based on the click rate of the candidate document.

[0099] Each module in the above document retrieval system can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0100] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface and the display unit are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a document retrieval method is realized. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, etc. The display screen can be a liquid crystal display screen or an electronic ink display screen.

[0101] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0102] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0103] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0104] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0105] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0106] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0107] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A document retrieval method, characterized in that: The method comprises: Extracting feature data from training data, wherein the training data includes user data and document data; Constructing at least one optimization loss function, the optimization loss function comprising: a first loss function constructed according to the distance between the feature data with the same feature domain; a second loss function constructed according to the distance between the feature data with different feature domains; inputting the feature data into an encoder to obtain a first compressed feature, inputting the feature data after perturbation processing into the encoder to obtain a second compressed feature, and constructing a third loss function according to the distance between the first compressed feature and the second compressed feature; Constructing a basic loss function for training the prediction model according to a predicted value of the click rate predicted by the prediction model based on the feature data and a true value of the feature data corresponding to the predicted value; Adjusting parameters of the prediction model according to at least one of the optimization loss function and the basic loss function; The click rate of the candidate document associated with the target user's request is predicted based on the prediction model after adjusting the parameters.

2. The method according to claim 1, characterized in that Construct at least one optimization loss function, including: Acquire a feature domain corresponding to the feature data; Calculating a virtual central node in the feature domain; The first loss function is constructed according to the distance between the feature data in the same feature domain and the virtual central node of the feature domain; wherein the first loss function is used to minimize the distance between the feature data corresponding to the same feature domain.

3. The method according to claim 1, characterized in that Construct at least one optimization loss function, including: Acquire a feature domain corresponding to the feature data; Calculating a virtual central node in the feature domain; The second loss function is constructed according to the distance between the virtual central nodes of multiple different feature domains; wherein the second loss function is used to maximize the distance between the feature data corresponding to different feature domains.

4. The method according to claim 2 or claim 3, characterized in that: Calculate the virtual central node in the feature domain, including: Normalizing a plurality of feature data corresponding to the same feature domain; The virtual central node is obtained according to the average of the plurality of characteristic data.

5. The method according to claim 1, characterized in that Construct at least one optimization loss function, including: Obtain a first encoder and a second encoder; wherein the first encoder and the second encoder have the same structure but different parameters; Inputting the feature data into the first encoder to obtain the first compression feature; Inputting the feature data after disturbance processing into the second encoder to obtain the second compressed feature; calculating a third distance between the first compressed feature and the second compressed feature; The third loss function is constructed according to the third distance; wherein the third loss function is used to minimize the distance between the first compressed feature and the second compressed feature.

6. The method according to claim 1, characterized in that According to the predicted value of the click rate predicted by the prediction model based on the feature data and the true value of the feature data corresponding to the predicted value, a basic loss function for training the prediction model is constructed, including: Performing feature conversion processing and dimensionality reduction processing on the feature data; Inputting the feature data into a first encoder to obtain a first compressed feature; Inputting the first compression feature into the prediction model to obtain a predicted value of the click rate output by the prediction model based on the first compression feature; The basic loss function is constructed based on the predicted value and the true value of the click rate corresponding to the predicted value in the feature data.

7. The method according to claim 1, characterized in that Predicting the click rate of the candidate document associated with the target user's request based on the prediction model after adjusting the parameters includes: Acquiring the feature data optimized during the parameter adjustment process of the prediction model; Acquire candidate feature data corresponding to the candidate document from the optimized feature data; The prediction model predicts the click rate of the candidate document according to the candidate feature data and the target user request; A search list of the document is obtained according to the click rate of the candidate document.

8. A document retrieval system, characterized in that: The system comprises: A training module is used to extract feature data from training data, wherein the training data includes user data and document data; construct at least one optimization loss function, wherein the optimization loss function includes: a first loss function constructed according to the distance between the feature data with the same feature domain; a second loss function constructed according to the distance between the feature data with different feature domains; inputting the feature data into an encoder to obtain a first compressed feature, inputting the feature data after perturbation processing into the encoder to obtain a second compressed feature, and constructing a third loss function according to the distance between the first compressed feature and the second compressed feature; constructing a basic loss function for training a prediction model according to a predicted value of a click-through rate predicted by a prediction model based on the feature data and a true value of the feature data corresponding to the predicted value; and adjusting the parameters of the prediction model according to at least one of the optimization loss function and the basic loss function; A click rate prediction module, used to predict the click rate of the candidate document associated with the target user's request based on the prediction model after adjusting the parameters; The sorting and display module is used to obtain a document search list according to the click rate of the candidate documents.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images