Offline strategy evaluation method and system suitable for social network, medium and equipment
By introducing network mixed inverse probability weighting methods and graph convolutional networks within the inverse probability weighting framework, the evaluation accuracy problem caused by network mixed effects in social networks is solved, and a more accurate evaluation of new recommendation strategies is achieved.
Patent Information
- Application Number
- CN202411158770.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-08-22
AI Technical Summary
Existing offline strategy evaluation methods ignore network confounding effects in social networks, resulting in a decrease in accuracy when evaluating new recommendation strategies in the presence of social network data.
Introducing the network mixed inverse probability weighting (IntIPW) method, by considering the impact of processing variables of social network data on neighbors within the inverse probability weighting framework, using graph convolutional network to train offline strategy evaluation model, calculate the weight of social network data and perform self-direction Normalize the evaluation of the value of the strategy.
Improved the accuracy of the evaluation of new recommendation strategies when there is network confounding effect, and more accurately assesses the effectiveness of new recommendation strategies from social network datasets.
Smart Images

Figure CN119089053A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of data mining, and in particular relates to an offline strategy evaluation method, system, medium and device suitable for social networks. Background Art
[0002] The contextual multi-armed bandit process is critical in many real-world applications, including precision medicine, recommendation systems, and advertising. In these interactive systems, the old recommendation strategy recommends content (behavioral treatment variables) based on user characteristics (covariates) and collects results (outcome variables), generating a large amount of logs. These logs generated by online interactions provide key data for offline strategy evaluation. Offline strategy evaluation aims to evaluate the performance of new recommendation strategies without relying on A / B testing, thereby alleviating high costs and potential ethical issues.
[0003] As the interest in the offline policy evaluation problem grows, many estimators have been developed to evaluate new policies using offline log data. Direct methods predict the outcome function based on log data. Inverse probability weighting assigns sample weights based on the probabilities of the old and new policies. Doubly robust methods combine direct methods and inverse probability weighting. These estimators have shown effectiveness in various fields. The effectiveness of these methods relies on the individual treatment stability assumption (SUTVA), which assumes that the outcome of any given sample is only affected by its covariates and its treatment variable. However, in many real scenarios involving social networks, this assumption often does not hold because samples are not independent.
[0004] In social networks, confounding occurs when the outcome of a sample is affected not only by its own covariates and treatment variables, but also by the treatment variables of its social neighbors, which is also known as network confounding. For example, in epidemiology, deciding whether to vaccinate someone may affect the health of people around them; in recommendation systems, recommending a product to someone may also indirectly affect the purchase intentions of others in their social circle through opinion propagation; in advertising, ad exposure may directly affect users' purchasing behavior and indirectly affect others in their social network through their acquisition behavior. However, current research on offline strategy evaluation ignores network confounding effects, which may cause these estimators to fail completely. Therefore, how to evaluate new recommendation strategies based solely on marketing datasets in the presence of confounding effects in social networks remains an open problem. Summary of the invention
[0005] The purpose of the present invention is to solve the problem of evaluating new recommendation strategies from marketing data under the network confounding effect in the prior art. Specifically, when a processing variable of social network data affects the outcome variable of its neighbors, how to use the marketing data set to more accurately estimate the effectiveness of the new recommendation strategy. In addition, the present invention provides an offline strategy evaluation method, system, medium and device suitable for social networks. In the method of the present invention, the network confounding inverse probability weighting (IntIPW) is introduced, which cleverly integrates the importance weights and comprehensively considers the impact of the processing variables of social network data on neighbors, thereby proposing a novel method within the inverse probability weighting framework, which effectively fills the existing gap.
[0006] In order to achieve the above-mentioned invention object, the present invention specifically adopts the following technical solutions:
[0007] In a first aspect, the present invention provides an offline strategy evaluation method applicable to a social network, and the specific steps are as follows:
[0008] S1. Based on the keywords, the social network data is divided into covariates, behavior processing variables, adjacency matrix and outcome variables according to the user's characteristic information, the content recommended to the user by the old content recommendation strategy, the friend relationship information between users and the user's preferences, and the social network data set is constructed using the divided social network data, and the content recommended to the user by the new content recommendation strategy is used as the evaluation processing variable;
[0009] S2. Training an offline policy evaluation model on a social network dataset and an evaluation processing variable, wherein the offline policy evaluation model comprises an embedding function and a binary classifier, wherein the binary classifier comprises a graph convolutional network;
[0010] In the offline strategy evaluation model, first, a learnable embedding function is used to map the behavior processing variable and the evaluation processing variable to the semantic space respectively, and a behavior processing embedding with a dimension consistent with the covariate is obtained for the behavior processing variable, and an evaluation processing embedding with a dimension consistent with the covariate is obtained for the evaluation processing variable, and the covariate and the behavior processing embedding of each marketing data are connected to form a behavior joint representation, and the covariate and the evaluation processing embedding of each marketing data are connected to form an evaluation joint representation, and corresponding labels are set for the behavior joint representation and the evaluation joint representation, and an adjacency matrix containing self-connection is formed by adding the adjacency matrix and the unit matrix, and the adjacency matrix containing self-connection, the evaluation joint representation and the behavior joint representation are input into the binary classifier, and the graph convolution network processes the adjacency matrix containing self-connection and the behavior joint representation to obtain a first output vector corresponding to the behavior joint representation, and the graph convolution network processes the adjacency matrix containing self-connection and the evaluation joint representation to obtain a second output vector corresponding to the evaluation joint representation;
[0011] S3. When the offline strategy evaluation model training is completed, the weight of the social network data result variable is calculated by the updated first output vector, the weight of the social network data result variable is weighted and summed with the corresponding social network data result variable to obtain the evaluation strategy value, the weights of the social network data result variables are added to obtain the normalized weight, and the evaluation strategy value is divided by the normalized weight to obtain the self-normalized evaluation strategy value, which is used to determine whether to adopt the new content recommendation strategy.
[0012] Based on the above scheme, each step can be implemented in the following preferred specific manner.
[0013] As a preferred embodiment of the first aspect, in step S2, the label of the behavior joint representation is set to 0, and the label of the evaluation joint representation is set to 1.
[0014] As a preferred embodiment of the first aspect, in step S2, the offline strategy evaluation model adopts binary cross entropy loss during training.
[0015] As a preferred embodiment of the first aspect, in step S3, the result variable r of the i-th social network data is calculated by using the updated first output vector. i The weight w i for:
[0016]
[0017] Among them, f 0i represents the i-th value in the first output vector f0.
[0018] As a preferred embodiment of the first aspect, in step S3, the self-normalized evaluation strategy value is calculated as follows:
[0019]
[0020] Among them, r i represents the outcome variable of the i-th social network data; N represents the number of social network data.
[0021] As a preferred embodiment of the above-mentioned first aspect, in step S3, the specific process of judging whether to adopt the new content recommendation strategy based on the self-normalized evaluation strategy value is: averaging all the result variables in the social network data set to obtain the value of the old content recommendation strategy, comparing the value of the old content recommendation strategy with the self-normalized evaluation strategy value; if the value of the old content recommendation strategy is less than the self-normalized evaluation strategy value, retaining the old content recommendation strategy; if the value of the old content recommendation strategy is greater than or equal to the self-normalized evaluation strategy value, replacing the old content recommendation strategy with the new content recommendation strategy.
[0022] In a second aspect, the present invention provides an offline strategy evaluation system applicable to social networks, comprising:
[0023] The data processing module divides the social network data into covariates, behavior processing variables, adjacency matrix and result variables according to the keywords, the user's characteristic information, the content recommended to the user by the old content recommendation strategy, the friend relationship information between users and the user's preferences, and constructs a social network data set using the divided social network data, and uses the content recommended to the user by the new content recommendation strategy as the evaluation processing variable;
[0024] A model optimization module, used for training an offline policy evaluation model on a social network dataset and an evaluation processing variable, wherein the offline policy evaluation model comprises an embedding function and a binary classifier, wherein the binary classifier comprises a graph convolutional network;
[0025] In the offline strategy evaluation model, first, a learnable embedding function is used to map the behavior processing variable and the evaluation processing variable to the semantic space respectively, and a behavior processing embedding with a dimension consistent with the covariate is obtained for the behavior processing variable, and an evaluation processing embedding with a dimension consistent with the covariate is obtained for the evaluation processing variable, and the covariate and the behavior processing embedding of each marketing data are connected to form a behavior joint representation, and the covariate and the evaluation processing embedding of each marketing data are connected to form an evaluation joint representation, and corresponding labels are set for the behavior joint representation and the evaluation joint representation, and an adjacency matrix containing self-connection is formed by adding the adjacency matrix and the unit matrix, and the adjacency matrix containing self-connection, the evaluation joint representation and the behavior joint representation are input into the binary classifier, and the graph convolution network processes the adjacency matrix containing self-connection and the behavior joint representation to obtain a first output vector corresponding to the behavior joint representation, and the graph convolution network processes the adjacency matrix containing self-connection and the evaluation joint representation to obtain a second output vector corresponding to the evaluation joint representation;
[0026] The result acquisition module is used to calculate the weight of the social network data result variable from the updated first output vector after the offline strategy evaluation model training is completed, and to perform weighted summation of the weight of the social network data result variable and the corresponding social network data result variable to obtain the evaluation strategy value, add the weights of the social network data result variables to obtain the normalized weight, and divide the evaluation strategy value by the normalized weight to obtain the self-normalized evaluation strategy value, which is used to determine whether to adopt the new content recommendation strategy.
[0027] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, an offline strategy evaluation method applicable to a social network as described in any of the solutions in the first aspect is implemented.
[0028] In a fourth aspect, the present invention provides a computer electronic device, characterized in that it includes a memory and a processor;
[0029] The memory is used to store computer programs;
[0030] The processor is used to implement an offline strategy evaluation method applicable to a social network as described in any solution of the first aspect when executing the computer program.
[0031] In a fifth aspect, the present invention provides a computer program product, including a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, it can implement an offline strategy evaluation method applicable to a social network as described in any scheme of the first aspect above.
[0032] Compared with the prior art, the present invention has the following beneficial effects:
[0033] Compared with the traditional offline strategy evaluation method, the present invention considers the influence of neighbors on each social network data through network confounding inverse probability weighting, thereby improving the accuracy of new recommendation strategy evaluation when there is a network confounding effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 Schematic diagram of the steps of the method of the present invention;
[0035] Figure 2 A flowchart of the offline strategy evaluation model training process of the present invention;
[0036] Figure 3 It is a schematic diagram of a situation in which a network promiscuity effect exists in a social network according to an embodiment of the present invention;
[0037] Figure 4 It is a system block diagram of the present invention. DETAILED DESCRIPTION
[0038] In order to make the above-mentioned purpose, features and advantages of the present invention more obvious and easy to understand, the specific implementation mode of the present invention is described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The technical features in each embodiment of the present invention can be combined accordingly without conflicting with each other.
[0039] In the description of the present invention, it should be understood that the terms "first" and "second" are only used for the purpose of distinguishing descriptions, and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features.
[0040] The purpose of the present invention is to solve the problem of evaluating new recommendation strategies from social network data in the presence of network confounding effects. Specifically, when a processing variable of social network data affects the outcome variable of its neighbors, how to use the social network data set to more accurately estimate the effectiveness of the new recommendation strategy. This embodiment gives an example of a small social network with network confounding effects, such as Figure 3 shown. Figure 3 In the equation, x, a, and y represent the covariate, treatment variable, and outcome variable of social network data, respectively. Figure 3 The left side of the dotted line (such as Figure 3 The left half) is a causal diagram under the common individual treatment stability assumption. Figure 3 The right side of the dotted line (such as Figure 3 The right half is the network connection topology and causal diagram under the setting. For the sake of simplicity, the result variable y1 of user 1 and the result variable y3 of user 3 are not shown. Figure 3 As shown in the left part, previous studies mainly follow the individual treatment stability hypothesis (SUTVA), that is, the potential outcome variable of each social network data will not be affected by the treatment variables of other social network data. However, in real-world situations, such as Figure 3 As shown in the right half of , there is usually a social network in which people with social connections influence each other, thus violating the individual treatment stability assumption. Specifically, the outcome variables of each social network data are affected not only by their own treatment variables and covariates, but also by the treatment variables of other social network data in their social connections.
[0041] like Figure 1 As shown, in a preferred implementation of the present invention, in order to solve the problem of evaluating new strategies from marketing data in the presence of network confusion, the present invention provides a novel offline strategy evaluation method applicable to social networks, and the offline strategy evaluation method applicable to social networks includes the following steps S1 to S3. The specific implementation process is described in detail below.
[0042] S1. Based on keywords, the social network data is divided into covariates, behavior processing variables, adjacency matrix and outcome variables according to the user's characteristic information, the content recommended to the user by the old content recommendation strategy, the friend relationship information between users and the user's preferences. The divided social network data is used to construct a social network dataset, and the content recommended to the user by the new content recommendation strategy is used as the evaluation processing variable.
[0043] It should be noted that in this embodiment, a social scenario is that a social network recommends content to a user. The above covariates are the user's characteristic information, such as gender, age, etc., recorded as an N×p-dimensional matrix x, where N is the number of users (i.e., the number of social network data) and p is the dimension of the user's characteristic information. The behavior processing variable is the content recommended to the user by the old strategy, recorded as an N-dimensional vector a0. The adjacency matrix is the friend relationship information between users, recorded as an N×N-dimensional matrix A. If user i and user j are friends, then A ij =A ji =1, otherwise A ij =A ji =0, where A ij , A ji They represent the elements in the i-th row and j-th column of the adjacency matrix, and the elements in the j-th row and i-th column of the adjacency matrix, respectively. The result variable is the user preference, recorded as an N-dimensional vector r. The evaluation processing variable is the content recommended to the user by the new strategy, recorded as an N-dimensional vector a1.
[0044] S2. Training an offline policy evaluation model on a social network dataset and an evaluation processing variable, wherein the offline policy evaluation model comprises an embedding function and a binary classifier, wherein the binary classifier comprises a graph convolutional network;
[0045] In the offline strategy evaluation model, first, a learnable embedding function is used to map the behavior processing variable and the evaluation processing variable to the semantic space respectively, and the behavior processing variable corresponds to a behavior processing embedding with a dimension consistent with the covariate, and the evaluation processing variable corresponds to an evaluation processing embedding with a dimension consistent with the covariate, and the covariate and the behavior processing embedding of each marketing data are connected to form a behavior joint representation, and the covariate and the evaluation processing embedding of each marketing data are connected to form an evaluation joint representation, and corresponding labels are set for the behavior joint representation and the evaluation joint representation, and an adjacency matrix containing self-connection is formed by adding the adjacency matrix and the unit matrix, and the adjacency matrix containing self-connection, the evaluation joint representation and the behavior joint representation are input into the binary classifier, and the graph convolution network processes the adjacency matrix containing self-connection and the behavior joint representation to obtain a first output vector corresponding to the behavior joint representation, and the graph convolution network processes the adjacency matrix containing self-connection and the evaluation joint representation to obtain a second output vector corresponding to the evaluation joint representation.
[0046] It should be noted that in this embodiment, when training the offline strategy evaluation model, Figure 2 As shown in , we first learn an embedding expression with a dimension consistent with the covariate for each value of the behavior treatment variable and the evaluation treatment variable. Specifically, we use the embedding function e φ Project the two processing variables into the semantic space and embed the function e φ The discrete treatment variable is converted into a p-dimensional vector, which has the same dimension as the covariate, and is called a treatment embedding. Therefore, the behavior treatment variable is matched with a behavior treatment embedding e with the same dimension as the covariate. φ (a0), the evaluation treatment variable corresponds to an evaluation treatment embedding e with the same dimension as the covariate φ (a1). Where φ is the embedding function e φ is a learnable parameter.
[0047] Then, the covariate of each social network data and the two processing embeddings obtained in S21 are connected respectively, which is called a joint representation. Specifically, the covariate of each social network data and the behavior processing embedding are connected to form a behavior joint representation h0 = [x, e φ (a0)], the covariates and evaluation processing of each social network data are embedded and connected to form the evaluation joint representation h1 = [x, e φ (a1)].
[0048] Next, a probability is learned through the graph convolution network of the binary classifier to distinguish the two joint representations, that is, to determine whether the joint representation of each social network data belongs to the behavior joint representation or the evaluation joint representation. Specifically, the classifier uses a graph convolution network (GCN) to aggregate the information of adjacent social network data. In addition, considering that the graph convolution network pays attention to the behavior of the neighbors of the social network data and the behavior of the sample itself at the same time, the present invention is composed of the adjacency matrix A plus the identity matrix I n Construct an adjacency matrix including self-connection, that is: After obtaining the adjacency matrix containing self-connection, the graph convolutional network processes the adjacency matrix containing self-connection and the behavior joint representation to obtain the first output vector corresponding to the behavior joint representation This vector represents the output vector corresponding to the behavior joint representation, that is, the true label is the behavior joint representation and it is hoped that the result of graph convolutional network learning is also a label close to the behavior joint representation; the graph convolutional network processes the adjacency matrix containing self-connections and the evaluation joint representation to obtain the second output vector corresponding to the evaluation joint representation This vector represents the output vector corresponding to the evaluation joint representation, that is, the true label is the evaluation joint representation and the result of graph convolutional network learning is also a label close to the evaluation joint representation. Therefore, the output vector f∈[0,1] can represent the probability that the graph convolutional network believes that the social network data belongs to the evaluation joint representation. Conversely, the probability that the joint representation belongs to the behavior joint representation is 1-f. Among them, θ is the graph convolutional network g θ is a learnable parameter of .
[0049] It should be noted that, in this embodiment, in order to optimize the offline policy evaluation model, before the end of each iteration of the training process, the embedding function e in the offline policy evaluation model is jointly optimized by minimizing the binary cross-entropy loss between the first output vector f0, the second output vector f1 and the true label of the social network data. φ and graph convolutional network g θ In addition, in order to facilitate the subsequent classification task, the present invention sets the label of the behavior joint representation h0 to 0 and the label of the evaluation joint representation h1 to 1. When the label of the behavior joint representation is set to 0 and the label of the evaluation joint representation is set to 1, the binary cross entropy loss The mathematical expression can be simplified to:
[0050]
[0051] Among them, f 0i represents the i-th value in the first output vector f0; f 1i Represents the i-th value in the second output vector f1.
[0052] S3. When the offline strategy evaluation model training is completed, the weight of the social network data result variable is calculated by the updated first output vector, the weight of the social network data result variable is weighted and summed with the corresponding social network data result variable to obtain the evaluation strategy value, the weights of the social network data result variables are added to obtain the normalized weight, and the evaluation strategy value is divided by the normalized weight to obtain the self-normalized evaluation strategy value, which is used to determine whether to adopt the new content recommendation strategy.
[0053] It should be noted that, in the present invention, a weight is assigned to each result variable in the social network data set according to the result given by the binary classifier. The final self-normalized evaluation strategy value (the evaluation result of the new content recommendation strategy) is the average value of the weighted result variable, and whether to adopt the new content recommendation strategy can be decided based on this evaluation result.
[0054] Specifically, training stops when the offline policy evaluation model converges. Based on the probability given by the binary classifier, a weight is assigned to the outcome variable r in the social network dataset. The outcome variable r of the i-th social network data i The weight w i for:
[0055]
[0056] While maintaining consistency, the present invention reduces variance by self-normalizing weights. Finally, the new content recommendation strategy can be evaluated in the presence of network confounding effects, and the self-normalized evaluation strategy value can be obtained. And decide whether to adopt a new content recommendation strategy based on the evaluation results:
[0057]
[0058] Among them, r i Represents the outcome variable of the i-th social network data.
[0059] It should be noted that in step S3 of the present invention, the specific process of judging whether to adopt the new content recommendation strategy based on the self-normalized evaluation strategy value is: averaging all the result variables in the social network data set to obtain the value of the old content recommendation strategy, comparing the value of the old content recommendation strategy with the self-normalized evaluation strategy value; if the value of the old content recommendation strategy is less than the self-normalized evaluation strategy value, retaining the old content recommendation strategy; if the value of the old content recommendation strategy is greater than or equal to the self-normalized evaluation strategy value, replacing the old content recommendation strategy with the new content recommendation strategy.
[0060] The present invention will now use a specific example to demonstrate the application effect of the offline strategy evaluation method applicable to social networks described in S1 to S3 in the above embodiments on a specific data set, so as to facilitate understanding of the essence of the present invention.
[0061] Example
[0062] The specific implementation process of the offline strategy evaluation method applicable to social networks adopted in this embodiment is as described above and will not be repeated here.
[0063] This embodiment obtains two social network data sets on a public social platform, namely a blog social (BlogCatalog) data set and a picture social (Flickr) data set. The above two social network data sets are introduced below.
[0064] The BlogCatalog dataset is a dataset that has been used for network analysis and social media research. It includes data about bloggers and their connections, organized in a way that represents the social network between them. The BlogCatalog dataset used in this embodiment contains 5196 social network data and 171743 edges connecting them. The covariate of each social network data is a 2246-dimensional vector.
[0065] The picture social (Flickr) dataset comes from a web-based platform for storing, organizing and sharing digital photos. This dataset is a collection of data extracted from social networks, focusing on the connections and interactions between users. The picture social dataset used in this embodiment includes social network data representing some users and the edges of the network representing the friend relationships between them. It contains 7575 social network data and 239738 edges connecting them. The covariate of each social network data is a 1205-dimensional vector.
[0066] Because the feature dimensions of the samples in these two data sets are too high, this embodiment uses the principal component analysis method to reduce the feature dimensions to 10 dimensions before conducting experiments.
[0067] In order to objectively evaluate the performance of this algorithm, this embodiment randomly generates 20 different result functions for the BlogCatalog dataset and the Flickr dataset, and trains the offline strategy evaluation model under 5 selection bias intensities. The selection bias controls an exploration-exploitation trade-off in the old recommendation strategy. A larger selection bias value means that the social network data has a greater probability of selecting the covariate that is best for its own results. Finally, the mean and standard deviation (mean±std) of the mean square error (MSE) of the 20 evaluation results of the offline strategy evaluation model in the BlogCatalog experiment and the Flickr experiment are calculated. The calculation formula for the mean square error is as follows:
[0068]
[0069] Among them, V(π) is the true result of the new recommendation strategy, Evaluation results for new recommended strategies; Indicates expectation,
[0070] The experimental results are shown in Table 1 and Table 2. The results show that the method of the present invention significantly improves the accuracy of estimating the new recommendation strategy. "IntIPW" in Table 1 and Table 2 represents the method of the present invention.
[0071] Table 1. MSE error (mean ± std) of evaluation results on the BlogCatalog dataset
[0072]
[0073] Table 2. MSE error (mean ± std) of evaluation results on the Flickr dataset
[0074]
[0075]
[0076] Among them, DM is a direct estimation method, which comes from the prior art literature: Alina Beygelzimer and John Langford. 2009. The offset tree for learning with partial labels. In Proceedings of the 15th ACM SIGKDD, 129–138; IPW is an inverse probability weighting method, which comes from the prior art literature: Alexander L. Strehl, John Langford, Lihong Li, and Sham M. Kakade. 2010. Learning from Logged Implicit Exploration Data. In Neural Information Processing Systems; BIPW is a balanced inverse probability weighting method, which comes from the prior art literature: Arjun Sondhi, David Arbour, and Drew Dimmery. 2020. Balanced Off-Policy Evaluation in General Action Spaces. In Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistic; SGIPW is a Sub-Gaussian Inverse Probability Weighting method, which comes from the prior art literature: AlbertoMaria Metelli.2021.Subgaussian and Differentiable Importance Sampling for Off-Policy Evaluation and Learning.In Neural Information Processing Systems; DR is a Doubly Robust method, which comes from the prior art literature: Yi Su, Lequn Wang, Michele Santacatterina, and Thorsten Joachims.2019.CAB: Continuous Adaptive Blending for Policy Evaluation and Learning. In Proceedings of the 36th International Conference on Machine Learning. PMLR, 6005–6014. SGDR is a Sub-Gaussian Doubly Robust method, which comes from the existing technical literature: Alberto Maria Metelli. 2021. Subgaussian and Differentiable Importance Sampling for Off-Policy Evaluation and Learning. In Neural Information Processing Systems. .
[0077] It should also be noted that the offline policy evaluation method applicable to social networks in the above embodiment can essentially be executed by a computer program or module. Therefore, similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides an offline policy evaluation system applicable to social networks corresponding to the offline policy evaluation method applicable to social networks provided in the above embodiment, such as Figure 4 As shown, it includes:
[0078] The data processing module divides the social network data into covariates, behavior processing variables, adjacency matrix and result variables according to the keywords, the user's characteristic information, the content recommended to the user by the old content recommendation strategy, the friend relationship information between users and the user's preferences, and constructs a social network data set using the divided social network data, and uses the content recommended to the user by the new content recommendation strategy as the evaluation processing variable;
[0079] A model optimization module, used for training an offline policy evaluation model on a social network dataset and an evaluation processing variable, wherein the offline policy evaluation model comprises an embedding function and a binary classifier, wherein the binary classifier comprises a graph convolutional network;
[0080] In the offline strategy evaluation model, first, a learnable embedding function is used to map the behavior processing variable and the evaluation processing variable to the semantic space respectively, and a behavior processing embedding with a dimension consistent with the covariate is obtained for the behavior processing variable, and an evaluation processing embedding with a dimension consistent with the covariate is obtained for the evaluation processing variable, and the covariate and the behavior processing embedding of each marketing data are connected to form a behavior joint representation, and the covariate and the evaluation processing embedding of each marketing data are connected to form an evaluation joint representation, and corresponding labels are set for the behavior joint representation and the evaluation joint representation, and an adjacency matrix containing self-connection is formed by adding the adjacency matrix and the unit matrix, and the adjacency matrix containing self-connection, the evaluation joint representation and the behavior joint representation are input into the binary classifier, and the graph convolution network processes the adjacency matrix containing self-connection and the behavior joint representation to obtain a first output vector corresponding to the behavior joint representation, and the graph convolution network processes the adjacency matrix containing self-connection and the evaluation joint representation to obtain a second output vector corresponding to the evaluation joint representation;
[0081] The result acquisition module is used to calculate the weight of the social network data result variable from the updated first output vector after the offline strategy evaluation model training is completed, and to perform weighted summation of the weight of the social network data result variable and the corresponding social network data result variable to obtain the evaluation strategy value, add the weights of the social network data result variables to obtain the normalized weight, and divide the evaluation strategy value by the normalized weight to obtain the self-normalized evaluation strategy value, which is used to determine whether to adopt the new content recommendation strategy.
[0082] It is understandable that the offline policy evaluation method applicable to social networks described in S1 to S3 above can be substantially implemented by a computer program. Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer program product corresponding to the offline policy evaluation method applicable to social networks provided in the above embodiment, which includes a computer program / instruction. When the computer program / instruction is executed by a processor, it can implement the offline policy evaluation method applicable to social networks described in the above embodiment.
[0083] Similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer electronic device corresponding to the offline strategy evaluation method for social networks provided in the above embodiment, which includes a memory and a processor;
[0084] The memory is used to store computer programs;
[0085] The processor is used to implement an offline strategy evaluation method applicable to a social network in the above embodiment when executing the computer program.
[0086] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.
[0087] Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer-readable storage medium corresponding to the offline policy evaluation method applicable to a social network provided in the above embodiment, and a computer program is stored on the storage medium. When the computer program is executed by the processor, it can implement the offline policy evaluation method applicable to a social network in the above embodiment.
[0088] It is understandable that the above storage medium may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. The storage medium may also be a U disk, a mobile hard disk, a magnetic disk or an optical disk, etc., which can store program codes.
[0089] It is understandable that the above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0090] It should also be noted that those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the various embodiments provided in this application, the division of steps or modules in the system and method is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or steps can be combined or integrated together, and a module or step can also be split.
[0091] The above-described embodiment is only a preferred solution of the present invention, but it is not intended to limit the present invention. A person skilled in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.
Claims
1. A method for offline strategy evaluation in social networks. It is characterized in that The following steps are involved: S1. Based on the keywords, the social network data is divided into covariates, behavior processing variables, adjacency matrix and outcome variables according to the user's characteristic information, the content recommended to the user by the old content recommendation strategy, the friend relationship information between users and the user's preferences, and the social network data set is constructed using the divided social network data, and the content recommended to the user by the new content recommendation strategy is used as the evaluation processing variable; S2. Training an offline policy evaluation model on a social network dataset and an evaluation processing variable, wherein the offline policy evaluation model comprises an embedding function and a binary classifier, wherein the binary classifier comprises a graph convolutional network; In the offline strategy evaluation model, first, a learnable embedding function is used to map the behavior processing variable and the evaluation processing variable to the semantic space respectively, and a behavior processing embedding with a dimension consistent with the covariate is obtained for the behavior processing variable, and an evaluation processing embedding with a dimension consistent with the covariate is obtained for the evaluation processing variable, and the covariate and the behavior processing embedding of each marketing data are connected to form a behavior joint representation, and the covariate and the evaluation processing embedding of each marketing data are connected to form an evaluation joint representation, and corresponding labels are set for the behavior joint representation and the evaluation joint representation, and an adjacency matrix containing self-connection is formed by adding the adjacency matrix and the unit matrix, and the adjacency matrix containing self-connection, the evaluation joint representation and the behavior joint representation are input into the binary classifier, and the graph convolution network processes the adjacency matrix containing self-connection and the behavior joint representation to obtain a first output vector corresponding to the behavior joint representation, and the graph convolution network processes the adjacency matrix containing self-connection and the evaluation joint representation to obtain a second output vector corresponding to the evaluation joint representation; S3. When the offline strategy evaluation model training is completed, the weight of the social network data result variable is calculated by the updated first output vector, the weight of the social network data result variable is weighted and summed with the corresponding social network data result variable to obtain the evaluation strategy value, the weights of the social network data result variables are added to obtain the normalized weight, and the evaluation strategy value is divided by the normalized weight to obtain the self-normalized evaluation strategy value, which is used to determine whether to adopt the new content recommendation strategy.
2. The offline strategy evaluation method applicable to social networks as claimed in claim 1, It is characterized in that In step S2, the label of the behavior joint representation is set to 0, and the label of the evaluation joint representation is set to 1.
3. The offline strategy evaluation method applicable to social networks as claimed in claim 1, It is characterized in that In step S2, the offline policy evaluation model adopts binary cross entropy loss during training.
4. The offline strategy evaluation method applicable to social networks as claimed in claim 1, It is characterized in that In step S3, the result variable r of the i-th social network data is calculated from the updated first output vector i The weight w i for: Among them, f 0i represents the first output vector f 0 The i-th value in .
5. The offline strategy evaluation method applicable to social networks as claimed in claim 4, It is characterized in that In step S3, the self-normalized evaluation strategy value is calculated as follows: Among them, r i represents the outcome variable of the i-th social network data; N represents the number of social network data.
6. The offline strategy evaluation method applicable to social networks as claimed in claim 1, It is characterized in that In step S3, the specific process of judging whether to adopt the new content recommendation strategy based on the self-normalized evaluation strategy value is as follows: averaging all the result variables in the social network data set to obtain the value of the old content recommendation strategy, comparing the value of the old content recommendation strategy with the self-normalized evaluation strategy value; if the value of the old content recommendation strategy is less than the self-normalized evaluation strategy value, retain the old content recommendation strategy; if the value of the old content recommendation strategy is greater than or equal to the self-normalized evaluation strategy value, replace the old content recommendation strategy with the new content recommendation strategy.
7. An offline strategy evaluation system for social networks, It is characterized in that include: The data processing module divides the social network data into covariates, behavior processing variables, adjacency matrix and result variables according to the keywords, the user's characteristic information, the content recommended to the user by the old content recommendation strategy, the friend relationship information between users and the user's preferences, and constructs a social network data set using the divided social network data, and uses the content recommended to the user by the new content recommendation strategy as the evaluation processing variable; A model optimization module, for training an offline policy evaluation model on a social network dataset and an evaluation processing variable, wherein the offline policy evaluation model comprises an embedding function and a binary classifier, wherein the binary classifier comprises a graph convolutional network; In the offline strategy evaluation model, first, a learnable embedding function is used to map the behavior processing variable and the evaluation processing variable to the semantic space respectively, and a behavior processing embedding with a dimension consistent with the covariate is obtained for the behavior processing variable, and an evaluation processing embedding with a dimension consistent with the covariate is obtained for the evaluation processing variable, and the covariate and the behavior processing embedding of each marketing data are connected to form a behavior joint representation, and the covariate and the evaluation processing embedding of each marketing data are connected to form an evaluation joint representation, and corresponding labels are set for the behavior joint representation and the evaluation joint representation, and an adjacency matrix containing self-connection is formed by adding the adjacency matrix and the unit matrix, and the adjacency matrix containing self-connection, the evaluation joint representation and the behavior joint representation are input into the binary classifier, and the graph convolution network processes the adjacency matrix containing self-connection and the behavior joint representation to obtain a first output vector corresponding to the behavior joint representation, and the graph convolution network processes the adjacency matrix containing self-connection and the evaluation joint representation to obtain a second output vector corresponding to the evaluation joint representation; The result acquisition module is used to calculate the weight of the social network data result variable from the updated first output vector after the offline strategy evaluation model training is completed, and to perform weighted summation of the weight of the social network data result variable and the corresponding social network data result variable to obtain the evaluation strategy value, add the weights of the social network data result variables to obtain the normalized weight, and divide the evaluation strategy value by the normalized weight to obtain the self-normalized evaluation strategy value, which is used to determine whether to adopt the new content recommendation strategy.
8. A computer-readable storage medium, It is characterized in that The storage medium stores a computer program, and when the computer program is executed by the processor, the offline strategy evaluation method applicable to a social network as claimed in any one of claims 1 to 6 is implemented.
9. A computer electronic device, It is characterized in that including memory and processor; The memory is used to store computer programs; The processor is used to implement the offline strategy evaluation method applicable to a social network as described in any one of claims 1 to 6 when executing the computer program.
10. A computer program product comprising a computer program / instructions, It is characterized in that When the computer program / instruction is executed by a processor, it can implement an offline strategy evaluation method applicable to a social network as described in any one of claims 1 to 6.
Citation Information
Patent Citations
Cantonese rumor detection method based on graph convolutional network and external knowledge embedding
CN116776889A
User-topic interestingness evaluation method based on hypergraph convolution in social network
CN118210984A
Methods, computer-accessible medium and systems for construction of and interference with networked data, for example, in a financial setting
US20130117278A1
System and Method for Social Engineering Cyber Security Training
US20220094702A1
