An algorithm discrimination identification method based on multi-dimensional data correlation analysis

Abstract

The application discloses a kind of algorithm discrimination identification methods based on multidimensional data correlation analysis, it is related to internet technology field, including, user attribute data, user behavior log data, content feature data and context information data are collected from short video platform, and carry out missing value filling, classification variable coding and continuous variable standardization processing, generate standardization data table;Standardization data table is converted into three-order tensor structure, and user behavior log data is interacted with content feature data in time slice frequency, the application is innovated multidimensional data correlation analysis method, effectively solve the dimensionality limitation and dynamic pattern recognition insufficient problem existing in current algorithm discrimination detection technology, construct user-content-time three-order tensor structure, using sliding window alignment and laplace smoothing processing, the accurate capture of space-time dynamic discrimination mode is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet, in particular to an algorithm discrimination identification method based on multi-dimensional data correlation analysis. BACKGROUND

[0002] With the rapid development of short video platforms, recommendation algorithms play an important role in improving user stickiness and the efficiency of content distribution. The potential discrimination problems in the algorithm recommendation process have gradually attracted the attention of academia and industry. Existing algorithm discrimination identification technologies are mainly based on two-dimensional data analysis methods, which identify potential discrimination patterns through user-content interaction matrices combined with statistical tests. Such methods usually rely on manually defined sensitive attributes and simple statistical significance tests. Although they can discover explicit discrimination phenomena to some extent, they have obvious limitations when dealing with high-dimensional and dynamic short video platform data, especially in capturing discrimination pattern changes in the time dimension and implicit discrimination caused by multi-dimensional synergies. Existing technologies lack effective modeling methods, and traditional methods often cannot distinguish between algorithm bias and user natural behavior differences, resulting in high false positive rates.

[0003] The main deficiency of the prior art is the limitation of the analysis dimension. Current algorithm discrimination detection methods are mostly based on two-dimensional matrix analysis, which cannot effectively integrate time dimension information, so they cannot capture the dynamic characteristics of discrimination behavior changes over time. Content may have systematic exposure differences for specific user groups at specific time periods, but this time-sensitive discrimination pattern is easily overlooked in static two-dimensional analysis. When dealing with high sparsity user behavior data, existing methods usually use simple linear interpolation or fixed rule filling, which may cause bias in discrimination pattern recognition. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides an algorithm discrimination identification method based on multi-dimensional data correlation analysis, which solves the problem that the prior art cannot effectively capture time dimension dynamic discrimination patterns and handle high sparsity behavior data.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In the first aspect, the present application provides an algorithm discrimination identification method based on multi-dimensional data correlation analysis, which includes collecting user attribute data, user behavior log data, content feature data and context information data from a short video platform, and performing missing value filling, categorical variable encoding and continuous variable standardization processing to generate a standardized data table.

[0008] The standardized data table is converted into a three-order tensor structure, user behavior log data is interacted with content feature data in a time slice, and through sliding time window alignment and Laplace smoothing processing, a dense three-order tensor is obtained;

[0009] The dense three-order tensor is subjected to CP tensor decomposition to obtain a user factor matrix, a content factor matrix and a time factor matrix, and a discrimination factor is extracted based on the user factor matrix and user attribute data;

[0010] A sensitive group hypergraph is constructed according to the discrimination factor to obtain user groups, content categories and time context data, and a hypergraph convolution network is used to obtain a discrimination strength score of a hyperedge;

[0011] A discrimination heat map and an algorithm discrimination audit report are generated based on the discrimination strength score of the hyperedge, and a high-risk discrimination mode and an optimization strategy table are marked.

[0012] As a preferred scheme of the algorithm discrimination identification method based on multi-dimensional data correlation analysis, wherein: user attribute data, user behavior log data, content feature data and context information data are collected from a short video platform, and missing value filling, classification variable encoding and continuous variable standardization processing are performed to generate a standardized data table, including the following steps,

[0013] The user ID, gender, age and region fields are obtained, and the user ID is subjected to SHA-256 desensitization processing to obtain user attribute data;

[0014] The user ID, content ID, interaction type, timestamp and play duration are recorded through a buried point to obtain user behavior log data;

[0015] Video text features are extracted using RoBERTa and cover image features are extracted using ResNet-50 for weighted fusion to obtain content feature data;

[0016] Device type, network environment and time period information are collected to obtain context information data;

[0017] The user attribute data, user behavior log data, content feature data and context information data are subjected to multi-table connection and integration to obtain an original data table;

[0018] The original data table is subjected to KNN interpolation and filling to obtain a complete data table, the original data table is encoded using a three-level hierarchical coding to obtain an encoded feature table, and the original data table is subjected to continuous variable standardization processing to obtain a standardized data table.

[0019] As a preferred scheme of the algorithm discrimination recognition method based on multi-dimensional data correlation analysis, wherein: the standardized data table is converted into a three-order tensor structure, the user behavior log data is interacted with the content feature data in the time slice, and a dense three-order tensor is obtained through sliding time window alignment and Laplace smoothing processing, including the following steps,

[0020] The timestamps in the standardized data table are divided into sliding windows to obtain behavior log data with window ID;

[0021] The user ID and content ID combination in each time slice window is counted and weighted frequency interaction is outputted to obtain a frequency matrix;

[0022] The normalized three-order tensor is outputted by dynamically scaling the frequency matrix;

[0023] The user-content bipartite graph is constructed through the normalized three-order tensor, the behavior patterns of similar users are aggregated through the GAT layer, the smoothed tensor is outputted, the actual observation number of the smoothed tensor is detected, and a sparse position is obtained;

[0024] The sparse position is added with pseudo-count based on Laplace correction to obtain a dense three-order tensor.

[0025] As a preferred scheme of the algorithm discrimination recognition method based on multi-dimensional data correlation analysis, wherein: the dense three-order tensor is subjected to CP tensor decomposition to obtain a user factor matrix, a content factor matrix and a time factor matrix, including the following steps,

[0026] The dense three-order tensor is unfolded according to the user, content and time modes to obtain a time mode unfolding matrix;

[0027] The time mode unfolding matrix is subjected to non-negative matrix decomposition to obtain a user factor matrix, a content factor matrix and a time factor matrix.

[0028] As a preferred scheme of the algorithm discrimination recognition method based on multi-dimensional data correlation analysis, wherein: a discrimination factor is extracted based on the user factor matrix and user attribute data, including the following steps,

[0029] The user attribute data is converted into a binary variable, the binary attribute is weighted and fused to generate a composite sensitive attribute vector;

[0030] The Pearson correlation coefficient of the user factor matrix and the composite sensitive attribute vector is analyzed through factor-attribute analysis to obtain a preliminary discrimination factor set;

[0031] The causal effect of the user factor on the content exposure quantity in the preliminary discrimination factor set is quantified through double machine learning to output a discrimination factor.

[0032] As a preferred scheme of the algorithm discrimination recognition method based on multi-dimensional data correlation analysis, wherein: according to the discrimination factor, a sensitive group hypergraph is constructed, and user groups, content categories and time context data are obtained, including the following steps,

[0033] K-means is performed on the user factor matrix based on the discrimination factor, and user groups are obtained;

[0034] The peak period of the time factor is extracted from the user group, and the content categories and time context data of the user group in the peak period are counted.

[0035] As a preferred scheme of the algorithm discrimination recognition method based on multi-dimensional data correlation analysis, wherein: the discrimination intensity score of the hyperedge is calculated by using a hypergraph convolution network, including the following steps,

[0036] The user group, content category and time context data are summarized into triple data;

[0037] The triple data is set as a hyperedge, and the hyperedge weight is calculated;

[0038] Based on the hyperedge weight, the user factor matrix and the content factor matrix, a hypergraph vertex feature matrix and a correlation matrix are constructed, and hypergraph structure data is obtained;

[0039] The vertex embedding is optimized using an adversarial network, and a vertex embedding matrix is obtained;

[0040] The discrimination intensity score of the hyperedge is calculated based on the vertex embedding matrix.

[0041] As a preferred scheme of the algorithm discrimination recognition method based on multi-dimensional data correlation analysis, wherein: based on the discrimination intensity score of the hyperedge, a discrimination heat map and an algorithm discrimination audit report are generated, and a high-risk discrimination mode and an optimization strategy table are marked, including the following steps,

[0042] Based on the discrimination intensity score of the hyperedge, the user group, content category, time context three-dimensional coordinates and score are extracted, and a structured heat data table is output, the platform historical discrimination intensity mean and standard deviation are generated, and a high-risk threshold is set;

[0043] Using D3.js, the structured heat data table is converted into an interactive three-dimensional heat map, the high-risk threshold is marked, the discrimination intensity score of the hyperedge greater than the high-risk threshold is screened out, and a high-risk discrimination mode is obtained;

[0044] Through the contribution degree of the sensitive attribute data to the high-risk mode, an attribute-contribution degree ranking table is obtained;

[0045] The suppressed group and the overexposed content are extracted in the attribute-contribution degree ranking table.

[0046] Calculate exposure compensation weight based on the inhibited group;

[0047] Weight reduction based on overexposed content to obtain an optimization strategy table.

[0048] In a second aspect, the present application provides a computer device comprising a memory and a processor, the memory storing a computer program, wherein: the computer program is executed by the processor to implement any step of the algorithm discrimination identification method based on multi-dimensional data correlation analysis according to the first aspect of the present application.

[0049] In a third aspect, the present application provides a computer readable storage medium having a computer program stored thereon, wherein: the computer program is executed by the processor to implement any step of the algorithm discrimination identification method based on multi-dimensional data correlation analysis according to the first aspect of the present application.

[0050] The present application has the following beneficial effects: through the innovative multi-dimensional data correlation analysis method, the dimension limitation and dynamic pattern recognition deficiency in the existing algorithm discrimination detection technology are effectively solved, the user-content-time three-order tensor structure is constructed, the sliding window alignment and Laplace smoothing processing are adopted, the accurate capture of the space-time dynamic discrimination pattern is realized, the sensitive group hypergraph is constructed based on the discrimination factor, the discrimination intensity score is calculated through the optimized hypergraph convolution network of the adversarial training, and the multi-entity collaborative discrimination effect can be identified. The technical scheme completely covers the whole process of data collection, tensor decomposition, hypergraph modeling and visualization analysis, breaks through the limitation of traditional two-dimensional analysis dimension, fuses tensor decomposition and causal inference to improve accuracy, and the generated audit report and optimization strategy can directly guide algorithm optimization. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical scheme of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0052] Fig. 1 Flowchart of the algorithm discrimination identification method based on multi-dimensional data correlation analysis;

[0053] Fig. 2 Schematic diagram of weight reduction of overexposed content;

[0054] Fig. 3 Schematic diagram of the standardized data table;

[0055] Fig. 4 Flowchart of the dense three-order tensor. DETAILED DESCRIPTION

[0056] In order to make the above objectives, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0057] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the concept of the present application, therefore the present application is not limited to the specific embodiments disclosed below.

[0058] Secondly, the "one embodiment" or "embodiment" referred to herein means that the specific features, structures or characteristics can be included in at least one implementation of the present application. "In one embodiment" appearing in different places in the specification does not mean the same embodiment, nor is it an embodiment that is independent of or mutually exclusive of other embodiments.

[0059] Reference Figs. 1-4 For one embodiment of the present application, the embodiment provides an algorithm discrimination recognition method based on multi-dimensional data correlation analysis, comprising the following steps:

[0060] S1, collecting user attribute data, user behavior log data, content feature data and context information data from a short video platform, and performing missing value filling, classification variable encoding and continuous variable standardization processing to generate a standardized data table.

[0061] S1.1, obtain the user ID, gender, age and region fields, and perform SHA-256 desensitization processing on the user ID to obtain the user attribute data.

[0062] Further, the user ID, gender, age and region fields are obtained from the short video platform API, wherein the user ID is desensitized by SHA-256 hash algorithm, and the processed user ID and the original fields of gender, age and region jointly constitute the user attribute data. First, the original user data is obtained by calling the short video platform API interface, and the original user data includes the user ID string, the gender classification value, the age value and the region code; then the SHA-256 hash algorithm is applied to the user ID string for one-way encryption conversion to generate a fixed-length hash value as the desensitized user ID, and the desensitized user ID is combined with the unprocessed gender, age and region fields according to the user dimension to form a structured user attribute data table.

[0063] S1.2, record the user ID, content ID, interaction type, timestamp and play duration by burying points to obtain the user behavior log data.

[0064] Furthermore, when a user triggers a play, like, comment, or share action, the event tracking program captures event information, including the user ID string, content ID string, interaction type classification value (e.g., 1 for play, 2 for like, 3 for comment, 4 for share), the UTC timestamp value of the action (accurate to milliseconds), and the duration value of the play action (in seconds). All captured fields are recorded as structured log entries in the order of event occurrence, and multiple log entries are arranged in chronological order to form a user behavior log dataset.

[0065] S1.3. Use RoBERTa to extract video text features and ResNet-50 to extract cover image features, and then perform weighted fusion to obtain content feature data.

[0066] Furthermore, the RoBERTa model was used to extract 768-dimensional text feature vectors from the video title and description text, and the ResNet-50 model was used to extract 2048-dimensional image feature vectors from the video cover image. After L2 normalization of the text feature vectors and image feature vectors respectively, they were weighted and fused in a weight ratio of 0.6:0.4 to generate the final content feature data.

[0067] S1.4 Collect device type, network environment and time period information to obtain context information data.

[0068] Furthermore, device type classification values ​​(e.g., 1 for iOS devices, 2 for Android devices, 3 for Web devices), network connection type classification values ​​(e.g., 1 for Wi-Fi connections, 2 for 4G networks, 3 for 5G networks), and current time period classification values ​​(e.g., 0-23 for 24-hour time periods) are obtained from the mobile device system API. The collected device type classification values, network type classification values, and time period classification values ​​are aligned according to the timestamp of the event to form structured contextual information data.

[0069] S1.5. Perform multi-table joins and integration of user attribute data, user behavior log data, content feature data, and context information data to obtain the original data table.

[0070] Furthermore, based on user behavior log data, user attribute information is obtained by left-joining the user ID field with the user ID field in the user attribute data; then, content feature information is obtained by left-joining the content ID field with the content ID field in the content feature data; and finally, the original data table is obtained by precisely matching and joining the timestamp field with the timestamp field in the context information data.

[0071] S1.6, KNN interpolation and padding are performed on the original data table to obtain a complete data table, the original data table is encoded using a three-level hierarchical encoding to obtain an encoded feature table, and the original data table is subjected to continuous variable standardization processing to obtain a standardized data table.

[0072] Further, first, the numerical fields such as age values and play duration values in the original data table are supplemented with missing values based on the KNN algorithm, and the classification fields such as gender classification values are filled with a special marker value -1 to form a complete data table without missing values. The region field in the complete data table is subjected to three-level hierarchical encoding, and the content category and other classification fields are subjected to One-Hot encoding conversion. The continuous variables in the encoded feature table are subjected to Robust standardization processing by dividing the difference from the median by the interquartile range to obtain a standardized data table.

[0073] S2, the standardized data table is converted into a three-order tensor structure, the user behavior log data is interacted with the content feature data in the time slice, and the sliding time window alignment and Laplace smoothing processing are performed to obtain a dense three-order tensor.

[0074] S2.1, the time stamp in the standardized data table is divided into a sliding window to obtain behavior log data with window ID.

[0075] Further, first, the time stamp values in the standardized data table are sorted by UTC time; then, starting from the starting time point, the time window is divided at intervals of 15 minutes, and each window covers a range of 900 seconds; then, the window is slid with a step of 5 minutes to ensure that adjacent windows have an overlapping area of 600 seconds. An incremental window ID value is assigned to each time window. Each behavior record in the standardized data table is matched to the corresponding time window according to the time stamp value, and a window ID field is added to the original record. Each record in the behavior log data with window ID contains the original fields of the standardized data table and the newly added window ID value field. The window ID value represents the time slice number to which the behavior belongs.

[0076] S2.2, the user ID and content ID combination in each time slice window is counted to obtain a weighted frequency interaction matrix.

[0077] Further, first, the behavior log data with window ID is grouped according to the window ID value; then, in each time slice window, the records with the same user ID and content ID combination are classified according to the interaction type classification value to assign weights (e.g., play = 1, like = 2, comment = 3), and the weighted frequency interaction sum is calculated. The user ID in each time slice window is used as the row index, the content ID is used as the column index, and the weighted frequency interaction is used as the element value to construct a frequency matrix.

[0078] S2.3, output the normalized third-order tensor by dynamically scaling the frequency matrix.

[0079] Further, first, the total interaction times of each user ID in all time slice windows are counted and divided by the number of days to obtain a user daily interaction value, and the total interaction times of each content ID in all time slice windows are counted and divided by the number of occurrences to obtain a content average interaction value. Each element value in the frequency matrix is normalized, and the normalized frequency matrix is organized according to the three dimensions of user ID, content ID and time slice window to form a third-order tensor structure.

[0080] S2.4, construct a user-content bipartite graph through the normalized third-order tensor, aggregate the behavior patterns of similar users through a GAT layer, output a smoothed tensor, and detect the actual observation times of the smoothed tensor to obtain sparse positions.

[0081] Further, first, the third-order tensor is averaged in the time dimension to obtain a user-content two-dimensional matrix as an adjacency matrix of a bipartite graph, and then a graph attention network (GAT) is used for two rounds of message passing of user vertices. In each round of message passing, the user vertices aggregate the features of their neighbor content vertices through an attention mechanism, and the updated user vertex features are remapped to the third-order tensor to replace the original user dimension features, generating a smoothed tensor. Finally, the actual observation times of each user-content-time slice combination in the smoothed tensor are counted, and positions with observation times less than 10 are marked as sparse positions.

[0082] S2.5, add pseudo counts to the sparse positions based on Laplace correction to obtain a dense third-order tensor.

[0083] Specifically, the expression is,

[0084] ,

[0085] wherein, is the dense third-order tensor, is the initial value of the user , content and time slice position in the smoothed tensor, is the actual observed total interaction times of the user and content in the time slice, K is the hyperparameter of Laplace smoothing, is the user index, is the content, is the time slice.

[0086] S3. Perform CP tensor decomposition on the dense third-order tensor to obtain the user factor matrix, content factor matrix and time factor matrix.

[0087] S3.1 Expand the dense third-order tensor according to the patterns of user, content and time to obtain the time pattern expansion matrix.

[0088] Furthermore, first, keeping the user dimension unchanged, we merge the content and time dimensions to generate a user pattern expansion matrix. Then, keeping the content dimension unchanged, we merge the user and time dimensions to generate a content pattern expansion matrix. Finally, keeping the time dimension unchanged, we merge the user and content dimensions to generate a time pattern expansion matrix.

[0089] S3.2 Perform non-negative matrix decomposition on the time pattern expansion matrix to obtain the user factor matrix, content factor matrix, and time factor matrix.

[0090] Furthermore, we first set the decomposition rank to 0, and then use the multiplication update rule to iteratively optimize the objective function to obtain the joint factor matrix. The joint factor matrix is ​​then split into user factor matrix, content factor matrix and time factor matrix according to user and content dimensions.

[0091] S4. Extract discrimination factors based on user factor matrix and user attribute data.

[0092] S4.1 Convert user attribute data into binary variables, weight and fuse the binary attributes to generate a composite sensitive attribute vector.

[0093] Furthermore, the gender field is first converted into a binary variable, the age field is converted into a binary variable, and the region field is encoded into a three-level binary vector of province-city-district according to the administrative division level. Then, weight coefficients are assigned to the three types of binary variables (gender 0.6, age 0.3, region 0.1), and the composite sensitive attribute value is obtained by weighted calculation, generating a composite sensitive attribute vector.

[0094] S4.2. By analyzing the Pearson correlation coefficient between the user factor matrix and the composite sensitive attribute vector using factor-attribute analysis, a preliminary set of discrimination factors is obtained, expressed as follows:

[0095] ;

[0096] Among them, the Pearson correlation coefficients between user factors and the sensitive attribute vector s The first user factor matrix There are 1 user factor, and s is a vector of sensitive attributes. Let be the standard deviation of the sensitive attribute vector s.

[0097] Further, for each column of the user factor matrix, the covariance with the composite sensitive attribute vector is calculated, and divided by the standard deviation of and the standard deviation of, respectively, to obtain the Pearson correlation coefficient, and the factor index that meets the condition is screened to form a preliminary discrimination factor set.

[0098] S4.3, quantifying the causal effect of the user factor in the preliminary discrimination factor set on the content exposure amount by double machine learning, outputting the discrimination factor.

[0099] Specifically, the expression is,

[0100] ,

[0101] wherein, is the causal effect estimate of the th user factor on the content exposure amount, is the total number of user samples, is the user sample, is the content exposure amount of the user sample , is the score of the th user factor, is the confusion variable vector of the user sample , is the exposure amount prediction, is the factor score prediction.

[0102] Further, first, train the content exposure amount prediction model to predict the content exposure amount with the user confusion variables (such as activity, device type) as input, and train the user factor score prediction model to predict the user factor score with the same confusion variables, then calculate the product mean of the residuals of the two models to obtain the causal effect estimate, and finally screen the factors whose causal effect absolute value exceeds the platform historical baseline value as the final discrimination factor.

[0103] S5, constructing a sensitive group hypergraph according to the discrimination factor, obtaining user groups, content categories and time context data.

[0104] S5.1, K-means is performed on the user factor matrix based on the discrimination factor to obtain user groups.

[0105] Further, based on the discrimination factor set, the corresponding column vectors in the user factor matrix are screened, and the K-means clustering algorithm is used to divide the users into 5 groups. First, each row of the corresponding column vector is subjected to L2 normalization processing, then 5 cluster center points are randomly initialized, and through iterative optimization, the user groups are output.

[0106] ​S5.2 Extract the peak time periods of time factors for user groups, and statistically analyze the content categories and time context data of user group interactions during the peak time periods.

[0107] Furthermore, we first filter all user behavior records during peak hours in a standardized data table, then statistically analyze the content categories that the user group interacts with most frequently during the period (sorted by frequency of interaction), and simultaneously extract the contextual information data (device type and network environment) corresponding to that period, thus obtaining the content categories and time context data of user group interactions during peak hours.

[0108] S6. Use a hypergraph convolutional network to calculate the discrimination intensity score of the hyperedge.

[0109] S6.1. Summarize user groups, content categories, and time context data into triplet data.

[0110] Furthermore, we first extract the content category with the highest frequency of interaction for each user group during peak hours, then obtain the main contextual features (device type ratio and network environment distribution) of the group during the time period from the standardized data table, and finally encapsulate the user group, content category and time context data into a complete triplet data.

[0111] S6.2 Set the triplet data as a superedge and calculate the superedge weight.

[0112] Specifically, the expression is,

[0113] ;

[0114] in, Let e ​​be the weight of the hyperedge. For user groups Percentage of total users proportion, For content categories The average factor score on the discrimination factor m, Peak periods in the time factor matrix W In user factors The weight value on.

[0115] S6.3 Based on the hyperedge weights, user factor matrix, and content factor matrix, construct the hypergraph vertex feature matrix and correlation matrix to obtain the hypergraph structure data.

[0116] Furthermore, the user factor matrix and content factor matrix are first concatenated row by row, and the time factor matrix is ​​supplemented with time vertex features to form a vertex feature matrix. Then, the correlation matrix is ​​constructed, and when a vertex belongs to a hyperedge, the hypergraph structure data is finally output.

[0117] S6.4, optimize the vertex embedding using the adversarial network to obtain a vertex embedding matrix.

[0118] Further, based on the generator network, output the fake vertex features, the discriminator network, and then optimize the objective through adversarial training. The generator tries to generate fake features similar to the distribution of real vertex features, and the discriminator learns to distinguish real and fake features. After training, input the original vertex features into the feedforward layer of the discriminator network, and extract the 512-dimensional activation values of the intermediate layer of the discriminator as the optimized vertex embedding matrix.

[0119] S6.5, calculate the discrimination intensity score of the hyperedge based on the vertex embedding matrix.

[0120] Specifically, the expression is,

[0121] ;

[0122] wherein, is the discrimination intensity score of the hyperedge e, is the average vertex embedding of the user group is the maximum vertex embedding of the content category is the vertex embedding of the temporal context is the vertex embedding of the temporal context is the output of the HGCN at the layer. L L

[0123] S7, generate a discrimination heat map and an algorithm discrimination audit report based on the discrimination intensity score of the hyperedge, and mark out high-risk discrimination patterns and optimization strategies.

[0124] S7.1, extract the user group, content category, and temporal context three-dimensional coordinates and scores based on the discrimination intensity score of the hyperedge, output a structured heat data table, generate the historical discrimination intensity mean and standard deviation of the platform, and set a high-risk threshold.

[0125] Further, first map each hyperedge to a user group classification code, map the content category to a content category code, and convert to a 24-hour period number to obtain the mean and standard deviation of all hyperedge scores of the platform in the past 30 days, and set a high-risk threshold.

[0126] S7.2, convert the structured heat data table to an interactive three-dimensional heat map using D3.js, mark out the high-risk threshold, filter out the hyperedge discrimination intensity score greater than the high-risk threshold, and obtain the high-risk discrimination pattern.

[0127] ​​Further, first, the user group encoding is mapped to the X-axis coordinate, the content category encoding is mapped to the Y-axis coordinate, the time encoding is mapped to the Z-axis coordinate, and the discrimination intensity score is converted into a color value (0-1 normalized corresponding to blue-yellow-red spectrum), then the cube data points are plotted in the three-dimensional space, the rotation and scaling interaction is realized through the Three.js engine, then the high-risk threshold is marked in the Z-axis direction, the data points with the discrimination intensity score of the hyperedge)>high-risk threshold are marked as flashing red, and finally all hyperedges exceeding the high-risk threshold are screened as a high-risk discrimination mode set.

[0128] S7.3, the contribution degree of sensitive attribute data to the high-risk mode is obtained, and an attribute-contribution degree ranking table is obtained.

[0129] Further, first, the sensitive attribute appearance frequency of all user groups in the high-risk hyperedge set is counted to obtain the attribute contribution degree, and the attribute-contribution degree ranking table is generated by sorting the contribution degrees from high to low. The table includes fields such as sensitive attribute name, appearance frequency, and contribution degree percentage.

[0130] S7.4, the suppressed group and the overexposed content are extracted from the attribute-contribution degree ranking table.

[0131] Further, first, the sensitive attributes with the top 20% contribution degrees are selected in the attribute-contribution degree ranking table, and the corresponding user groups are marked as the suppressed group. The appearance frequency of each content category in the high-risk hyperedge set is counted, and the categories (such as game category and short video category) with an appearance frequency more than twice the average exposure of the same type of content on the platform are selected as the overexposed content. Finally, the suppressed group list and the overexposed content list are output.

[0132] S7.5, based on the suppressed group, the exposure compensation weight is calculated.

[0133] Specifically, the expression is,

[0134] ,

[0135] wherein, is the exposure compensation weight, is the discrimination intensity score of the hyperedge e, is the mean of the discrimination score, is the standard deviation of the discrimination score.

[0136] Further,

[0137] S7.6, based on the overexposed content, the weight is reduced to obtain an optimization strategy table, and the expression is,

[0138] ;

[0139] wherein, The adjusted new weight, The original recommendation weight of the content, The maximum value of the super-edge discrimination score.

[0140] Further, the original recommendation weight of the content category is obtained according to the platform recommendation, the maximum value of all super-edge discrimination scores is found, the adjusted weight is calculated, and finally the optimization strategy table is generated.

[0141] The embodiment also provides a computer device suitable for the algorithm discrimination identification method based on multi-dimensional data correlation analysis, including a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to realize the algorithm discrimination identification method based on multi-dimensional data correlation analysis provided in the above embodiment.

[0142] The computer device can be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device. In addition, the input device can also be an external keyboard, touchpad or mouse, etc.

[0143] The embodiment also provides a storage medium on which a computer program is stored, the program being executed by a processor to implement the algorithm discrimination method for realizing multi-dimensional data correlation analysis as proposed in the above embodiment; the storage medium can be realized by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk.

[0144] To sum up, the present application effectively solves the problems of dimension limitation and insufficient dynamic pattern recognition in the existing algorithm discrimination detection technology by the innovative multi-dimensional data correlation analysis method, constructs a user-content-time three-order tensor structure, realizes accurate capture of the spatio-temporal dynamic discrimination pattern by using sliding window alignment and Laplace smoothing processing, constructs a sensitive group hypergraph based on a discrimination factor, calculates a discrimination intensity score by an optimized hypergraph convolution network through adversarial training, and can identify a multi-entity collaborative discrimination effect. The technical solution completely covers the whole process of data collection, tensor decomposition, hypergraph modeling, and visualization analysis, breaks through the limitation of the traditional two-dimensional analysis dimension, fuses tensor decomposition and causal inference to improve accuracy, and the generated audit report and optimization strategy can directly guide algorithm optimization.

[0145] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application but not limit the present application, although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application, and all of them should be covered in the scope of the claims of the present application.

Claims

1. An algorithm discrimination identification method based on multi-dimensional data correlation analysis, characterized in that: The application relates to a method for auditing algorithm discrimination, and belongs to the technical field of algorithm discrimination auditing. Collect user attribute data, user behavior log data, content feature data and context information data from a short video platform, and perform missing value filling, classification variable coding and continuous variable standardization processing to generate a standardized data table; Convert the standardized data table into a three-order tensor structure, and perform frequency interaction on the content feature data within a time slice based on the user behavior log data, and through sliding time window alignment and Laplace smoothing processing, a dense three-order tensor is obtained; Perform CP tensor decomposition on the dense three-order tensor to obtain a user factor matrix, a content factor matrix and a time factor matrix, and extract a discrimination factor based on the user factor matrix and the user attribute data; According to the discrimination factor, a sensitive group hypergraph is constructed, user groups, content categories and time context data are obtained, and a hypergraph convolution network is used to calculate the discrimination intensity score of the hyperedge; Based on the discrimination intensity score of the hyperedge, a discrimination heat map and an algorithm discrimination audit report are generated, and a high-risk discrimination mode and an optimization strategy table are marked out; Based on the user factor matrix and the user attribute data, a discrimination factor is extracted, including the following steps, Convert the user attribute data into a binary variable, and perform weighted fusion on the binary attribute to generate a composite sensitive attribute vector; Analyze the Pearson correlation coefficient of the user factor matrix and the composite sensitive attribute vector through factor-attribute analysis to obtain a preliminary discrimination factor set; Quantify the causal effect of the user factor on the content exposure quantity in the preliminary discrimination factor set through double machine learning to output the discrimination factor; According to the discrimination factor, a sensitive group hypergraph is constructed, user groups, content categories and time context data are obtained, including the following steps, Based on the discrimination factor, K-means is performed on the user factor matrix to obtain user groups; Extract the peak period of the time factor from the user groups, and count the content categories and time context data interacted by the user groups in the peak period; Using a hypergraph convolution network, the discrimination intensity score of the hyperedge is calculated, including the following steps, The user groups, content categories and time context data are summarized into triple data; The triple data is set as a hyperedge, and the hyperedge weight is calculated; Based on the hyperedge weight, the user factor matrix and the content factor matrix, a hypergraph vertex feature matrix and a correlation matrix are constructed to obtain hypergraph structure data; The vertex embedding is optimized using an adversarial network to obtain a vertex embedding matrix; Based on the vertex embedding matrix, the discrimination intensity score of the hyperedge is calculated. 2.The method of claim 1, wherein the method further comprises: Collect user attribute data, user behavior log data, content feature data and context information data from a short video platform, and perform missing value filling, classification variable coding and continuous variable standardization processing to generate a standardized data table, including the following steps, Obtain the user ID, gender, age and region fields, and perform SHA-256 desensitization processing on the user ID to obtain user attribute data; Record the user ID, content ID, interaction type, timestamp and play duration through burying to obtain user behavior log data; Weighted fusion is performed on the video text features extracted by RoBERTa and the cover image features extracted by ResNet-50 to obtain content feature data; Collect device type, network environment and time period information to obtain context information data; Connect and integrate user attribute data, user behavior log data, content feature data and context information data to obtain an original data table; Perform KNN interpolation and padding on the original data table to obtain a complete data table, encode the original data table using a three-level hierarchical coding to obtain an encoded feature table, and perform continuous variable standardization processing on the original data table to obtain a standardized data table. 3.The method of claim 2, wherein: Convert the standardized data table into a three-order tensor structure, and perform frequency interaction on the content feature data within a time slice by using the user behavior log data, and perform sliding time window alignment and Laplace smoothing processing to obtain a dense three-order tensor, including the following steps, Divide the timestamps in the standardized data table into sliding windows to obtain behavior log data with window ID; For each user ID and content ID combination in a time slice window, count the weighted frequency interaction and output a frequency matrix; Perform dynamic scaling on the frequency matrix to output a normalized three-order tensor; Construct a user-content bipartite graph based on the normalized three-order tensor, aggregate similar user behavior patterns through a GAT layer, output a smoothed tensor, detect the actual observation number of the smoothed tensor, and obtain a sparse position; Add pseudo counts to the sparse position based on Laplace correction to obtain a dense three-order tensor. 4.The method of claim 3, wherein the method further comprises: Perform CP tensor decomposition on the dense three-order tensor to obtain a user factor matrix, a content factor matrix and a time factor matrix, including the following steps, Expand the dense three-order tensor according to the user, content and time modes to obtain a time mode expansion matrix; Perform non-negative matrix decomposition on the time mode expansion matrix to obtain a user factor matrix, a content factor matrix and a time factor matrix.

5. The method of claim 1, wherein: Generate a discrimination heat map and an algorithm discrimination audit report based on the discrimination intensity score of the hyperedge, mark out high-risk discrimination patterns and optimization strategy table, including the following steps, Extract user groups, content categories, time context three-dimensional coordinates and scores based on the discrimination intensity score of the hyperedge, output a structured heat data table, generate a platform historical discrimination intensity mean and standard deviation, and set a high-risk threshold; Convert the structured heat data table into an interactive three-dimensional heat map using D3.js, mark out the high-risk threshold, filter out the hyperedge whose discrimination intensity score is greater than the high-risk threshold, and obtain a high-risk discrimination pattern; Obtain an attribute-contribution ranking table through the contribution of sensitive attribute data to the high-risk pattern; Extract the suppressed group and the overexposed content from the attribute-contribution ranking table; Calculate exposure compensation weights based on the suppressed group; Perform weight reduction based on the overexposed content to obtain an optimization strategy table. 6.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is characterized in that: The processor executes the computer program to implement the steps of the algorithm discrimination identification method based on multi-dimensional data correlation analysis of any one of claims 1-5.

7. A computer readable storage medium having stored thereon a computer program, characterized in that: The computer program is executed by the processor to implement the steps of the algorithm discrimination identification method based on multi-dimensional data correlation analysis of any one of claims 1-5.

Images