Data processing method and apparatus

Abstract

The application discloses a data processing method and device, and relates to the technical field of data processing. A specific implementation of the method comprises the following steps: acquiring a plurality of user data of a target user; constructing a feature vector of the target user according to the plurality of user data; determining a target spectrum graph filter, using the target spectrum graph filter to determine a plurality of principal component characteristic values of the feature vector, and filter parameters of the target spectrum graph filter being obtained by training using a principal component analysis algorithm; determining a target cluster of the target user according to the plurality of principal component characteristic values of the feature vector; and performing corresponding processing on the target user according to the target cluster. The implementation can enable the system to accurately determine the target cluster corresponding to the user, determine the characteristic information of the user through the target cluster, and perform corresponding processing on the user, thereby improving the service experience of the user.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data processing method and apparatus. Background Technology

[0002] Systems often use user data to determine various potential user states in order to provide more tailored services. As the operational time of various application systems increases, the amount of user data also grows. This may result in some useless, redundant, or even erroneous user data within the system. This misleading data often causes the system to incorrectly identify user characteristics and provide services that do not meet user needs, thus degrading the user experience. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a data processing method and apparatus that enable the system to accurately determine the target cluster corresponding to a user, determine the user's feature information through the target cluster, and perform corresponding processing on the user to improve the user's service experience.

[0004] In a first aspect, embodiments of the present invention provide a data processing method, including:

[0005] Obtain multiple user data points for the target user;

[0006] Based on the multiple user data, construct the feature vector of the target user;

[0007] A target spectrogram filter is determined, and multiple principal component eigenvalues ​​of the feature vector are determined using the target spectrogram filter. The filter parameters of the target spectrogram filter are obtained by training using a principal component analysis algorithm.

[0008] The target cluster of the target user is determined based on the multiple principal component feature values ​​of the feature vector.

[0009] Based on the target cluster, the target user is processed accordingly.

[0010] Optionally, the user data is user tags;

[0011] The acquisition of multiple user data of the target user includes:

[0012] Obtain multiple user tags of the target user;

[0013] The step of constructing the feature vector of the target user based on the multiple user data includes:

[0014] Determine the vector representation corresponding to each of the user tags;

[0015] By combining the vector representations corresponding to each user tag, a feature vector of the target user is generated;

[0016] The step of processing the target user according to the target cluster includes:

[0017] Based on the target clustering, determine the demand characteristic information corresponding to the target user;

[0018] Recommendation processing is performed on the target user based on the required feature information.

[0019] Optionally, the user data consists of pixels in a user image;

[0020] The acquisition of multiple user data of the target user includes:

[0021] Receive a verification request for the target user and obtain the verification image corresponding to the verification request;

[0022] The verification image is processed to obtain a user image of a preset size;

[0023] Obtain multiple pixels from the user image;

[0024] The step of constructing the feature vector of the target user based on the multiple user data includes:

[0025] The multiple pixels are combined to generate the feature vector of the target user;

[0026] The step of processing the target user according to the target cluster includes:

[0027] Determine whether the target cluster corresponds to the target user, and obtain the determination result;

[0028] Based on the judgment result, the verification request of the target user is verified.

[0029] Optionally, the step of verifying the target user based on the judgment result includes:

[0030] In response to the determination result indicating that the target cluster corresponds to the target user, it is determined that the target user's verification request has passed verification;

[0031] In response to the judgment result indicating that the target cluster does not correspond to the target user, it is determined that the verification request of the target user has failed verification.

[0032] Optionally, before obtaining multiple user data of the target user, the method further includes:

[0033] Obtain multiple user data groups, each of which includes: k user data and clustering labels;

[0034] Generate feature matrices corresponding to the multiple user data groups;

[0035] Determine the parameters of the candidate filters, and construct the spectral filter corresponding to the feature matrix based on the candidate filter parameters;

[0036] Based on the spectral filter, the principal component matrix corresponding to the feature matrix is ​​obtained;

[0037] Based on the principal component matrix, determine d principal component eigenvalues ​​corresponding to each user data group, where d is less than k;

[0038] Cluster the multiple user data groups based on the d principal component feature values ​​corresponding to the user data groups;

[0039] The target filter parameters are determined based on the clustering results of the multiple user data groups.

[0040] Optionally, determining the target filter parameters based on the clustering results of the multiple user data groups includes:

[0041] The clustering accuracy is determined based on the clustering results and clustering labels of the multiple user data groups.

[0042] In response to the clustering accuracy being greater than a preset threshold, the candidate filter parameters are determined as the target filter parameters.

[0043] Optionally, after determining the clustering accuracy based on the clustering results and clustering labels of the multiple user data, the method further includes:

[0044] In response to the clustering accuracy not being greater than the preset threshold, the filter parameters are modified;

[0045] Based on the modified filter parameters, the spectral filter corresponding to the feature matrix is ​​reconstructed.

[0046] The clustering accuracy corresponding to the reconstructed spectrogram filter is determined. If the clustering accuracy corresponding to the reconstructed spectrogram filter is not greater than the preset threshold, the filter parameters are further modified.

[0047] The current filter parameter is determined as the target filter parameter until the clustering accuracy corresponding to the current filter parameter is greater than the preset threshold.

[0048] Optionally, generating the feature matrix corresponding to the plurality of user data groups includes:

[0049] For each user data group: determine the vector representation corresponding to each user data in the user data group; generate the feature vector of the user data group based on the vector representation corresponding to each user data;

[0050] The feature vectors of each user data group are combined to generate a feature matrix corresponding to the multiple user data groups.

[0051] Secondly, embodiments of the present invention provide a data processing apparatus, comprising:

[0052] The data acquisition module is used to acquire multiple user data of the target user;

[0053] A vector construction module is used to construct a feature vector of the target user based on the multiple user data.

[0054] The eigenvalue determination module is used to determine the target spectrogram filter, and using the target spectrogram filter, to determine multiple principal component eigenvalues ​​of the eigenvector. The filter parameters of the target spectrogram filter are obtained by training using a principal component analysis algorithm.

[0055] The clustering module is used to determine the target cluster of the target user based on the multiple principal component feature values ​​of the feature vector;

[0056] The processing module is used to perform corresponding processing on the target user based on the target cluster.

[0057] Optionally, the user data is user tags;

[0058] The data acquisition module is specifically used for:

[0059] Obtain multiple user tags of the target user;

[0060] The vector construction module is specifically used for:

[0061] Determine the vector representation corresponding to each of the user tags;

[0062] By combining the vector representations corresponding to each user tag, a feature vector of the target user is generated;

[0063] The processing module is specifically used for:

[0064] Based on the target clustering, determine the demand characteristic information corresponding to the target user;

[0065] Recommendation processing is performed on the target user based on the required feature information.

[0066] Optionally, the user data consists of pixels in a user image;

[0067] The data acquisition module is specifically used for:

[0068] Receive a verification request for the target user and obtain the verification image corresponding to the verification request;

[0069] The verification image is processed to obtain a user image of a preset size;

[0070] Obtain multiple pixels from the user image;

[0071] The vector construction module is specifically used for:

[0072] The multiple pixels are combined to generate the feature vector of the target user;

[0073] The processing module is used for:

[0074] Determine whether the target cluster corresponds to the target user, and obtain the determination result;

[0075] Based on the judgment result, the verification request of the target user is verified.

[0076] Optionally, the processing module is specifically used for:

[0077] In response to the determination result indicating that the target cluster corresponds to the target user, it is determined that the target user's verification request has passed verification;

[0078] In response to the judgment result indicating that the target cluster does not correspond to the target user, it is determined that the verification request of the target user has failed verification.

[0079] Thirdly, embodiments of the present invention provide an electronic device, comprising:

[0080] One or more processors;

[0081] Storage device for storing one or more programs.

[0082] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the above embodiments.

[0083] Fourthly, embodiments of the present invention provide a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in any of the above embodiments.

[0084] Fifthly, embodiments of the present invention provide a computer program product, including a computer program that, when executed by a processor, implements the methods described in any of the above embodiments.

[0085] One embodiment of the above invention has the following advantages or beneficial effects: It utilizes a target spectrogram filter to determine multiple principal component feature values ​​of the target user. The filter parameters of the target spectrogram filter are obtained through training using a principal component analysis algorithm. Through the spectrogram filter, multi-order local structural relationships in the user data can be obtained to construct richer nearest neighbor relationships between samples, thereby compensating for the poor robustness of the principal component analysis algorithm. Therefore, in the solution of this embodiment, the principal component feature values ​​obtained by combining the spectrogram filter algorithm and the principal component analysis algorithm can effectively describe the user's feature information.

[0086] Furthermore, based on the user's principal component eigenvalues, the system accurately determines the target cluster corresponding to the user, identifies the user's feature information through the target cluster, and processes the user accordingly, thereby improving the user's service experience.

[0087] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0088] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein:

[0089] Figure 1 This is a flowchart illustrating a data processing method provided in the first embodiment of the present invention;

[0090] Figure 2 This is a flowchart illustrating a tag data processing method provided in the second embodiment of the present invention;

[0091] Figure 3 This is a flowchart illustrating a method for determining filter parameters provided in the third embodiment of the present invention;

[0092] Figure 4 This is a schematic diagram of the structure of a data processing device provided in an embodiment of the present invention;

[0093] Figure 5 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers of the present invention. Detailed Implementation

[0094] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0095] The acquisition, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.

[0096] To facilitate understanding of the embodiments of the present invention, some technical terms involved in the embodiments of the present invention will be explained below.

[0097] Feature mapping: This involves mapping raw data from a high-dimensional feature space to a low-dimensional feature space, preserving key feature information, and learning richer and more effective features from the sample data, thereby improving the performance of machine learning tasks based on the new representation. In this low-dimensional space, the source domain data and the target domain data have the same distribution. This allows us to train a classifier using labeled source domain sample data represented in the low-dimensional space to predict target test data.

[0098] Principal Component Analysis (PCA) algorithm: By using linear orthogonal transformations to map the original data into a subspace, redundant and correlated features are removed, which can reflect the original information to a large extent. However, this method is quite sensitive to data corruption and has poor robustness.

[0099] Spectral graph filters: Spectral graph filters are based on convolution operations performed on graphs in the Fourier domain. They are a key component of graph convolutional neural networks. By performing convolution operations with data, spectral graph filters obtain multi-order local structural relationships within the data, thus providing a better and more accurate description of the connections between data points.

[0100] The formula for the filter is: Where L is the Laplace matrix constructed from the data, and θ k Chebyshev coefficient. T k (L) is a Chebyshev polynomial, which is recursively defined as T. k (φ)=2φT k-1 (φ)-T k-2 (φ). And T0(φ)=1, T1(φ)=φ, λ max This represents the largest eigenvalue of L. The Chebyshev polynomial is a multi-order polynomial based on the Laplace matrix. It only relates to those direct or indirect nodes that are at a maximum distance of K steps (K-order neighborhood) from a specific center node; these direct or indirect nodes can influence the center node. After passing through the spectral filter, the data is reconstructed into a weighted sum of multi-order graph structure data due to convolution.

[0101] Laplacian matrix: The Laplacian matrix is ​​an important concept in graph theory, used to describe the connectivity between vertices in an undirected graph. The Laplacian matrix L = DW. Here, W is the adjacency matrix, obtained by calculating the weights between neighboring nodes. D is a diagonal matrix, D = diag(d1, d2, ..., dn).n ), d i =∑ j W ij .

[0102] When constructing the adjacency matrix W, it is necessary to calculate the weights between neighboring nodes. The weights between nodes can be calculated using methods such as 0-1 weights, hot kernels, or dot products.

[0103] The neighboring nodes of a given node can be determined in the following two ways:

[0104] Method 1: Based on the specified number of adjacent nodes k, find the k nearest nodes to the node and designate these k nodes as the node's neighbor nodes.

[0105] Method 2: Specify a distance threshold ∈. If the distance between two nodes is less than the threshold ∈, the two nodes are considered to be neighboring nodes.

[0106] The steps for constructing the spectral filter corresponding to the feature matrix include: determining the adjacency matrix and diagonal matrix of the feature matrix; constructing the Laplacian matrix of the feature matrix based on the adjacency matrix and diagonal matrix; and generating the spectral filter corresponding to the feature matrix based on the Laplacian matrix and filter parameters.

[0107] Figure 1 This is a flowchart illustrating a data processing method provided in the first embodiment of the present invention, as shown below. Figure 1 As shown, the method includes:

[0108] Step 101: Obtain multiple user data for the target user.

[0109] User data can be selected based on requirements. User data may include: user tags, pixels in user images, user preference data, user behavior data, etc.

[0110] Step 102: Construct the feature vector of the target user based on multiple user data.

[0111] The feature vector has a dimension of 1*k, where k is the number of user data points. The user data points can be quantified first, and then combined to form the feature vector for the target user.

[0112] Step 103: Determine the target spectrogram filter. Using the target spectrogram filter, determine the eigenvalues ​​of multiple principal components of the eigenvector. The filter parameters of the target spectrogram filter are obtained by training using the principal component analysis algorithm.

[0113] The filter parameters are those required to construct the spectrogram filter, such as Chebyshev coefficients. Using the target spectrogram filter, the principal component matrix of the target user is obtained. Then, the top d largest eigenvalues ​​from the principal component matrix are selected as the principal component eigenvalues. d is an integer greater than 1.

[0114] There are several ways to determine the target spectrogram filter. One approach is to save the trained spectrogram filter and use it as the target spectrogram filter. The filter parameters of this target spectrogram filter are obtained using principal component analysis (PCA) algorithm training. Alternatively, directly using PCA, we have: Y = (-X T X+γg θ (L))(-X T X+γg θ (L)) T The principal component matrix can then be obtained. Here, X represents the eigenvectors, Y represents the principal component matrix, and g... θ (L) is the target spectrum filter.

[0115] The target spectral filter can also be generated based on the filter parameters and the Laplacian matrix. The Laplacian matrix is ​​reconstructed based on the feature vector of the target user. Specifically, this is achieved using the principal component algorithm formula. Transform it into the formula (-X) T X+γg θ The problem of solving (L))=μY. Where X is the eigenvector, U represents the principal direction matrix, g represents the projection transformation, Y is the principal component matrix, and g is the eigenvector. θ (L) is the target spectrum filter.

[0116] Through the following formula: (-X) T X+γg θ (L))(-X T X+γg θ (L)) T Calculate the principal component matrix. Select the eigenvectors corresponding to the top d largest eigenvalues ​​from the principal component matrix to obtain the principal component eigenvalues ​​of the target user.

[0117] In one embodiment of the present invention, if the user data of the target user has a new category, the Laplacian matrix is ​​reconstructed based on the feature vector of the target user, and the target spectral filter is constructed using the trained filter parameters and the reconstructed Laplacian matrix, and feature mapping is performed to obtain the principal component matrix.

[0118] If the category corresponding to the target user's user data is included in the existing categories, then the formula (-X) can be used directly. T X+γg θ (L))(-X T X+γg θ (L))T The principal component matrix is ​​obtained. Where g θ (L) represents the saved trained spectral filter. The principal component eigenvalues ​​of the target user can be obtained by selecting the eigenvectors corresponding to the first d largest eigenvalues ​​from the principal component matrix.

[0119] Step 104: Determine the target cluster of the target user based on the multiple principal component feature values ​​of the feature vector.

[0120] The process of grouping similar users into the same category is called clustering. Target clusters for target users can be determined in various ways, such as partitioning clustering, density-based clustering, and the OPTICS clustering method.

[0121] Step 105: Based on the target cluster, process the target users accordingly.

[0122] The processing for target users can be configured according to specific needs. For example, target users can be recommended, verified, or promoted.

[0123] This invention provides a novel feature mapping method constructed using spectral filters and principal component analysis, as shown in the following specific form: As shown. Where X represents the eigenvector, U represents the principal direction matrix, represents the projection transformation, Y is the principal component matrix, and g... θ (L) represents the spectral filter. By passing the spectral filter through the user data, multi-order local structural relationships can be obtained, enabling the construction of richer nearest neighbor relationships between samples, thus compensating for the poor robustness of the principal component analysis algorithm. Therefore, the scheme of this embodiment, by combining the spectral filter algorithm and the principal component analysis algorithm, obtains principal component eigenvalues ​​that can effectively describe the user's feature information.

[0124] Furthermore, based on the user's principal component eigenvalues, the system accurately determines the target cluster corresponding to the user, identifies the user's feature information through the target cluster, and processes the user accordingly, thereby improving the user's service experience.

[0125] In one embodiment of the present invention, the user data is pixels in a user image; acquiring multiple user data of a target user includes: receiving a verification request for the target user and acquiring a verification image corresponding to the verification request; performing image processing on the verification image to obtain a user image of a preset size; acquiring multiple pixels in the user image; constructing a feature vector of the target user based on the multiple user data includes: combining multiple pixels to generate a feature vector of the target user; performing corresponding processing on the target user based on a target cluster includes: determining whether the target cluster corresponds to the target user and obtaining a determination result; and performing verification processing on the verification request of the target user based on the determination result.

[0126] Specifically, if the judgment result indicates that the target cluster corresponds to the target user, the verification request of the target user is determined to pass the verification; if the judgment result indicates that the target cluster does not correspond to the target user, the verification request of the target user is determined to fail the verification.

[0127] The verification image is processed to obtain a user image of a preset size, so that the dimension of the generated feature vector is a preset value. Assuming that the user image has k pixels, if the user image is a color image, the dimension of the generated feature vector is 1*k. Each element in the feature vector consists of 3 values, which correspond to the three RGB channels respectively.

[0128] Before processing the verification image, it can be first processed into grayscale to generate a corresponding grayscale image. This grayscale image is then processed to obtain a user image of a preset size. Assuming the user image has k pixels, if it is a grayscale image, the generated feature vector will have a dimension of 1*k, and each element in the feature vector will consist of a single numerical value. This method reduces system verification time and improves system verification efficiency.

[0129] For user images, the system pre-defines multiple clusters, with different clusters corresponding to different users. Images of the same user are grouped into the same cluster. If the judgment result indicates that the target cluster corresponds to the target user, then all feature vectors in the target cluster correspond to images of the target user, and the user corresponding to the verification image is also the target user. In this case, the verification request for the target user is confirmed to be successful.

[0130] The system retains user image information for various verification and security checks. However, images acquired during daily use can become corrupted due to noise, occlusion, and other issues, impacting the verification process. The method described in this invention combines a spectral filter algorithm and principal component analysis, reducing the negative impact of noise and occlusion on verification and further improving accuracy.

[0131] Figure 2 This is a flowchart illustrating a tag data processing method provided in the second embodiment of the present invention, as shown below. Figure 2 As shown, the method includes:

[0132] Step 201: Obtain multiple user tags for the target user.

[0133] User tags can come from one or more application systems. Multiple user tags often result in redundancy, duplication, and inaccuracy.

[0134] Step 202: Determine the vector representation corresponding to each user tag.

[0135] Step 203: Combine the vector representations corresponding to each user label to generate the feature vector of the target user.

[0136] Assuming the target has k user labels, the vector representations corresponding to each user label are combined sequentially to generate the feature vector of the target user, which has a dimension of 1*k.

[0137] Step 204: Determine the target spectrogram filter. Using the target spectrogram filter, determine the eigenvalues ​​of multiple principal components of the eigenvector. The filter parameters of the target spectrogram filter are obtained by training using the principal component analysis algorithm.

[0138] The trained spectrogram filter can be saved and used as the target spectrogram filter. Alternatively, the target spectrogram filter can be generated based on the filter parameters and the Laplacian matrix. The Laplacian matrix is ​​reconstructed based on the target user's feature vector.

[0139] By using the target spectrum filter, the principal component matrix corresponding to the target user is generated. By selecting a preset number of the largest eigenvalues ​​from the principal component matrix, the principal component eigenvalues ​​of the target user can be obtained.

[0140] The feature vectors of target users are filtered through a spectral filter to obtain multi-order local structural relationships in user labels, which can construct richer neighbor relationships between samples, thereby making up for the poor robustness of the principal component analysis algorithm.

[0141] Step 205: Determine the target cluster of the target user based on the multiple principal component eigenvalues ​​of the feature vector.

[0142] Step 206: Based on the target cluster, determine the demand feature information corresponding to the target user.

[0143] Step 207: Perform recommendation processing based on the demand feature information of the target users.

[0144] Recommendation processing may include recommending product information, news information, and content information that correspond to the target user's needs and characteristics.

[0145] In this embodiment of the invention, as the operation and maintenance time of various application systems increases, multiple user tags often become redundant and duplicated. The solution provided by this embodiment can reduce the interference of redundant tag information, accurately determine the target user's category, and provide recommendations that meet the user's needs.

[0146] Figure 3 This is a flowchart illustrating a method for determining filter parameters according to the third embodiment of the present invention. Figure 3 As shown, the method includes:

[0147] Step 301: Obtain multiple user data groups, each user data group including: k user data and cluster labels.

[0148] User data can include user tags, pixels in user images, user preference data, user behavior data, etc. Cluster labels are used to characterize the clusters corresponding to groups of user data.

[0149] Step 302: Generate feature matrices corresponding to multiple user data groups.

[0150] For each user data set: determine the vector representation of each user data in the user data set; generate the feature vector of the user data set based on the vector representation of each user data; combine the feature vectors of each user data set to generate feature matrices corresponding to multiple user data sets.

[0151] Suppose there are n user datasets, each consisting of k user data points. For each user dataset, combine the vector representations of the k user data points to generate a feature vector of dimension 1*k. Combine the feature vectors of the n user datasets to generate a feature matrix of dimension n*k.

[0152] Step 303: Determine the parameters of the candidate filters, and construct the spectral filter corresponding to the feature matrix based on the parameters of the candidate filters.

[0153] It can randomly generate alternative filter parameters, or manually set filter parameters based on experience.

[0154] Step 304: Based on the spectral filter, obtain the principal component matrix corresponding to the feature matrix.

[0155] This invention utilizes a novel feature mapping method constructed using spectral filters and principal component analysis, as detailed below: As shown. Where X represents the eigenvector, U represents the principal direction matrix, represents the projection transformation, Y is the principal component matrix, and g...θ (L) is the spectral filter. The top d largest eigenvalues ​​from the principal component matrix are selected as the principal component eigenvalues.

[0156] Step 305: Based on the principal component matrix, determine the d principal component eigenvalues ​​corresponding to each user data group, where d is less than k.

[0157] By combining spectral filtering and principal component analysis algorithms, feature mapping is performed to map k user data points in a user dataset into d principal component eigenvalues, thus preserving some key feature dimensions of each user dataset in a low-dimensional space. These key feature dimensions are then used to cluster multiple user datasets.

[0158] Step 306: Cluster multiple user data groups based on the d principal component feature values ​​corresponding to the user data groups.

[0159] Step 307: Determine the clustering accuracy based on the clustering results and clustering labels of multiple user data groups.

[0160] Step 308: Determine whether the clustering accuracy is greater than the preset threshold.

[0161] The preset threshold can be set according to requirements. If the clustering accuracy is greater than the preset threshold, the current filter parameters meet the requirements, and step 309 is executed. If the clustering accuracy is not greater than the preset threshold, the current filter parameters do not meet the requirements, and training needs to continue, and step 310 is executed.

[0162] Step 309: Determine the candidate filter parameters as the target filter parameters.

[0163] Step 310: Modify the filter parameters, and based on the modified filter parameters, reconstruct the spectral filter corresponding to the feature matrix; determine the clustering accuracy corresponding to the reconstructed spectral filter.

[0164] Repeat step 308. Determine if the current filter parameters meet the requirements by assessing the clustering accuracy. If the current filter parameters do not meet the requirements, continue training until the clustering accuracy corresponding to the current filter parameters is greater than a preset threshold. Then, determine the current filter parameters as the target filter parameters.

[0165] It should be noted that the target filter parameters generated in the embodiments of the present invention are the same filter parameters used in the first and second embodiments of the present invention. A target spectrogram filter is constructed using these filter parameters to determine multiple principal component eigenvalues ​​of the feature vector.

[0166] The solution of this invention uses a spectral filter in the principal component analysis method. User data can be reconstructed into a weighted combination of multi-order graph structure data through the spectral filter, constructing richer nearest neighbor relationships between samples, thereby better describing the data, coping with a wider range of application scenarios, and improving the robustness of the principal component analysis method.

[0167] The fourth embodiment of the present invention also provides a method for determining filter parameters, the method comprising:

[0168] Step 01: Construct the dataset: For k labels for each user, each user represents one row of data, and n users form an R dataset. n×k A two-dimensional matrix. For user photo information, the two-dimensional photo data is stretched and transformed into a one-dimensional vector, and n users form an R matrix. n×k A two-dimensional matrix.

[0169] Step 02: Construct a test dataset: Select a portion of user data with clustering labels from the dataset to form a test dataset, which is used to verify the performance of the spectrogram filter and determine whether the filter parameters are suitable.

[0170] Step 03: Construct the Laplacian matrix: First, construct the nearest neighbor graph of the test dataset, then construct the adjacency matrix W based on the nearest neighbor graph to obtain the diagonal matrix D, and finally calculate the Laplacian matrix according to the formula L = DW.

[0171] Step 04: Construct the spectral filter: Randomly generate filter parameters and, using the Laplacian matrix, construct the spectral filter g. θ (L).

[0172] Step 05, Feature Mapping: Using the formula Transform it into the formula (-X) T X+γg θ The problem of solving (L))=μY. Where X represents the eigenvectors, U represents the principal direction matrix, g represents the projection transformation, Y is the principal component matrix, and g... θ (L) is the spectral filter.

[0173] By calculating the principal component matrix (-X) T X+γg θ (L))(-X T X+γg θ (L)) T By selecting the eigenvectors corresponding to the first preset number of largest eigenvalues, the principal component eigenvalues ​​of the mapped data can be obtained.

[0174] Step 6: Using the obtained principal component feature values, clustering or classification tasks can be performed, and the clustering accuracy can be calculated. If the clustering accuracy is not greater than a preset threshold, the parameters are adjusted and the filter is reconstructed. If the clustering accuracy is greater than the preset threshold, the current filter parameters are recorded.

[0175] Step 7: For newly input target user data, if the target user data has a new category, reconstruct the Laplacian matrix and construct a spectral filter using the filter parameters obtained from cross-validation. Then, perform feature mapping to obtain the principal component matrix. If the target user data's category includes existing categories, the principal component matrix can be directly obtained using the formula.

[0176] Step 08: Use the principal component matrix to obtain multiple principal component feature values ​​of the target user.

[0177] The solution in this invention utilizes a spectroscopic filter in principal component analysis, a method with closed-form solutions that requires minimal training. User data, processed by the spectroscopic filter, can be reconstructed into a weighted combination of multi-order graph structure data. Compared to the traditional Laplacian matrix, this constructs richer nearest neighbor relationships between samples, thus better describing the user data. The spectroscopic filter feature mapping is more robust to data corruption, handling a wider range of applications such as face recognition, and also enhancing the robustness of the feature mapping process.

[0178] Figure 4 This is a schematic diagram of the structure of a data processing device provided in one embodiment of the present invention, as shown below. Figure 4 As shown, the device includes:

[0179] Data acquisition module 401 is used to acquire multiple user data of the target user;

[0180] Vector construction module 402 is used to construct the feature vector of the target user based on the multiple user data;

[0181] The eigenvalue determination module 403 is used to determine the target spectrum filter, and to determine multiple principal component eigenvalues ​​of the feature vector using the target spectrum filter. The filter parameters of the target spectrum filter are obtained by training using a principal component analysis algorithm.

[0182] Clustering module 404 is used to determine the target cluster of the target user based on multiple principal component feature values ​​of the feature vector;

[0183] The processing module 405 is used to perform corresponding processing on the target user based on the target cluster.

[0184] Optionally, the user data is user tags;

[0185] The data acquisition module 401 is specifically used for:

[0186] Obtain multiple user tags of the target user;

[0187] The vector construction module 402 is specifically used for:

[0188] Determine the vector representation corresponding to each of the user tags;

[0189] By combining the vector representations corresponding to each user tag, a feature vector of the target user is generated;

[0190] The processing module 405 is specifically used for:

[0191] Based on the target clustering, determine the demand characteristic information corresponding to the target user;

[0192] Recommendation processing is performed on the target user based on the required feature information.

[0193] Optionally, the user data consists of pixels in a user image;

[0194] The data acquisition module 401 is specifically used for:

[0195] Receive a verification request for the target user and obtain the verification image corresponding to the verification request;

[0196] The verification image is processed to obtain a user image of a preset size;

[0197] Obtain multiple pixels from the user image;

[0198] The vector construction module 402 is specifically used for:

[0199] The multiple pixels are combined to generate the feature vector of the target user;

[0200] The processing module 405 is used for:

[0201] Determine whether the target cluster corresponds to the target user, and obtain the determination result;

[0202] Based on the judgment result, the verification request of the target user is verified.

[0203] Optionally, the processing module 405 is specifically used for:

[0204] In response to the determination result indicating that the target cluster corresponds to the target user, it is determined that the target user's verification request has passed verification;

[0205] In response to the judgment result indicating that the target cluster does not correspond to the target user, it is determined that the verification request of the target user has failed verification.

[0206] Optionally, it also includes:

[0207] Training module 406 is used to acquire multiple user data groups, each of which includes: k user data and clustering labels;

[0208] Generate feature matrices corresponding to the multiple user data groups;

[0209] Determine the parameters of the candidate filters, and construct the spectral filter corresponding to the feature matrix based on the candidate filter parameters;

[0210] Based on the spectral filter, the principal component matrix corresponding to the feature matrix is ​​obtained;

[0211] Based on the principal component matrix, determine d principal component eigenvalues ​​corresponding to each user data group, where d is less than k;

[0212] Cluster the multiple user data groups based on the d principal component feature values ​​corresponding to the user data groups;

[0213] The target filter parameters are determined based on the clustering results of the multiple user data groups.

[0214] Optionally, the training module 406 is specifically used for:

[0215] The clustering accuracy is determined based on the clustering results and clustering labels of the multiple user data groups.

[0216] In response to the clustering accuracy being greater than a preset threshold, the candidate filter parameters are determined as the target filter parameters.

[0217] Optionally, the training module 406 is specifically used for:

[0218] In response to the clustering accuracy not being greater than the preset threshold, the filter parameters are modified;

[0219] Based on the modified filter parameters, the spectral filter corresponding to the feature matrix is ​​reconstructed.

[0220] The clustering accuracy corresponding to the reconstructed spectrogram filter is determined. If the clustering accuracy corresponding to the reconstructed spectrogram filter is not greater than the preset threshold, the filter parameters are further modified.

[0221] The current filter parameter is determined as the target filter parameter until the clustering accuracy corresponding to the current filter parameter is greater than the preset threshold.

[0222] Optionally, the training module 406 is specifically used for:

[0223] For each user data group: determine the vector representation corresponding to each user data in the user data group; generate the feature vector of the user data group based on the vector representation corresponding to each user data;

[0224] The feature vectors of each user data group are combined to generate a feature matrix corresponding to the multiple user data groups.

[0225] This invention provides an electronic device, comprising:

[0226] One or more processors;

[0227] Storage device for storing one or more programs.

[0228] When one or more programs are executed by one or more processors, the one or more processors implement the methods of any of the above embodiments.

[0229] This invention provides a computer program product, including a computer program that, when executed by a processor, implements the enterprise risk assessment method of this invention.

[0230] The following is for reference. Figure 5 It shows a schematic diagram of the structure of a computer system 500 suitable for implementing a terminal device of the present invention. Figure 5 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0231] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 502 or programs loaded from storage section 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the system 500. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0232] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 510 as needed so that computer programs read from it can be installed into storage section 508 as needed.

[0233] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs the functions defined above in the system of this invention.

[0234] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0235] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0236] The modules described in the embodiments of the present invention can be implemented in software or hardware. The described modules can also be housed in a processor, and for example, can be described as: a data acquisition module, a vector construction module, a feature value determination module, a clustering module, and a processing module. The names of these modules do not necessarily limit the module itself; for example, the data acquisition module can also be described as "a module for acquiring multiple user data of a target user".

[0237] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to include:

[0238] Obtain multiple user data points for the target user;

[0239] Based on the multiple user data, construct the feature vector of the target user;

[0240] A target spectrogram filter is determined, and multiple principal component eigenvalues ​​of the feature vector are determined using the target spectrogram filter. The filter parameters of the target spectrogram filter are obtained by training using a principal component analysis algorithm.

[0241] The target cluster of the target user is determined based on the multiple principal component feature values ​​of the feature vector.

[0242] Based on the target cluster, the target user is processed accordingly.

[0243] According to the technical solution of this invention, a target spectrogram filter is used to determine multiple principal component feature values ​​of a target user. The filter parameters of the target spectrogram filter are obtained by training using a principal component analysis algorithm. By passing the spectrogram filter through the user data, multi-order local structural relationships in the user data can be obtained, which can construct richer nearest neighbor relationships between samples, thereby compensating for the poor robustness of the principal component analysis algorithm. Therefore, the principal component feature values ​​obtained by combining the spectrogram filter algorithm and the principal component analysis algorithm through the solution of this invention can effectively describe the user's feature data.

[0244] Furthermore, based on the user's principal component eigenvalues, the system accurately determines the target cluster corresponding to the user, identifies the user's feature information through the target cluster, and processes the user accordingly, thereby improving the user's service experience.

[0245] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A data processing method, characterized in that, include: Acquiring multiple user data of a target user includes: receiving a verification request for the target user; acquiring a verification image corresponding to the verification request; performing image processing on the verification image to obtain a user image of a preset size; and acquiring multiple pixels in the user image; wherein the user data is the pixels in the user image. Constructing a feature vector for the target user based on the multiple user data includes: combining the multiple pixels to generate the feature vector for the target user; A target spectrogram filter is determined, and multiple principal component eigenvalues ​​of the feature vector are determined using the target spectrogram filter. The filter parameters of the target spectrogram filter are obtained by training using a principal component analysis algorithm. The target cluster of the target user is determined based on the multiple principal component feature values ​​of the feature vector. Based on the target cluster, the target user is processed accordingly, including: determining whether the target cluster corresponds to the target user, and obtaining a determination result; and processing the verification request of the target user based on the determination result. It also includes: performing recommendation and promotion processing on the target users based on the target cluster; Specifically, for user images: multiple clusters are preset, with different clusters corresponding to different users, and images of the same user are classified into the same cluster; Before acquiring multiple user data sets of the target user, the method further includes: acquiring multiple user data groups, each user data group including: k user data sets and clustering labels; generating feature matrices corresponding to the multiple user data groups; determining candidate filter parameters; constructing a spectral filter corresponding to the feature matrix based on the candidate filter parameters; obtaining the principal component matrix corresponding to the feature matrix based on the spectral filter; determining d principal component eigenvalues ​​corresponding to each user data group based on the principal component matrix, where d is less than k; clustering the multiple user data groups based on the d principal component eigenvalues ​​corresponding to the user data groups; and determining target filter parameters based on the clustering results of the multiple user data groups. The step of determining the target filter parameters based on the clustering results of the multiple user data groups includes: determining the clustering accuracy based on the clustering results of the multiple user data groups and the clustering labels of the multiple user data groups; and determining the candidate filter parameters as the target filter parameters in response to the clustering accuracy being greater than a preset threshold. After determining the clustering accuracy based on the clustering results and clustering labels of the multiple user data, the method further includes: modifying the filter parameters in response to the clustering accuracy not being greater than the preset threshold; reconstructing the spectral filter corresponding to the feature matrix based on the modified filter parameters; determining the clustering accuracy corresponding to the reconstructed spectral filter; and continuing to modify the filter parameters in response to the clustering accuracy corresponding to the reconstructed spectral filter not being greater than the preset threshold; until the clustering accuracy corresponding to the current filter parameters is greater than the preset threshold, and then determining the current filter parameters as the target filter parameters. It also includes: before performing image processing on the verification image, performing grayscale processing on the verification image to generate a grayscale image corresponding to the verification image; and performing image processing on the grayscale image to obtain a user image of a preset size.

2. The method according to claim 1, characterized in that, The user data is user tags; The acquisition of multiple user data of the target user includes: Obtain multiple user tags of the target user; The step of constructing the feature vector of the target user based on the multiple user data includes: Determine the vector representation corresponding to each of the user tags; By combining the vector representations corresponding to each user tag, a feature vector of the target user is generated; The step of processing the target user according to the target cluster includes: Based on the target clustering, determine the demand characteristic information corresponding to the target user; Recommendation processing is performed on the target user based on the required feature information.

3. The method according to claim 1, characterized in that, The step of verifying the target user based on the judgment result includes: In response to the determination result indicating that the target cluster corresponds to the target user, it is determined that the target user's verification request has passed verification; In response to the judgment result indicating that the target cluster does not correspond to the target user, it is determined that the verification request of the target user has failed verification.

4. The method according to claim 1, characterized in that, Generating the feature matrix corresponding to the multiple user data groups includes: For each user data group: determine the vector representation corresponding to each user data in the user data group; generate the feature vector of the user data group based on the vector representation corresponding to each user data; The feature vectors of each user data group are combined to generate a feature matrix corresponding to the multiple user data groups.

5. A data processing apparatus, characterized in that, include: The data acquisition module is used to acquire multiple user data of the target user; A vector construction module is used to construct a feature vector of the target user based on the multiple user data. The eigenvalue determination module is used to determine the target spectrogram filter, and using the target spectrogram filter, to determine multiple principal component eigenvalues ​​of the eigenvector. The filter parameters of the target spectrogram filter are obtained by training using a principal component analysis algorithm. The clustering module is used to determine the target cluster of the target user based on the multiple principal component feature values ​​of the feature vector; The processing module is used to perform corresponding processing on the target user based on the target cluster; The user data consists of pixels in the user's image. The data acquisition module is specifically used for: receiving a verification request for the target user; acquiring a verification image corresponding to the verification request; performing image processing on the verification image to obtain a user image of a preset size; and acquiring multiple pixels in the user image. The vector construction module is specifically used to: combine the multiple pixels to generate the feature vector of the target user; The processing module is used to: determine whether the target cluster corresponds to the target user and obtain a determination result; perform verification processing on the verification request of the target user according to the determination result; for user images: preset multiple clusters, different clusters correspond to different users, and images of the same user are classified into the same cluster; The processing module is specifically used to: perform recommendation and promotion processing on the target users based on the target cluster; The data acquisition module is specifically used for: performing grayscale processing on the verification image before image processing on the verification image to generate a grayscale image corresponding to the verification image; performing image processing on the grayscale image to obtain a user image of a preset size; It also includes: a training module for acquiring multiple user data groups, each user data group including: k user data and clustering labels; generating feature matrices corresponding to the multiple user data groups; determining candidate filter parameters; constructing a spectroscopic filter corresponding to the feature matrix based on the candidate filter parameters; obtaining a principal component matrix corresponding to the feature matrix based on the spectroscopic filter; determining d principal component eigenvalues ​​corresponding to each user data group based on the principal component matrix, where d is less than k; clustering the multiple user data groups based on the d principal component eigenvalues ​​corresponding to the user data groups; and determining target filter parameters based on the clustering results of the multiple user data groups. The training module is specifically used to: determine the clustering accuracy based on the clustering results and clustering labels of the multiple user data groups; and, in response to the clustering accuracy being greater than a preset threshold, determine the candidate filter parameters as the target filter parameters. The training module is specifically used for: modifying the filter parameters in response to the clustering accuracy not being greater than the preset threshold; reconstructing the spectral filter corresponding to the feature matrix based on the modified filter parameters; determining the clustering accuracy corresponding to the reconstructed spectral filter; and continuing to modify the filter parameters in response to the clustering accuracy corresponding to the reconstructed spectral filter not being greater than the preset threshold; until the clustering accuracy corresponding to the current filter parameters is greater than the preset threshold, and determining the current filter parameters as the target filter parameters.

6. The apparatus according to claim 5, characterized in that, The user data is user tags; The data acquisition module is specifically used for: Obtain multiple user tags of the target user; The vector construction module is specifically used for: Determine the vector representation corresponding to each of the user tags; By combining the vector representations corresponding to each user tag, a feature vector of the target user is generated; The processing module is specifically used for: Based on the target clustering, determine the demand characteristic information corresponding to the target user; Recommendation processing is performed on the target user based on the required feature information.

7. The apparatus according to claim 5, characterized in that, The processing module is specifically used for: In response to the determination result indicating that the target cluster corresponds to the target user, it is determined that the target user's verification request has passed verification; In response to the judgment result indicating that the target cluster does not correspond to the target user, it is determined that the verification request of the target user has failed verification.

8. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-4.

9. A computer-readable medium having a computer program stored thereon, characterized in that... When the program is executed by the processor, it implements the method as described in any one of claims 1-4.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-4.

Images