User information data processing method and device, electronic equipment and medium
By filtering and clustering data in user information data processing and filling in missing fields, the problem of inaccurate user preference analysis caused by missing fields is solved, and more accurate and efficient data analysis is achieved.
Patent Information
- Application Number
- CN202411478339.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-22
- Publication Date
- 2025-02-18
AI Technical Summary
The prior art is difficult to conduct accurate user preference analysis in the absence of fields, resulting in complex data integration and analysis, affecting the availability and quality of data.
By obtaining the required target fields, filtering the data in the database, removing the target fields to build the third data, and clustering the second data and the third data, determining the target fields contained in the first data and filling it into the second data, Form a complete data set for user preference analysis.
User preference analysis in the case of missing fields is realized, the accuracy and efficiency of data processing are improved, and the analysis inaccuracy caused by missing data is avoided.
Smart Images
Figure CN119474529A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, electronic device and medium for processing user information data. Background Art
[0002] In the current technological environment, it is common for user information obtained through various channels to have inconsistent fields, which poses a significant challenge to data integration and analysis. Data collection from different channels may lead to fragmented information. For example, channel A may provide the user's age, interests, and purchasing power, but lack gender information; while channel B may provide gender and occupation, but no contact information. This inconsistency in fields not only complicates the construction of the overall user portrait, but also affects the availability and quality of the data.
[0003] When companies conduct data analysis and market forecasting, if they do not effectively handle this inconsistent information, it may lead to inaccurate analysis results. For example, if an e-commerce platform ignores gender information when analyzing user purchasing behavior, it may incorrectly assess the target market of certain products, such as believing that football-related products are mainly popular among men and ignoring the potential needs of female consumers. In addition, the lack of key contact information may also lead to inefficient marketing activities because it is impossible to accurately contact and communicate with target customers.
[0004] Therefore, a method for performing preference analysis in the presence of missing fields is needed. Summary of the invention
[0005] Therefore, the present invention provides a method, device, electronic device and medium for processing user information data to solve the problem in the prior art of how to perform accurate user preference analysis when fields are missing.
[0006] The present invention provides a method for processing user information data, comprising:
[0007] Get the fields required for preference analysis as target fields;
[0008] The data including the target field is filtered out from the database and is called the first data, and the data excluding the target field is called the second data;
[0009] Eliminate the target field in the first data to construct the third data; cluster the second data and the third data to obtain multiple clusters, and mark the third data and the second data in each cluster;
[0010] According to the marked third data in each cluster, determine the target field contained in the corresponding first data, and fill the target field into the second data in the same cluster to obtain the target second data;
[0011] Perform user preference analysis based on the first data and the target second data.
[0012] Furthermore, clustering the second data and the third data to obtain a plurality of clusters includes:
[0013] Clustering the first data using a preset hierarchical clustering method to obtain a clustering tree;
[0014] According to the characteristic branches of the clustering tree, the target K value of K-means clustering is determined;
[0015] Constructing a target K-means clustering algorithm with the target K value;
[0016] The first data is clustered using a target K-means clustering algorithm to obtain a plurality of clusters.
[0017] Furthermore, the target K-means clustering algorithm is used to cluster the first data to obtain a plurality of clusters, including:
[0018] Obtaining an average similarity between each data object and other objects in the first data, and determining data objects whose average similarity is greater than a preset density threshold as core objects;
[0019] Select multiple data objects with the lowest average similarity between each other from the core objects as the initial clustering centers of the K-means clustering algorithm;
[0020] The first data is clustered based on the initial cluster center to obtain a plurality of clusters.
[0021] Furthermore, the average similarity between each data object and other objects in the first data can be expressed by the following formula:
[0022]
[0023] Among them, S represents the average similarity, a k and b k Respectively represent the weight of the kth feature item in data a and b, n represents the number of feature items in a and b, t a It represents the number of times the kth eigenvalue appears in a, and φ is the similarity adjustment coefficient.
[0024] Furthermore, after determining the target field included in the corresponding first data according to the third data marked in each cluster, the method further includes:
[0025] Clustering the first data to obtain multiple clusters, and marking the cluster with the largest amount of data; obtaining a first filling value according to the value of the target field in the cluster with the largest amount of data;
[0026] According to the first filling value, the target field included in the first data in each cluster is modified to obtain a target filling value;
[0027] The target filling value is filled into the second data in the same cluster to obtain the target second data.
[0028] Furthermore, the method of modifying the target field included in the first data in each cluster according to the first filling value to obtain the target filling value includes:
[0029] The preset K-nearest neighbors are used to modify the target field according to the first fill value.
[0030] Further, filling the target filling value into the second data in the same cluster to obtain the target second data includes:
[0031] Determine the center value of the cluster in the target fill value;
[0032] The cluster center value is filled into the second data to obtain the target second data.
[0033] The present invention also provides a user information data processing device, comprising:
[0034] The acquisition module is used to obtain the fields required for preference analysis as target fields;
[0035] A data classification module is used to screen out data including a target field from a database, which is referred to as first data, and data not including the target field, which is referred to as second data;
[0036] A clustering module, used to remove the target field in the first data to construct the third data; cluster the second data and the third data to obtain a plurality of clusters, and mark the third data and the second data in each cluster;
[0037] A filling module, used for determining a target field included in the corresponding first data according to the third data marked in each cluster, and filling the target field into the second data in the same cluster to obtain target second data;
[0038] The preference analysis module is used to perform user preference analysis based on the first data and the target second data.
[0039] The present invention also provides an electronic device, comprising:
[0040] Memory and processor;
[0041] The memory is used to store programs, and the processor is used to execute the steps in any one of the above-mentioned methods for processing user information data when executing the program.
[0042] The present invention also provides a computer-readable storage medium for storing computer-readable programs or instructions. When the program or instructions are executed by a processor, the steps in the method for processing user information data in any of the above-mentioned implementation modes can be implemented.
[0043] The beneficial effects of adopting the above embodiment are: first, the fields required for preference analysis are obtained as target fields; by determining the target fields, the user's preferences can be better analyzed, and then the data including the target fields are screened out from the database as the first data, and the data not including the target fields are called the second data; the target fields in the first data are removed to construct the third data; the second data and the third data are clustered to obtain multiple clusters, and the third data and the second data in each cluster are marked; thereby, the potential relationship between the second data and the third data is explored, and the user's preference settings are better understood through similarity clustering. Then, according to the marked third data in each cluster, the target field contained in the corresponding first data is determined, and the target field is filled into the second data in the same cluster to obtain the target second data; thereby completing the filling of missing data and avoiding the problem of inaccurate user preference analysis caused by missing data. Finally, user preference analysis is performed based on the first data and the target second data. Thus, the accuracy and efficiency of data processing are improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 A method flow chart of an embodiment of a method for processing user information data provided by the present invention;
[0045] Figure 2 for Figure 1 A method flow chart of an embodiment of step S103;
[0046] Figure 3 A flow chart of a method for clustering first data according to an embodiment;
[0047] Figure 4 It is a schematic diagram of an embodiment of a device for selecting an optimal anchor position for a ship provided by the present invention;
[0048] Figure 5 It is a schematic diagram of the operating environment of an electronic device provided by the present invention. DETAILED DESCRIPTION
[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0050] The present invention provides a method for processing user information data. Figure 1 ,include:
[0051] S101, obtaining the fields required for preference analysis as target fields;
[0052] S102, filtering out data including a target field from a database as first data, and filtering out data not including the target field as second data;
[0053] S103, removing the target field in the first data to construct the third data; clustering the second data and the third data to obtain a plurality of clusters, and marking the third data and the second data in each cluster;
[0054] S104, determining a target field included in the corresponding first data according to the third data marked in each cluster, and filling the target field into the second data in the same cluster to obtain target second data;
[0055] S105: Perform user preference analysis based on the first data and the target second data.
[0056] In this embodiment, the fields required for preference analysis are first obtained as target fields; by determining that the target field can better understand the user's preferences, the data including the target field is then screened out from the database as the first data, and the data not including the target field is called the second data; the target field in the first data is removed to construct the third data; the second data and the third data are clustered to obtain multiple clusters, and the third data and the second data in each cluster are marked; thereby exploring the potential relationship between the second data and the third data, and better understanding the user's preference settings through similarity clustering. Subsequently, according to the marked third data in each cluster, the target field contained in the corresponding first data is determined, and the target field is filled into the second data in the same cluster to obtain the target second data; thereby completing the filling of missing data and avoiding the problem of inaccurate user preference analysis caused by missing data. Finally, user preference analysis is performed based on the first data and the target second data. Thus, the accuracy and efficiency of data processing are improved.
[0057] It should be noted that the target field is determined based on the actual user preference analysis requirements, including but not limited to the following aspects:
[0058] Basic personal information: such as name, gender, age, geographic location, etc. This information can help distinguish different user groups and better understand their preferences and behaviors.
[0059] Consumption behavior data: including purchase records, consumption frequency, consumption amount, etc. These data can reflect users' consumption preferences and purchasing behaviors, and help with personalized recommendations and marketing strategies.
[0060] Browsing behavior data: including browsing history, number of clicks, length of stay, etc. These data can reveal users’ interests and focus, and provide a basis for content recommendations and advertising.
[0061] Interactive behavior data: User interactive behaviors on the platform, such as comments, sharing, and likes, can reflect the user's level of participation and social activity, which is particularly important for social platforms and content creators.
[0062] Preference tags or tag systems: including custom tags or system-generated tags, which are used to describe user interests, preferences, likes and other characteristics, and are an important basis for user analysis and recommendation systems.
[0063] In step S102, if the target field is gender, the data set containing gender information is used as the first data, and the data set without gender information is used as the second data. Alternatively, the data set with age information is used as the first data, and the data set without age information is used as the second data.
[0064] In step S103, by clustering the second data and the third data, similarity clusters of the two groups of data except for the target field can be obtained, so that data with similar information expressed by the first data but lacking the target field can be screened out, and complete data can be obtained by filling the target field contained in the corresponding first data into the second data, thereby avoiding the problem of inaccurate user preference analysis due to missing key information in the data.
[0065] In some embodiments, the second data and the third data are clustered to obtain a plurality of clusters. Figure 2 ,include:
[0066] S201, clustering the first data using a preset hierarchical clustering method to obtain a clustering tree;
[0067] S202, determining a target K value for K-means clustering according to the characteristic branches of the clustering tree;
[0068] S203, constructing a target K-means clustering algorithm with the target K value;
[0069] S204: clustering the first data using a target K-means clustering algorithm to obtain a plurality of clusters.
[0070] In this embodiment, the first data is first clustered using a preset hierarchical clustering method to obtain a cluster tree. Hierarchical clustering is a clustering method that organizes data hierarchically. It constructs a hierarchical structure by calculating the similarity between data points, and finally forms a tree structure. Next, the target K value of K-means clustering is determined based on the characteristic branches of the clustering tree. By analyzing the structure of the clustering tree, the appropriate number of clusters can be determined. The target K-means clustering algorithm is constructed with the determined target K value. Through this process, the first data can be divided into multiple clusters, so as to better understand the intrinsic structure and pattern of the data, and provide a basis for subsequent preference analysis and personalized recommendations.
[0071] It should be noted that in hierarchical clustering, the clustering process is gradual, and data points gradually merge into larger and larger clusters until a cluster containing all data points is formed. If K-means clustering is used directly, the number of clusters K needs to be determined in advance, while hierarchical clustering can select an appropriate K value by observing the structure of the clustering tree, and a more reasonable number of clusters can be obtained, which can better divide the data and avoid over-clustering or under-clustering, thereby more accurately reflecting the internal structure and pattern of the data.
[0072] It is understandable that the K value may also be determined by means of silhouette coefficient or interval statistics.
[0073] In some embodiments, clustering the first data using a target K-means clustering algorithm to obtain a plurality of clusters includes:
[0074] Obtaining an average similarity between each data object and other objects in the first data, and determining data objects whose average similarity is greater than a preset density threshold as core objects;
[0075] Select multiple data objects with the lowest average similarity between each other from the core objects as the initial clustering centers of the K-means clustering algorithm;
[0076] The first data is clustered based on the initial cluster center to obtain a plurality of clusters.
[0077] In this embodiment, by calculating the average similarity between data, and taking data objects with average similarity greater than a preset density threshold as core objects, and selecting the most dissimilar points from the core objects as clustering centers, not only can the adverse effect of the initial clustering center selecting isolated points that may cause the clustering results to fall into the local optimum be eliminated, but also the accuracy and stability of preference data clustering are improved, and user characteristics can be quickly and accurately discovered from a large amount of data.
[0078] The average similarity between each data object and other objects in the first data can be expressed by the following formula:
[0079]
[0080] Among them, S represents the average similarity, a k and b k Respectively represent the weight of the kth feature item in data a and b, n represents the number of feature items in a and b, t a represents the number of times the kth eigenvalue appears in a, is the similarity adjustment coefficient.
[0081] In some embodiments, see Figure 3 , after determining the target field included in the corresponding first data according to the third data marked in each cluster, it also includes:
[0082] S301, clustering the first data to obtain multiple clusters, and marking the cluster with the largest amount of data; obtaining a first filling value according to the value of the target field in the cluster with the largest amount of data;
[0083] S302: According to the first filling value, modify the target field included in the first data in each cluster to obtain a target filling value;
[0084] S303: Fill the target filling value into the second data in the same cluster to obtain the target second data.
[0085] In this embodiment, by clustering the first data and correcting the filling value of the second data based on the results of two clusterings, the accuracy of missing data filling can be improved.
[0086] It should be noted that in step S301, in the cluster with the largest amount of labeled data, females account for the majority, so females are the filler value. Or the average age is 27 years old, so 27 years old can be used as the filler value.
[0087] The target field included in the first data in each cluster is corrected according to the filling value obtained by clustering the first data to obtain the target filling value, which can improve the accuracy of clustering and filling.
[0088] In step S302, in order to ensure a better correction effect, the target field is corrected using a K-nearest neighbor method.
[0089] Furthermore, in step S303, the cluster center of the target filling value is first determined, and then the value corresponding to the cluster center is used as the target value and filled into the second data, thereby obtaining the target second data, thereby ensuring the objectivity and effectiveness of the filling.
[0090] Based on the above method for processing user information data, an embodiment of the present invention further provides a device for processing user information data. Figure 4 , including: an acquisition module 410, a data classification module 420, a clustering module 430, a filling module 440 and a preference analysis module 450.
[0091] An acquisition module 410 is used to acquire fields required for preference analysis as target fields;
[0092] A data classification module 420, for screening out data including a target field from a database as first data, and data not including the target field as second data;
[0093] The clustering module 430 is used to remove the target field in the first data to construct the third data; cluster the second data and the third data to obtain multiple clusters, and mark the third data and the second data in each cluster;
[0094] A filling module 440 is used to determine the target field included in the corresponding first data according to the third data marked in each cluster, and fill the target field into the second data in the same cluster to obtain the target second data;
[0095] The preference analysis module 450 is used to perform user preference analysis based on the first data and the target second data.
[0096] like Figure 5 As shown, based on the above method for processing user information data, the present invention also provides an electronic device, which can be a computing electronic device such as a mobile terminal, a desktop computer, a notebook, a palm computer, and a server. The electronic device includes a processor 510, a memory 520, and a display 530. Figure 5 Only some components of the electronic device are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.
[0097] In some embodiments, the memory 520 may be an internal storage unit of the electronic device, such as a hard disk or memory of the electronic device. In other embodiments, the memory 520 may also be an external storage electronic device of the electronic device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the memory 520 may also include both an internal storage unit of the electronic device and an external storage electronic device. The memory 520 is used to store application software and various types of data installed in the electronic device, such as program codes for installing the electronic device. The memory 520 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a user information data processing program 540 is stored on the memory 520, and the user information data processing program 540 can be executed by the processor 510, thereby realizing the user information data processing method of each embodiment of the present application.
[0098] In some embodiments, the processor 510 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run program codes or process data stored in the memory 520, such as executing a method for processing user information data.
[0099] In some embodiments, the display 530 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 530 is used to display information of the electronic device processing the user information data and to display a visual user interface. The components 510-530 of the electronic device communicate with each other via a system bus.
[0100] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for processing user information data, characterized in that: include: Get the fields required for preference analysis as target fields; The data including the target field is filtered out from the database and is called the first data, and the data excluding the target field is called the second data; Eliminate the target field in the first data to construct the third data; Clustering the second data and the third data to obtain a plurality of clusters, and marking the third data and the second data in each cluster; According to the marked third data in each cluster, determine the target field contained in the corresponding first data, and fill the target field into the second data in the same cluster to obtain the target second data; Perform user preference analysis based on the first data and the target second data.
2. The method for processing user information data according to claim 1, characterized in that: The clustering of the second data and the third data to obtain a plurality of clusters includes: Clustering the first data using a preset hierarchical clustering method to obtain a clustering tree; According to the characteristic branches of the clustering tree, the target K value of K-means clustering is determined; Constructing a target K-means clustering algorithm with the target K value; The first data is clustered using a target K-means clustering algorithm to obtain a plurality of clusters.
3. The method for processing user information data according to claim 2, characterized in that: The target K-means clustering algorithm is used to cluster the first data to obtain multiple clusters, including: Obtaining an average similarity between each data object and other objects in the first data, and determining data objects whose average similarity is greater than a preset density threshold as core objects; Select multiple data objects with the lowest average similarity between each other from the core objects as the initial clustering centers of the K-means clustering algorithm; The first data is clustered based on the initial cluster center to obtain a plurality of clusters.
4. The method for processing user information data according to claim 3, characterized in that: The average similarity between each data object and other objects in the first data can be expressed by the following formula: Among them, S represents the average similarity, a k and b k Respectively represent the weight of the kth feature item in data a and b, n represents the number of feature items in a and b, t a represents the number of times the kth eigenvalue appears in a, is the similarity adjustment coefficient.
5. The method for processing user information data according to claim 1, characterized in that: After determining the target field included in the corresponding first data according to the third data marked in each cluster, the method further includes: Clustering the first data to obtain multiple clusters, and marking the cluster with the largest amount of data; obtaining a first filling value according to the value of the target field in the cluster with the largest amount of data; According to the first filling value, the target field included in the first data in each cluster is modified to obtain a target filling value; The target filling value is filled into the second data in the same cluster to obtain the target second data.
6. The method for processing user information data according to claim 5, characterized in that: The method of modifying the target field included in the first data in each cluster according to the first filling value to obtain the target filling value includes: The preset K-nearest neighbors are used to modify the target field according to the first fill value.
7. The method for processing user information data according to claim 5, characterized in that: Filling the target filling value into the second data in the same cluster to obtain the target second data includes: Determine the center value of the cluster in the target fill value; The cluster center value is filled into the second data to obtain the target second data.
8. A user information data processing device, characterized in that: include: The acquisition module is used to obtain the fields required for preference analysis as target fields; A data classification module is used to screen out data including a target field from a database, which is referred to as first data, and data not including the target field, which is referred to as second data; A clustering module, used for removing the target field in the first data to construct the third data; Clustering the second data and the third data to obtain a plurality of clusters, and marking the third data and the second data in each cluster; A filling module, used for determining a target field included in the corresponding first data according to the third data marked in each cluster, and filling the target field into the second data in the same cluster to obtain target second data; The preference analysis module is used to perform user preference analysis based on the first data and the target second data.
9. An electronic device, characterized in that: include: Memory and processor; The memory is used to store programs, and the processor is used to execute the steps in the method for processing user information data according to any one of claims 1 to 7 when executing the program.
10. A computer-readable storage medium, characterized in that: Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps in the method for processing user information data described in any one of claims 1 to 7.