Data clustering processing methods, devices, computer equipment, and storage media

Abstract

This application provides a method, apparatus, computer device, and storage medium for clustering data. The method includes: acquiring sample feature data of M samples, wherein the sample feature data includes N sample features, and each sample feature corresponds to a sample feature type; determining multiple outlier sample groups based on the sample features of Q samples under P sample feature types, wherein each sample feature under the i-th sample feature type in the i-th outlier sample group satisfies the outlier condition; wherein P is an integer greater than 1 and less than or equal to N, i is an integer greater than 1 and less than or equal to P, and Q is an integer greater than 1 and less than or equal to M; calculating the cluster centers of each outlier sample group based on the sample features included in each outlier sample group to obtain P first cluster centers; and performing clustering processing on the classification features of R classification samples based on the P first cluster centers to obtain clustering results, which can improve the accuracy of data clustering.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to methods, apparatus, computer devices, and storage media for clustering data. Background Technology

[0002] With the development of computer technology, people can send and receive various kinds of information more conveniently and quickly, resulting in an explosive growth in the amount of data generated. In the field of data processing, it is often necessary to analyze large amounts of data to obtain valuable analytical results. Clustering algorithms are an important algorithm used in the field of data mining for data analysis. Currently, clustering algorithms generally require randomly initializing cluster centers and then clustering the data based on these initial centers to obtain the clustering results; however, the clustering results obtained by the current method are sometimes inaccurate. Summary of the Invention

[0003] This application provides a method, apparatus, computer device, and storage medium for clustering data, which can improve the accuracy of data clustering.

[0004] On the one hand, embodiments of this application provide a method for clustering data, the method comprising:

[0005] Obtain sample feature data for M samples, wherein the sample feature data includes N sample features, each sample feature corresponds to a sample feature type, and M and N are positive integers greater than 1;

[0006] Based on the sample features of Q samples under P sample feature types, multiple outlier sample groups are determined. In the i-th outlier sample group, each sample feature under the i-th sample feature type satisfies the outlier condition. Here, P is an integer greater than 1 and less than or equal to N, i is an integer greater than 1 and less than or equal to P, and Q is an integer greater than 1 and less than or equal to M.

[0007] Based on the characteristics of the samples included in each outlier group, the cluster center of each outlier group is calculated to obtain P first cluster centers.

[0008] Clustering is performed on the classification features of R classification samples based on the P first cluster centers to obtain the clustering result, where R is an integer greater than or equal to 1.

[0009] In one embodiment, determining multiple outlier sample groups based on the sample features of Q samples under P sample feature types includes:

[0010] Differential analysis was performed on the sample characteristics of Q samples under P sample feature types to obtain the outlier feature reference value for each sample under each sample feature type.

[0011] The K samples corresponding to the outlier reference values ​​that meet the outlier conditions under the target sample feature type from the P sample feature types are determined as the outlier sample group corresponding to the target sample feature type, where K is an integer greater than or equal to 1 and less than or equal to Q.

[0012] In one embodiment, the step of clustering the R classification samples based on the classification features of the P first cluster centers to obtain the clustering result includes:

[0013] Obtain the threshold n for the number of clusters;

[0014] Based on P first cluster centers, n target cluster centers are obtained;

[0015] Based on the classification features of the R classification samples and the n target cluster centers, the R classification samples are clustered to obtain n target cluster sets;

[0016] Wherein, if n is greater than P, the n target cluster centers are obtained by combining P first cluster centers and nP second cluster centers, and each of the nP second cluster centers is determined based on the sample characteristics in the supplementary sample group obtained from the M samples; if n is less than or equal to P, the n target cluster centers are selected from P first cluster centers.

[0017] In one embodiment, the step of clustering the R classification samples based on the classification features of the R classification samples and the n target cluster centers to obtain n target cluster sets includes:

[0018] Based on the classification features of the R classification samples and the n target cluster centers, the R classification samples are clustered to obtain n initial cluster sets;

[0019] Based on the n initial cluster sets, the number of samples in each initial cluster set, the current number of clustering iterations, and the iteration conditions, n target cluster sets are determined.

[0020] In one embodiment, determining n target cluster sets based on the n initial cluster sets, the number of samples in each initial cluster set, the current clustering count, and iteration conditions includes:

[0021] Determine whether the n initial cluster sets include the target initial cluster set, wherein the number of classified samples included in the target initial cluster set is less than a number threshold;

[0022] If the n initial cluster sets include the target initial cluster set, then the sample clustering calculation is performed again on each classification sample in the target initial cluster set to obtain L first updated cluster sets;

[0023] If the n initial cluster sets do not include the target initial cluster set, determine whether the current clustering count satisfies the iteration condition;

[0024] If the current number of clustering iterations satisfies the iteration condition, then the n initial clustering sets are determined as n target clustering sets.

[0025] In one embodiment, after re-clustering the samples in the initial target clustering set to obtain L first updated clustering sets, the method further includes:

[0026] Compare L with n;

[0027] If L is greater than n, then the L first updated cluster sets are merged to obtain S1 second updated cluster sets;

[0028] If L is less than n, then the first updated cluster set with the largest sample feature variance under the sample feature type is split to obtain S2 third updated cluster sets.

[0029] In one embodiment, after re-clustering the samples in the initial target clustering set to obtain L first updated clustering sets, the method further includes:

[0030] If S1 equals n, then determine whether the current clustering count satisfies the iteration condition. If the current clustering count satisfies the iteration condition, then determine the S1 second updated clustering sets as n target clustering sets.

[0031] Alternatively, if S2 equals n, then determine whether the current clustering count satisfies the iteration condition. If the current clustering count satisfies the iteration condition, then determine the S2 third updated clustering sets as n target clustering sets.

[0032] In one embodiment, before determining whether the current clustering count satisfies the iteration condition, the method further includes:

[0033] Get updated cluster centers;

[0034] The updated cluster centers are calculated based on L first updated cluster sets. The updated cluster centers are used to re-cluster the classification features of the R classification samples or to add cluster labels to the classification samples in the first updated cluster sets.

[0035] Alternatively, the updated cluster centers are calculated based on S1 second updated cluster sets. The updated cluster centers are used to re-cluster the classification features of the R classification samples or to add cluster labels to the classification samples in the second updated cluster sets.

[0036] Alternatively, the updated cluster centers are calculated based on S2 third updated cluster sets. These updated cluster centers are used to re-cluster the classification features of the R classification samples or to add cluster labels to the classification samples in the third updated cluster sets.

[0037] In one embodiment, the method further includes:

[0038] The initial clustering model is invoked to perform clustering analysis on m classification samples with clustering labels among the R classification samples, and the predicted labels of the m classification samples are obtained, where m is an integer greater than or equal to 1 and less than or equal to R;

[0039] The initial clustering model is optimized based on the predicted label and cluster label of each of the m classification samples to obtain the target clustering model.

[0040] In one embodiment, the step of clustering the R classification samples based on the classification features of the R classification samples and the n target cluster centers to obtain n target cluster sets includes:

[0041] Based on the classification features included in each of the R classification samples, calculate the distance from each of the R classification samples to each of the n target cluster centers;

[0042] Each classified sample is recorded in the cluster set containing the target cluster center with the smallest distance, resulting in n target cluster sets.

[0043] In one embodiment, the step of calculating the cluster centers of each outlier sample group based on the sample characteristics included in each outlier sample group to obtain P first cluster centers includes:

[0044] The average value of each sample feature belonging to the same sample feature type in each outlier sample group is calculated to obtain the average value corresponding to each sample feature type in each outlier sample group.

[0045] The average values ​​corresponding to each sample feature type in each outlier sample group are combined to obtain P first cluster centers.

[0046] On one hand, embodiments of this application provide a data clustering processing apparatus, the apparatus comprising:

[0047] The acquisition unit is used to acquire sample feature data of M samples. The sample feature data includes N sample features, each sample feature corresponds to a sample feature type, and M and N are positive integers greater than 1.

[0048] The processing unit is used to determine multiple outlier sample groups based on the sample features of Q samples under P sample feature types, wherein each sample feature under the i-th sample feature type in the i-th outlier sample group satisfies the outlier condition; where P is an integer greater than 1 and less than or equal to N, i is an integer greater than 1 and less than or equal to P, and Q is an integer greater than 1 and less than or equal to M.

[0049] The processing unit is also used to calculate the cluster center of each outlier sample group based on the sample characteristics included in each outlier sample group, and obtain P first cluster centers.

[0050] The processing unit is further configured to perform clustering processing on the classification features of R classification samples based on the P first cluster centers to obtain clustering results, where R is an integer greater than or equal to 1.

[0051] On one hand, the application provides a computer device including an input device, an output device, a processor, and a computer storage medium, wherein the processor and the computer storage medium are interconnected, the computer storage medium is used to store a computer program, and the processor is configured to call the computer program to execute the above-described data clustering processing method.

[0052] On the one hand, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, is used to implement the above-described data clustering processing method.

[0053] On one hand, embodiments of this application provide a computer program product or a computer program, the computer program product including a computer program; the computer program is stored in a computer-readable storage medium, and when the computer program is executed by the processor of a computer device, it performs the above-described data clustering processing method.

[0054] In this embodiment, after acquiring the sample feature data of M samples, the computer device can determine multiple outlier sample groups based on the sample features of Q samples under P sample feature types. Each sample feature under the i-th sample feature type in the i-th outlier sample group satisfies the outlier condition. Based on the sample features included in each outlier sample group, the cluster centers of each outlier sample group are calculated, resulting in P first cluster centers. The classification features of R categorized samples are then clustered based on the P first cluster centers to obtain clustering results. Samples corresponding to the outlier conditions under the same feature type can be grouped into the same outlier sample group. Then, the cluster centers of multiple outlier sample groups are calculated. The cluster centers obtained in this way are more accurate than randomly assigned cluster centers. Further clustering of categorized samples based on the obtained cluster centers improves the accuracy of the clustering results. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1a This is a schematic diagram of the architecture of a data clustering processing system provided in an embodiment of this application;

[0057] Figure 1b This is a schematic diagram of another data clustering processing system provided in an embodiment of this application;

[0058] Figure 1c This is a flowchart illustrating a data clustering method provided in an embodiment of this application;

[0059] Figure 2 This is a flowchart illustrating another data clustering method provided in an embodiment of this application;

[0060] Figure 3 This is a schematic diagram of a process for obtaining a target cluster set provided in an embodiment of this application;

[0061] Figure 4 This is a flowchart illustrating another data clustering method provided in the embodiments of this application;

[0062] Figure 5 This is a schematic flowchart of a model training method provided in an embodiment of this application;

[0063] Figure 6This is a schematic diagram of the structure of a data clustering processing device provided in an embodiment of this application;

[0064] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0065] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0066] This application provides a data clustering processing scheme. The data referred to is data suitable for clustering, such as user data. This user data can be clustered into a single category, facilitating classification and analysis of users and other objects. The general principle of this data clustering processing scheme is as follows: When performing cluster analysis on samples, sample feature data of multiple samples can be obtained. This sample feature data can include N sample features, each corresponding to a sample feature type. Multiple outlier sample groups can then be determined based on the sample features under each sample feature type. In specific implementation, some significant sample features are obtained from the N sample features included in the multiple samples. Then, based on these significant sample features, samples corresponding to significant sample features under the same sample feature type can be grouped into an outlier sample group, ultimately resulting in multiple outlier sample groups. Here, significant sample features refer to sample features that satisfy the outlier condition; that is, all sample features under the i-th sample feature type in the i-th outlier sample group satisfy the outlier condition. After obtaining multiple outlier sample groups, P first cluster centers are obtained by calculating the cluster centers of each outlier sample group. Based on these P first cluster centers, multiple classified samples are then clustered to obtain the clustering result. This clustering result can refer to the classification result of all samples. Based on this clustering result, training samples can be obtained, using the samples as training data and the classification result as supervision labels. These training samples can be used to train classification models, or data can be directly pushed to users corresponding to the samples based on the classification results. Here, P is an integer greater than or equal to 1, and the multiple classified samples can be all or part of the multiple samples obtained above, or they can be other samples besides the multiple samples obtained above, etc.

[0067] In one embodiment, a threshold n for the required number of clusters can be preset. The significance of this value is how many clusters are desired. For example, for 10,000 user data, if only 10 cluster sets are desired, then n is equal to 10. At the same time, n target cluster centers are further obtained based on the P first cluster centers obtained above.

[0068] After obtaining n target cluster centers, during the clustering process for the samples to be classified, firstly, it can be determined whether the number P of the aforementioned cluster centers is less than the cluster size threshold n. If the number P of the aforementioned cluster centers is greater than the cluster size threshold n, then nP supplementary cluster centers can be obtained, and n target cluster centers can be obtained by combining these nP supplementary cluster centers with the P cluster centers. If the number P of the aforementioned cluster centers is greater than the cluster size threshold n, then n target cluster centers can be selected from the P cluster centers. Then, based on the classification features in the R classification samples and the n target cluster centers, the R classification samples are clustered to obtain n target cluster sets.

[0069] In one embodiment, the classification features included in each of the R classification samples can be calculated, and each of the R classification samples can be calculated to each of the n target cluster centers. Then, each classification sample is recorded in the cluster set where the target cluster center with the smallest distance is located, and finally n target cluster sets are obtained, where R is an integer greater than or equal to 1.

[0070] By using the above data clustering processing scheme, samples corresponding to significant sample features under the same feature type can be divided into the same outlier sample group, and the cluster centers of multiple outlier sample groups can be calculated. The cluster centers obtained in this way will be more accurate. Further clustering processing of R classified samples based on the obtained cluster centers will also result in more accurate clustering results.

[0071] Based on the above-described data clustering processing scheme, this application also provides a data clustering processing system. Please refer to [link to relevant documentation]. Figure 1aThe data clustering processing system may include at least one data acquisition device 11, at least one clustering analysis device 12, and at least one data recommendation device 13. The data acquisition device 11, clustering analysis device 12, and data recommendation device 13 may be terminal devices or servers; the embodiments described herein are not limited to these. The data acquisition device 11 and the clustering analysis device 12 may be directly or indirectly connected via wired or wireless communication, and the clustering analysis device 12 and the data recommendation device 13 may also be directly or indirectly connected via wired or wireless communication. The terminal devices may be smartphones, tablets, laptops, desktop computers, etc.; the servers may be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0072] In one embodiment, the data acquisition device 11 can collect sample feature data from multiple samples. For example, the data acquisition device 11 can collect user data (such as whether the user frequently travels), or it can use a camera component to collect image feature data. When the data acquisition device 11 collects user data, a data acquisition interface can be displayed. This interface includes a first prompt message, a confirmation button, and a cancel button. It can also display the specific type of data to be collected, which can be shown to the user through examples. The first prompt message can be used to inform the user that user data needs to be collected and / or to indicate the relevant processing to be performed after the user data is collected. If the user agrees to the data acquisition device collecting their user data, they can trigger the confirmation button to authorize the data acquisition device 11 to collect the user data. If the user does not agree to the data acquisition device collecting their user data, they can trigger the cancel button, in which case the data acquisition device 11 cannot collect the user data. The above-mentioned methods for triggering the confirmation or cancel button can include clicking, double-clicking, swiping, etc.

[0073] In one embodiment, when the data acquisition device 11 successfully acquires sample feature data of multiple samples (such as successfully acquiring user data), it can display a data acquisition interface. This interface includes the acquired sample feature data (such as user data), a second prompt message, a confirmation button, and a cancel button. The second prompt message indicates that the acquired sample feature data (such as user data) needs to be sent to the clustering analysis device 12 for clustering analysis based on the user data. If the sample (i.e., the user) agrees to send the sample feature data (i.e., user data) to the clustering analysis device 12, the confirmation button is triggered, and the data acquisition device 11 can send the sample feature data (i.e., user data) to the clustering analysis device 12. If the sample (i.e., the user) does not agree to send the sample feature data (i.e., user data) to the clustering analysis device 12, the cancel button is triggered, and the data acquisition device 11 cannot send the data to the clustering analysis device 12. This method ensures that user data is collected only with user authorization, improving user data security and effectively preventing unauthorized acquisition of user data.

[0074] After the data acquisition device 11 sends the collected sample feature data to the clustering analysis device 12, the clustering analysis device 12 can analyze and process the sample feature data of multiple samples to obtain multiple target cluster centers. Based on these target cluster centers, the multiple categorized samples are clustered into multiple cluster sets. These cluster sets are then pushed to the data recommendation device 13. The data recommendation device 13 can push information to the terminal where each categorized sample is located based on the received cluster sets, or it can create a sample profile for each categorized sample based on the received cluster sets. The information push here can be related to the clustering labels of each cluster set obtained by the clustering analysis device 12. For example, if the cluster label of a certain cluster set is "food," the information push can be related to food.

[0075] In one embodiment, after receiving sample feature data from multiple samples, the clustering analysis device 12 can include N sample features, each corresponding to a sample feature type. The clustering analysis device 12 can determine multiple outlier sample groups based on the sample features under each sample feature type. Specifically, the samples corresponding to multiple sample features that satisfy the outlier condition under each sample feature type can be determined as the outlier sample group corresponding to that sample feature type. That is, in the i-th outlier sample group, each sample feature under the i-th sample feature type satisfies the outlier condition. After obtaining multiple outlier sample groups, P first cluster centers can be obtained by calculating the cluster centers of each outlier sample group, and n target cluster centers can be further obtained based on the P first cluster centers obtained above. After obtaining n target cluster centers, during the clustering process of the samples to be classified, the classification features of each of the R classification samples can be calculated, the distance of each of the R classification samples to each of the n target cluster centers can be calculated, and then each classification sample can be recorded in the cluster set of the target cluster center with the smallest distance, and finally n target cluster sets can be obtained.

[0076] In this embodiment, the sample feature data of multiple samples, the threshold number of clusters, the classified samples and the classification features of each classified sample, the n target cluster centers, and the n target cluster sets involved can all be stored on the blockchain to prevent tampering. Taking the storage of n target cluster sets on the blockchain as an example, the entire on-chain process is illustrated. First, the blockchain network includes multiple nodes. The clustering analysis device 12 can send n target cluster sets to any node in the blockchain network. After receiving the n target cluster sets, any node can perform a validity check on the n target cluster sets. After the validity check is passed, the n target cluster sets generate a block and send the block to other nodes in the blockchain network for consensus processing. After consensus is reached by other nodes, the block is stored in the blockchain network.

[0077] It should be noted that, Figure 1a This merely exemplifies the system architecture of a data clustering system and does not limit the specific architecture of the data clustering system. For example, Figure 1a In this embodiment, the data acquisition device 11, cluster analysis device 12, and data recommendation device 13 can all be located outside the blockchain network; however, in other embodiments, the clustering processing system can also be deployed based on the blockchain network. This means that the data acquisition device 11, cluster analysis device 12, and data recommendation device 13 can all be deployed within the blockchain network, or the data acquisition device 11 can be deployed outside the blockchain network, while the cluster analysis device 12 and data recommendation device 13 can be deployed within the blockchain network, and so on. For example... Figure 1b The data acquisition device 11 and data recommendation device 13 are deployed outside the blockchain network, while the clustering analysis device 12 is deployed inside the blockchain network. Figure 1b The blockchain network shown is for illustrative purposes only.

[0078] The blockchain mentioned here refers to a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0079] The underlying blockchain platform can include modules for user management, basic services, smart contracts, and operational monitoring. The user management module is responsible for managing the identity information of all blockchain participants, including maintaining public and private key generation (account management), key management, and maintaining the correspondence between user identities and blockchain addresses (access management). Under authorization, it also monitors and audits transactions of certain real identities and provides risk control rule configuration (risk control audit). The basic services module is deployed on all blockchain node devices to verify the validity of business requests. After consensus is reached on valid requests, they are recorded in storage. For a new business request, the basic services first perform interface adaptation parsing and authentication (interface adaptation), and then encrypt the business information using a consensus algorithm (consensus management). The blockchain process involves several layers of data transmission and processing. The first layer, after encryption, transmits the encrypted data consistently and completely to the shared ledger (network communication) and stores it. The second layer, the smart contract module, is responsible for contract registration, issuance, triggering, and execution. Developers can define contract logic using a programming language and publish it to the blockchain (contract registration). Based on the contract terms, the module invokes keys or other events to trigger execution, completing the contract logic. It also provides functions for contract upgrades and cancellations. The third layer, the operation monitoring module, is primarily responsible for deployment, configuration modification, contract settings, cloud adaptation, and real-time visualization of the product's operational status during product release. This includes alerts, network status monitoring, and node device health monitoring. The platform's product service layer provides basic capabilities and implementation frameworks for typical applications. Developers can leverage these capabilities and add business characteristics to implement business logic on the blockchain. The fourth layer provides blockchain-based application services to business stakeholders.

[0080] By employing the aforementioned data clustering scheme and system, samples corresponding to significant features of the same type can be grouped into the same outlier group. Cluster centers for multiple outlier groups are then calculated, resulting in more accurate cluster centers. Further clustering of R categorized samples based on these centers yields even more accurate results. This allows for subsequent information recommendations or sample profiling based on the clustering results, improving the accuracy of these recommendations or profiling.

[0081] Based on the above data clustering processing scheme, please refer to Figure 1c , Figure 1c This is a flowchart illustrating a data clustering method provided in an embodiment of this application. The data clustering method can be executed by a computer device, which can be a clustering analysis device within a data clustering system. This clustering analysis device can be a terminal device or a server. The terminal device can be a smartphone, tablet, laptop, desktop computer, smart vehicle, etc.; the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, etc. The data clustering method described in this embodiment may include the following steps S101-S104.

[0082] S101. Obtain sample feature data for M samples. The sample feature data includes N sample features, each corresponding to a sample feature type. M and N are positive integers greater than 1. The samples can be users, images, or other objects to be clustered. When a sample is a user, the corresponding sample feature data may include user age, user gender, user consumption scenario, active duration of the user consumption scenario, etc., and the sample feature types corresponding to the N sample features are user age, user gender, user consumption scenario, etc. When a sample is an image, the corresponding sample feature data may include image type, image capture time, geographical location, image color, etc., and the sample feature types corresponding to the N sample features are image type, image capture time, geographical location, image color.

[0083] In practical implementation, the computer device can obtain sample feature data of M samples from various data statistics platforms. When the samples are images, the computer device can also acquire M images through a camera component and extract sample features from these M images to obtain sample feature data for each image. When the samples are users, the computer device can also obtain sample feature data about M users from various application software, such as instant messaging software, video software, travel software, etc.; the sample feature data of these M users is shown in Table 1.

[0084] Table 1

[0085]

[0086]

[0087] In other embodiments, the samples can also be various applications and other objects that need to be classified, and the corresponding sample feature data can include data such as the type of application, the developer, the release time, etc.

[0088] S102. Based on the sample features of Q samples under P sample feature types, determine multiple outlier sample groups. In the i-th outlier sample group, each sample feature under the i-th sample feature type satisfies the outlier condition. Wherein, P is an integer greater than 1 and less than or equal to N, i is an integer greater than 1 and less than or equal to P, and Q is an integer greater than 1 and less than or equal to M.

[0089] As shown in step S101, the sample feature data includes N sample features, which correspond to N sample feature types. In step S102, the P sample feature types can be some or all of the N sample feature types. The Q samples can be some or all of the M samples. Each outlier sample group can include one or more outlier samples. An outlier sample can be understood as a sample feature of the outlier sample under at least one target sample feature type, which differs significantly from the sample features of other samples under that target sample feature type. The samples in each outlier sample group come from all or some of the Q samples.

[0090] In one embodiment, a computer device can select P sample feature types from N sample feature types, and then select Q sample features from M samples based on the P sample feature types. That is, each of the selected Q samples includes sample features belonging to all P sample feature types. For example, the computer device can select only two sample feature types from Table 1: "Most Frequent Consumption Scenario: Traffic" and "Active Duration," where N=3 and P=2. Then, based on the sample feature types "Most Frequent Consumption Scenario: Traffic" and "Active Duration," it can select 3 sample features from M samples, where M=10 and Q=3. The 3 samples are Sample 1, Sample 2, and Sample 3, where Sample 1 has a sample feature of 100, Sample 2 has a sample feature of -9, and Sample 3 has a sample feature of 8. It is understood that this is merely an example, and the quantities of M, N, P, and Q mentioned above are defined according to actual needs. For example, in most cases, the actual number of samples M can be equal to tens of thousands or even millions.

[0091] The computer device can also select Q samples corresponding to each of the P sample feature types from the M samples. That is, one sample feature type can select Q corresponding samples. Then, it determines whether the sample features of the Q samples under each of the P sample feature types satisfy the outlier condition, and takes the K samples corresponding to the sample features that satisfy the outlier condition under each sample feature type as an outlier sample group, ultimately obtaining multiple outlier sample groups. It should be understood that each sample feature type can obtain an outlier sample group as needed. In step S102, there are P sample feature types, and ultimately P outlier sample groups can be obtained.

[0092] In one embodiment, the outlier condition being met for each sample feature under the i-th sample feature type in the i-th outlier sample group means that the difference between each sample feature under the i-th sample feature type in the i-th outlier sample group and the reference feature under the i-th sample feature type satisfies the difference condition. This can be understood as the difference between each sample feature under the target sample feature type in the i-th outlier sample group and the reference feature under that target sample feature type satisfying the difference condition. For example, the difference between the sample feature value of each sample feature under the target sample feature type in the i-th outlier sample group and the reference feature value (such as the reference average) under that target sample feature type satisfies the difference threshold.

[0093] In one embodiment, the difference between each sample feature under the i-th sample feature type in the i-th outlier sample group and the reference feature under the i-th sample feature type satisfies the difference condition if: the difference between the sample feature value of each sample feature under the i-th sample feature type in the i-th outlier sample group and the reference average value under the i-th sample feature type is greater than a preset difference threshold. The computer device can calculate the average value of the sample feature values ​​under the i-th sample feature type to obtain the reference average value under the i-th sample feature type, then calculate the difference between each sample feature under the i-th sample feature type and the reference average value to obtain the difference value between each sample feature value under the i-th sample feature type and the reference average value, and add K samples corresponding to the sample feature values ​​with difference values ​​greater than the difference threshold to the outlier sample group corresponding to the i-th sample feature type. K is an integer greater than or equal to 1 and less than or equal to Q. Taking the 10 samples in Table 1 as an example, the average active duration is calculated to be 50 (reference average). Then, after calculating the difference between each sample feature under "active duration" and 50, the two samples with the largest difference are sample 2 and sample 10 (both are greater than the difference threshold of 50 and are considered outliers, or 20% of the samples from largest to smallest are considered outliers). Therefore, sample 2 and sample 10 constitute the outlier sample group {sample 2, sample 10} under "active duration".

[0094] In one embodiment, when the sample features in the sample are of type such as character, the computer device can first perform numerical processing on the sample features of Q samples under P sample feature types to obtain the sample feature values ​​of the sample features of Q samples under each sample feature type.

[0095] In one embodiment, when there are character-type sample features among the sample features of Q samples under P sample feature types (such as the most frequent consumption scenario of transportation in Table 1), the computer device can quantify the character-type sample features of the Q samples according to a pre-set conversion rule.

[0096] For example, the character-type sample features of Q samples can be one-hot encoded, converting them into a numerical matrix containing only 0 and 1 values. When the sample feature type is "most frequent consumption scenario: transportation," if one of the Q samples has the character-type sample feature "most frequent consumption scenario: transportation," the sample feature value for that sample is 1; if the other Q samples do not have the character-type sample feature "most frequent consumption scenario: transportation," the sample feature value for the other samples is 0; or, the sample feature "most frequent consumption scenario: transportation" can be pre-set to be converted to the value 5, and samples with other features can be converted to the value 0. Then, the computer can directly quantify the sample feature "most frequent consumption scenario: transportation" among the Q samples to obtain the sample feature value 5 corresponding to that sample's "most frequent consumption scenario: transportation" feature.

[0097] In one embodiment, the numerical values ​​corresponding to certain data can also be directly defined. For example, in Table 1, the numerical value corresponding to catering is 5, the numerical value corresponding to transportation is 4, the numerical value corresponding to medical care is 3, the numerical value corresponding to communication is 2, and so on, so as to complete the numerical processing of the sample features corresponding to each sample feature type in the sample.

[0098] In one embodiment, when acquiring the sample feature data of M samples, data filtering can also be performed according to filtering rules to remove dirty data. When the sample features of Q samples include numerical sample features (such as the numerical sample feature of a sample in Table 1 being -9), the computer device can identify the samples with numerical sample features less than 0 in the Q samples as dirty data and remove the samples with numerical sample features less than 0 (i.e., remove sample 2), retaining the samples with sample features greater than 0 in the Q samples.

[0099] After processing the character-type sample features and numerical features of each of the Q samples in the above manner, the final numerical matrix can be obtained, as shown in Table 2: Each column of the numerical feature represents the sample feature value of each sample under a sample feature type, and each row represents the sample feature value of all sample features included in each sample.

[0100] Table 2

[0101]

[0102] For example, in Table 2, the computer device can calculate the reference average value corresponding to "Whether it is the most frequent consumption scenario_dining" based on the sample feature values ​​of the five samples under the sample feature type "Whether it is the most frequent consumption scenario_dining". The reference average value for "Whether it is the most frequent consumption scenario_dining" is 0.4. Then, the difference between the sample feature values ​​of the five samples and the reference average value of 0.4 is compared. The differences are: Sample 1's sample feature value is 0.6, Sample 3's is 0.4, Sample 4's is 0.4, Sample 5's is 0.6, and Sample 6's is 0.4. The computer device can then identify Sample 1 and Sample 5, whose sample feature values ​​are greater than the first difference threshold of 0.4, as an outlier group, i.e., K=2. Sample 1 and Sample 5 constitute the outlier group corresponding to "Whether it is the most frequent consumption scenario_dining". This outlier group can be shown in Table 3.

[0103] Table 3

[0104]

[0105] The outlier reference value Z-score of each sample feature under the i-th sample feature type in the i-th outlier sample group is greater than the preset reference value. The computer device can perform difference analysis on the sample features of Q samples under the i-th sample feature type to obtain the outlier reference value of each sample under the i-th sample feature type. Then, the K samples corresponding to the outlier reference values ​​that meet the outlier conditions are determined as an outlier sample group.

[0106] In one embodiment, the outlier condition being met for each sample feature under the i-th sample feature type in the i-th outlier sample group can be defined as follows: the outlier feature reference value (e.g., Z-score) of each sample feature under the i-th sample feature type in the i-th outlier sample group satisfies the outlier condition. Specifically, this can be that the outlier feature reference value is greater than a preset threshold, or that the top x outlier feature reference values ​​arranged in descending order satisfy the outlier condition. The samples corresponding to the outlier reference values ​​that satisfy the outlier condition are recorded in the outlier sample group. The outlier feature reference value is used to represent the difference between the feature value corresponding to the sample feature data and the mean of the sample feature under that sample feature type.

[0107] By performing difference analysis on the sample characteristics of Q samples under P sample feature types, the difference between the sample feature of the i-th sample feature type and the mean of the overall sample feature under the i-th sample feature type can be evaluated. This difference analysis can assess the distance of each sample feature to the mean sample feature; the larger the distance value, the greater the difference, and the corresponding sample can be added as an outlier to the corresponding outlier sample group. This can be understood as follows: computer equipment can perform difference analysis on the sample characteristics of Q samples under P sample feature types to obtain the outlier feature reference value for each sample under each sample feature type. Then, the K samples corresponding to the outlier feature reference values ​​that meet the outlier criteria under the target sample feature type are determined as the outlier sample group corresponding to the target sample feature type. Here, the target sample feature type can be any one of the P sample feature types.

[0108] In one embodiment, since the outlier reference values ​​of each sample under the i-th sample feature type can be processed to present a normal distribution, the outlier condition can be an outlier reference value above a preset quantile. For example, if the outlier condition is an outlier reference value above the 90th quantile, then the K samples corresponding to the outlier reference values ​​above the 90th quantile under the i-th sample feature type are added to the outlier sample group corresponding to the i-th sample feature type. That is, the outlier reference values ​​corresponding to each sample can be arranged in order of size, and the K samples (1-90%=) 10% are selected as the outlier sample group corresponding to the i-th sample feature type in descending order. For example, if there are a total of 100 samples, then for the i-th sample feature type, the 10 samples with the largest outlier reference values ​​are recorded as the outlier sample group corresponding to the i-th sample feature type. In some other embodiments, the outlier condition can be an outlier reference value greater than a preset reference value. For the i-th sample feature type, the K samples corresponding to outlier feature reference values ​​greater than a preset reference value can be identified as the outlier sample group corresponding to the i-th sample feature type. For example, for the i-th sample feature type, if the outlier feature reference value (e.g., Z-score) of sample 1 and the outlier feature reference value (Z-score) of sample 2 are both greater than the preset reference value, then the outlier sample group corresponding to the i-th sample feature type is {sample 1, sample 2}.

[0109] In one embodiment, the computer device can perform differential analysis on the sample features of Q samples under the i-th sample feature type to obtain the outlier reference value of each sample under the i-th sample feature type. Specifically, this can be achieved by: quantifying the sample features of Q samples under P sample feature types to obtain the sample feature values ​​of the Q samples under the P sample feature types; then, standardizing the sample features of each sample under the i-th sample feature type based on the sample feature values ​​(i.e., calculating the Z-score of the sample features of each sample under each sample feature type) to obtain the outlier reference value of each sample under the i-th sample feature type.

[0110] It should be noted that, in the embodiments of this application, each of the P sample feature types can refer to the implementation method for determining the outlier sample group corresponding to the i-th sample feature type to determine the corresponding outlier sample group.

[0111] S103. Based on the sample features included in each outlier sample group, calculate the cluster centers of each outlier sample group to obtain P first cluster centers. As mentioned above, each of the P sample feature types corresponds to one outlier sample group. Therefore, by calculating the cluster centers of each outlier sample group, we can ultimately obtain P first cluster centers. The sample features included in each outlier sample group can be either the aforementioned N sample features or the sample features corresponding to each of the aforementioned P sample feature types.

[0112] In one embodiment, as described above, each outlier sample group includes a sample feature value for each outlier sample's feature. A computer device can average the sample feature values ​​of each outlier sample belonging to the same feature type within each outlier sample group to obtain the average value corresponding to each feature type in each outlier sample group. Then, the average values ​​corresponding to each feature type in each outlier sample group are combined to obtain P first cluster centers. Each first cluster center can be understood as a vector composed of the average values ​​corresponding to each feature type in the corresponding outlier sample group. For example, through step S102 above, an outlier sample group as shown in Table 3 can be obtained. The computer device can average the sample feature values ​​of each sample feature under the sample feature type "Whether it is the most frequent consumption scenario_Dining" in the outlier sample group, and obtain the average value corresponding to the sample feature type as 0.5 (i.e., calculate the average value of the "Whether it is the most frequent consumption scenario_Dining" column in Table 3). Similarly, the average value corresponding to the sample feature type "Whether it is the most frequent consumption scenario_Transportation" can be calculated as 0, the average value corresponding to the sample feature type "Whether it is the most frequent consumption scenario_Communication" can be calculated as 0, the average value corresponding to the sample feature type "Whether it is the most frequent consumption scenario_Medical" can be calculated as 0, and the average value corresponding to the sample feature type "Activity Duration" can be calculated as 61. The first cluster center of this outlier sample group can be represented as (0.5, 0, 0, 0, 61). The formula for calculating the average value corresponding to each sample feature type is as follows:

[0113]

[0114] Among them, I j c represents the average value corresponding to the feature type of the j-th sample. j This represents the number of samples under the feature type of the j-th sample, where i is greater than or equal to 1 and less than or equal to c. j an integer; x i This represents the sample feature of the i-th sample under the j-th sample feature type.

[0115] S104. Based on the P first cluster centers, perform clustering processing on the classification features of the R classification samples to obtain the clustering result, where R is an integer greater than or equal to 1. The R classification samples can be some or all of the M samples mentioned above, and can also be other samples besides the M samples.

[0116] In one embodiment, the clustering result may include P cluster sets, each corresponding to a first cluster center. A computer device can calculate the distance from the classification features of each classification sample to each of the P first cluster centers based on the classification features of each sample. Then, each classification sample is recorded in the cluster set containing the first cluster center with the smallest distance, resulting in P first cluster sets. The computer device can use the Euclidean distance formula to calculate the Euclidean distance from each classification sample to each first cluster center based on the classification features of each sample. The Euclidean distance measures the spatial distance between a classification sample and each first cluster center; the farther a classification sample is from a certain first cluster center, the further it is from that first cluster center. The classification features of each classification sample can also be represented as a vector, similar to the first cluster centers of the outlier sample group described above. For example, the first cluster center 1 is (0.5, 0, 0, 0, 61), and the first cluster center 2 is (0.5, 0, 0.5, 0, 35). The classification feature of classification sample 1 is (0.5, 0, 0.5, 0, 30). Then, the computer device can calculate the distance from classification sample 1 to the first cluster center 1 according to the Euclidean distance formula based on the classification feature (0.5, 0, 0.5, 0, 30), obtaining a distance 1 of 31. Similarly, the computer device can calculate the distance from classification sample 1 to the first cluster center 2 according to the Euclidean distance formula based on the classification feature (0.5, 0, 0.5, 0, 30), obtaining a distance 2 of 5.02. It can be seen that the distance 2 from classification sample 1 to the first cluster center 2 is less than the distance 1 from classification sample 1 to the first cluster center 1. Therefore, the computer device can record classification sample 1 into the cluster set containing the first cluster center 2.

[0117] In one embodiment, a cluster size threshold n can be preset. The specific implementation of step S103 by the computer device can be as follows: obtain the cluster size threshold n, and compare the cluster size threshold n with P. If the cluster size threshold n is greater than P, then nP second cluster centers need to be obtained, and n target cluster centers are obtained based on P first cluster centers and nP second cluster centers. For example, combining P first cluster centers and nP second cluster centers yields n target cluster centers. If the cluster size threshold n is less than P, then n first cluster centers can be selected from P first cluster centers, and these selected n first cluster centers are determined as n target cluster centers. The selection method can be random selection, or sorting the outlier sample groups by size and selecting the n first cluster centers corresponding to the n outlier sample groups with the most data as n target cluster centers. If the cluster size threshold n is equal to P, then the P first cluster centers can be directly used as n target cluster centers.

[0118] The n target cluster centers can be represented as D = {D1, D2, D3, ..., Dn}. Then, the computer device clusters the R classification samples based on their classification features and the n target cluster centers, resulting in n target cluster sets. At this point, the clustering result includes n target cluster sets. The specific implementation method for clustering the R classification samples based on their classification features and the n target cluster centers to obtain n target cluster sets can be referenced above, and will not be elaborated further here.

[0119] In this embodiment, after acquiring the sample feature data of M samples, the computer device can determine multiple outlier sample groups based on the sample features of Q samples under P sample feature types. Each sample feature under the i-th sample feature type in the i-th outlier sample group satisfies the outlier condition. Based on the sample features included in each outlier sample group, the cluster centers of each outlier sample group are calculated, resulting in P first cluster centers. The classification features of R categorized samples are then clustered based on the P first cluster centers to obtain clustering results. Samples corresponding to the outlier conditions under the same feature type can be grouped into the same outlier sample group. Then, the cluster centers of multiple outlier sample groups are calculated. The cluster centers obtained in this way are more accurate than randomly assigned cluster centers. Further clustering of the categorized samples based on the obtained cluster centers also yields relatively accurate clustering results.

[0120] Based on the above data clustering processing scheme, please refer to Figure 2 , Figure 2This is a flowchart illustrating a data clustering method provided in an embodiment of this application. The data clustering method can be executed by a computer device, which can be a clustering analysis device within a data clustering system. This clustering analysis device can be a terminal device or a server. The terminal device can be a smartphone, tablet, laptop, desktop computer, smart vehicle, etc.; the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, etc. The data clustering method described in this embodiment may include the following steps S201-S206.

[0121] S201. Obtain the sample feature data of M samples. The sample feature data includes N sample features, each of which corresponds to a sample feature type. M and N are positive integers greater than 1.

[0122] S202. Based on the sample features of Q samples under P sample feature types, determine multiple outlier sample groups. In the i-th outlier sample group, each sample feature under the i-th sample feature type satisfies the outlier condition. Wherein, P is an integer greater than 1 and less than or equal to N, i is an integer greater than 1 and less than or equal to P, and Q is an integer greater than 1 and less than or equal to M.

[0123] S203. Based on the characteristics of the samples included in each outlier sample group, calculate the cluster center of each outlier sample group to obtain P first cluster centers.

[0124] The specific implementation methods of steps S201-S203 can be found in the specific implementation methods of steps S101-S103, which will not be repeated here.

[0125] S204. Obtain the cluster size threshold n. This cluster size threshold can be determined based on the sample type and number to be clustered, or it can be determined based on the clustering requirements. For example, the cluster size threshold can be set to 6, 7, etc.

[0126] S205. Obtain n target cluster centers based on P first cluster centers. The computer can determine whether n is greater than P; if n is greater than P, the n target cluster centers are obtained by combining the P first cluster centers and nP second cluster centers, where each of the nP second cluster centers is determined based on the characteristics of samples in a supplementary sample group obtained from M samples; if n is less than or equal to P, the n target cluster centers are selected from the P first cluster centers.

[0127] In one embodiment, if n is greater than P, the n target cluster centers are obtained by combining P first cluster centers and nP second cluster centers. Each of the nP second cluster centers is determined based on the sample characteristics of a supplementary sample group obtained from M samples. Specifically, the computer device can select multiple samples from the M samples and form a supplementary sample group, ultimately obtaining nP supplementary sample groups. The computer device calculates the second cluster center of each supplementary sample group based on the sample characteristics of each of the nP supplementary sample groups, obtaining nP second cluster centers. The supplementary sample group may include multiple samples randomly selected from the M samples; or, the supplementary sample group may also include multiple samples selected from the M samples in a certain order, for example, selecting samples with even numbers (sample numbers 2, 4, 6, etc.) from the M samples. Then, the computer device can combine the P first cluster centers and nP second cluster centers to obtain the n target cluster centers. The calculation of the second cluster center of the corresponding supplementary sample group based on the sample characteristics in each supplementary sample group can refer to the calculation method of the P first cluster centers mentioned above, and will not be repeated here.

[0128] In one embodiment, if n is less than or equal to P, the specific implementation of selecting n target cluster centers from P first cluster centers can be as follows: the computer device can randomly select n target cluster centers from the P first cluster centers; or, the computer device can also determine the number of samples in the outlier sample group corresponding to each first cluster center, sort the P first cluster centers according to the number of samples from largest to smallest, select the top n first cluster centers based on the sorting result, and determine these n first cluster centers as n target cluster centers.

[0129] S206. Based on the classification features of R classification samples and n target cluster centers, cluster the R classification samples to obtain n target cluster sets. The computer equipment can calculate the distance from each of the R classification samples to each of the n target cluster centers based on the classification features included in each of the R classification samples; then, each classification sample is recorded in the cluster set containing the target cluster center with the smallest distance, resulting in n target cluster sets. Each classification feature also corresponds to a sample feature type.

[0130] In one embodiment, step S206 can be further implemented as follows: Based on the classification features of the R classification samples and n target cluster centers, cluster the R classification samples to obtain n initial cluster sets. Then, based on the n initial cluster sets, the number of classification samples included in each initial cluster set, the current clustering count, and the iteration conditions, determine the n target cluster sets. The iteration conditions can be that the current clustering count is greater than or equal to the maximum iteration count, which is typically set to around 2000, for example, 2000, 20001, etc. The specific implementation of the computer device determining the n target cluster sets based on the n initial cluster sets, the number of classification samples included in each initial cluster set, the current clustering count, and the iteration conditions can be as follows: Figure 3 As shown, the steps s11-s14 may be included:

[0131] s11. Determine whether the n initial cluster sets include the target initial cluster set, wherein the number of samples in the target initial cluster set is less than a threshold. The threshold can be preset based on clustering requirements, the type of samples, or the number of samples. For example, when the type of samples is "user," the threshold can be set to 20; when the type of samples is "image," the threshold can be set to 15. For example, with 1000 samples, the threshold can be 5% of the sample count, i.e., 50. The computer device can count the number of samples in each initial cluster set and determine whether the n initial cluster sets include the target initial cluster set whose sample count is less than the threshold. If the n initial cluster sets include the target initial cluster set whose sample count is less than the threshold, proceed to step s12; if the n initial cluster sets do not include the target initial cluster set whose sample count is less than the threshold, proceed to step s13. The number of target initial cluster sets can be one or more.

[0132] For example, there are three initial cluster sets: initial cluster set 1, initial cluster set 2, and initial cluster set 3. Initial cluster set 1 includes 1 sample of categorized samples, initial cluster set 2 includes 3 samples of categorized samples, and initial cluster set 3 includes 5 samples of categorized samples. The computer device can determine that the number of categorized samples included in initial cluster set 1 is less than the number threshold 3, that is, the target initial cluster set is included among the three initial cluster sets, and the target initial cluster set is initial cluster set 1.

[0133] s12. If the n initial cluster sets include the target initial cluster set, then the sample clustering calculation is performed again for each category sample in the target initial cluster set to obtain L first updated cluster sets. Here, re-clustering the sample clustering calculation for each category sample in the target initial cluster set means re-clustering each category sample in the target initial cluster set into other initial cluster sets among the n initial cluster sets. These other initial cluster sets are not initial cluster sets with a sample size less than a threshold. The L first updated cluster sets are obtained based on the n initial cluster sets.

[0134] In one embodiment, if the n initial cluster sets include a target initial cluster set, the computer device can calculate the allocation distance from each classification sample in the target initial cluster set to the target cluster center corresponding to other initial cluster sets based on the classification characteristics of each classification sample in the target initial cluster set. Each classification sample in the target initial cluster set is then recorded in the initial cluster set corresponding to the target cluster center with the smallest allocation distance. Finally, one or more target initial cluster sets are deleted, and the sample clustering calculation is re-performed for each sample in the target initial cluster set. After deleting one or more target initial cluster sets, the new initial cluster sets included in the n initial cluster sets are determined as L first updated cluster sets. Continuing with the above example, the computer device determines that one target initial cluster set is included in the three initial cluster sets; this target initial cluster set is initial cluster set 1. The initial cluster set includes classification sample 1. The computer device can calculate the allocation distance from classification sample 1 in the initial cluster set 1 to the target cluster center 1 corresponding to the initial cluster set 2 based on the classification characteristics of classification sample 1, thus obtaining the allocation distance value 1 between classification sample 1 and the target cluster center 1. It can also calculate the allocation distance from classification sample 1 to the target cluster center 2 corresponding to the initial cluster set 3 based on the classification characteristics of classification sample 1, thus obtaining the allocation distance value 2 between classification sample 1 and the target cluster center 2. Then, if the computer device determines that the allocation distance value 1 is less than the allocation distance value 2, it records classification sample 1 in the initial cluster set 2 where the target cluster center 1 is located. Then, the computer device deletes the target initial cluster set, i.e., the initial cluster set 1, and combines the new initial cluster set 2 and the initial cluster set 3 to obtain L first updated cluster sets (at this time, L = 2).

[0135] In one embodiment, after executing step s12, the computer device can calculate the cluster centers of each first updated cluster set based on the classification features of the classification samples in each first updated cluster set, thereby obtaining the updated cluster centers of each first updated cluster set. In one embodiment, the updated cluster centers can be used to re-cluster the classification features of R classification samples, or the updated cluster centers can be used to add cluster labels to the classification samples in the first updated cluster sets.

[0136] The computer device can compare L with n. ① If L is greater than n, then the L first updated cluster sets are merged to obtain S1 second updated cluster sets. These S1 second updated cluster sets are derived from the L first updated cluster sets. Specifically, the computer device can calculate the distance between the updated cluster centers of every two first updated cluster sets and merge the two sets with the smallest distance to obtain a merged cluster set. This merged cluster set includes all classification samples from the two sets with the smallest distance. Then, the unmerged first updated cluster sets and the merged cluster set are combined to obtain S1 second updated cluster sets. The distance between the updated cluster centers of every two first updated cluster sets can be calculated using the Euclidean distance formula. Each merging process can merge only the two first updated cluster sets with the smallest distance. In other embodiments, three or even several first updated cluster sets with distance values ​​less than a threshold can be merged as needed.

[0137] Then, the computer device can calculate the cluster centers of each second updated cluster set based on the classification features of the classification samples in each second updated cluster set, thereby obtaining the updated cluster centers of each second updated cluster set. In one embodiment, the updated cluster centers of each second updated cluster set are used to re-cluster the classification features of R classification samples, or to add cluster labels to the classification samples in the second updated cluster set.

[0138] The computer can then determine whether S1 is greater than n. If S1 is greater than n, the S1 second updated cluster sets are merged until the number of merged updated cluster sets is no greater than n. After the number of merged updated cluster sets is no greater than n, it can then determine whether the number of merged multiple updated cluster sets is less than n. If the number of merged multiple updated cluster sets is less than n, the first updated cluster set with the largest variance of sample features under the sample feature type is split until the number of split multiple updated cluster sets is no less than n.

[0139] If the number of updated cluster sets after merging S1 of the second updated cluster sets equals n, then it is determined whether the current clustering count satisfies the iteration condition. If the current clustering count satisfies the iteration condition, then S1 of the second updated cluster sets are determined as n target cluster sets. If the current clustering count does not satisfy the iteration condition, then based on the classification characteristics of the classification samples included in each second updated cluster set, the updated cluster centers of each second updated cluster set are calculated, and the classification characteristics of R classification samples are clustered based on the updated cluster centers of each second updated cluster set to obtain S1 target cluster sets. The specific implementation method of the computer device clustering the classification characteristics of R classification samples based on the updated cluster centers of each second updated cluster set to obtain S1 target cluster sets can be found in the specific implementation method of steps s11-s14 above. The iteration condition can be that the number of iterations needs to be greater than the iteration threshold.

[0140] ② If L is less than n, then the first updated cluster set with the largest sample feature variance under each sample feature type is split to obtain S2 third updated cluster sets. These S2 third updated cluster sets are also derived from the L first updated cluster sets. The computer device can calculate the sample feature variance value for each sample feature type in each first updated cluster set, and then split the first updated cluster set corresponding to the largest sample feature variance value under each sample feature type to obtain S2 third updated cluster sets. Here, the splitting process can refer to randomly splitting the first updated cluster set corresponding to the sample feature type with the largest sample feature variance value into at least two split cluster sets. The number of samples in each split cluster set can be the same or different for the classification samples of the first updated cluster sets. For example, if n is 4, there are 3 (i.e., L = 3) first updated cluster sets: first updated cluster set 1, second updated cluster set 2, and third cluster set 3. Then, the computer device can calculate the sample feature variance value corresponding to each sample feature type based on the classification features of each classification sample in the first updated cluster set 1. Suppose that the first updated cluster set 1 includes three categorical samples: sample 1, sample 2, and sample 3. The sample feature type in the first updated cluster set 1 has the largest variance. The three categorical samples in the first updated cluster set are then split. For example, two of the three categorical samples in the first updated cluster set 1 (e.g., sample 1 and sample 2) are split into one split cluster set, and the other categorical sample (e.g., sample 3) is split into another split cluster set. These two split cluster sets, the first updated cluster set 2, and the first updated cluster set 3 then constitute four (S2 = 4) third updated cluster sets. Alternatively, the three categorical samples in the first updated cluster set 1 can be split into three separate split cluster sets, each containing one categorical sample from the first updated cluster set 1.

[0141] Then, the computer device calculates the cluster centers for each third updated cluster set based on the classification features of the categorized samples in each third updated cluster set. These updated cluster centers can then be used to re-cluster the R categorized samples based on their classification features or to add cluster labels to the categorized samples in the third updated cluster set. Next, the computer device continues to determine if S2 is greater than n. If S2 is greater than n, then, following the specific implementation method described above (if L is greater than n), the L first updated cluster sets are merged to obtain S1 second updated cluster sets. The S2 third updated cluster sets are then merged until the number of merged cluster sets is no greater than n. If S2 is less than n, then following the same implementation method as if L is less than n, the first updated cluster set with the largest sample feature variance under the sample feature type is split to obtain S2 third updated cluster sets. This process continues until the number of split cluster sets is not less than n. If S2 equals n, it is determined whether the current clustering count meets the iteration condition. If it does, the S2 third updated cluster sets are determined as n target cluster sets. If the current clustering count does not meet the iteration condition, the updated cluster centers of each third updated cluster set are used to cluster the classification features of R categorized samples to obtain S2 target cluster sets.

[0142] ③ If L equals n, it can be determined whether the current clustering count satisfies the iteration condition. If the current clustering count satisfies the iteration condition, the L first updated cluster sets can be directly determined as n target cluster sets. If the current clustering count does not satisfy the iteration condition, then the R classification samples are clustered according to the updated cluster centers of each first updated cluster set and the classification characteristics of the R classification samples. The specific implementation of clustering the R classification samples according to the updated cluster centers of each first updated cluster set and the classification characteristics of the R classification samples can be found above in the section on clustering the R classification samples according to the n target cluster centers and the classification characteristics of the R classification samples to obtain n target cluster sets, and will not be repeated here.

[0143] s13. If the n initial cluster sets do not include the target initial cluster set, recalculate the cluster centers for each initial cluster set based on the classification characteristics of the classification samples included in each initial cluster set to obtain the updated cluster centers for each initial cluster set. Then, determine whether the current number of clustering iterations meets the iteration conditions.

[0144] s14. If the current clustering count satisfies the iteration condition, then the n initial cluster sets are determined as n target cluster sets. If the current clustering count does not satisfy the iteration condition, then based on the classification features of the R classification samples and the n updated cluster centers, the R classification samples are clustered to obtain n target cluster sets. The specific implementation of clustering the R classification samples based on the classification features of the R classification samples and the n updated cluster centers to obtain n target cluster sets can be found in s11-s12 above, and will not be repeated here.

[0145] In one embodiment, before determining whether the current clustering count meets the iteration condition, updated cluster centers are obtained; wherein, the updated cluster centers are calculated based on L first updated cluster sets, and the updated cluster centers are used to re-cluster the classification features of R classification samples, or to add cluster labels to the classification samples in the first updated cluster sets; or, the updated cluster centers are calculated based on S1 second updated cluster sets, and the updated cluster centers are used to re-cluster the classification features of R classification samples, or to add cluster labels to the classification samples in the second updated cluster sets; or, the updated cluster centers are calculated based on S2 third updated cluster sets, and the updated cluster centers are used to re-cluster the classification features of R classification samples, or to add cluster labels to the classification samples in the third updated cluster sets.

[0146] It should be noted that the comparisons of S1, S2, L, and n mentioned above are all for the purpose of ultimately obtaining n target cluster sets. In this embodiment, the aforementioned processes of recalculating sample clusters, updating the merging and splitting of cluster sets, and continuing iterative processing will be repeatedly executed according to the comparison results of S1, S2, L, and n and the iteration conditions, until n target cluster sets are obtained and the current clustering count satisfies the iteration count defined in the iteration conditions.

[0147] In this embodiment, a computer device can acquire sample feature data of M samples, the sample feature data including N sample features, each sample feature corresponding to a sample feature type. Based on the sample features of Q samples under P sample feature types, multiple outlier sample groups are determined. Based on the sample features included in each outlier sample group, the cluster center of each outlier sample group is calculated to obtain P first cluster centers. A cluster number threshold n is obtained. Based on the P first cluster centers, n target cluster centers are obtained. Based on the classification features in R classification samples and the n target cluster centers, the R classification samples are clustered to obtain n target cluster sets. The first cluster centers calculated from outlier sample groups are more accurate than randomly assigned cluster centers. Furthermore, when the number of first cluster centers obtained from multiple outlier sample groups is not equal to the cluster size threshold n, n target cluster centers can be formed by supplementing the first cluster centers together with the existing cluster centers, or n target cluster centers can be selected from P first cluster centers. Then, the R classification samples are clustered based on the n cluster centers to obtain n target cluster sets. This allows for the free adjustment of the number of target cluster centers according to clustering needs while improving the accuracy of the clustering results.

[0148] Based on the data clustering method provided in the embodiments of this application, please refer to [link to relevant documentation]. Figure 4 , Figure 4 This application also provides a more specific method for clustering data, the specific implementation process of which is as follows:

[0149] S401. Read in the sample feature data of M samples. The sample feature data may include N sample features. The computer device can perform numerical transformation according to the feature type of each sample feature to obtain a matrix result table consisting of the N sample features of each sample and the sample feature value of each sample feature. This matrix result table can be as shown in Table 2 above, where the sample feature value in each column of the matrix result table is greater than or equal to 0. The method for performing numerical transformation according to the feature type of each sample feature can be found in the corresponding implementation steps in step S102 above, and will not be repeated here.

[0150] S402. Based on the sample features of Q samples under P sample feature types, outlier sample groups are determined using the Z-score method. In step S401, a matrix result table was obtained. The matrix result table shows that each column represents the sample feature of each sample under a sample feature type. The computer device can then calculate the Z-score of each sample in each column based on the sample features of each sample in each of the P columns. Finally, K samples corresponding to Z-scores that meet the second difference threshold are selected from each column as outlier sample groups under the corresponding sample feature type, resulting in P outlier sample groups.

[0151] S403. Based on the sample characteristics in each outlier sample group, calculate the cluster center of each outlier sample group to obtain P first cluster centers.

[0152] The specific implementation of steps S401-S402 can be found in the above embodiments. Figure 1c and Figure 2 The implementation methods of the corresponding steps are not elaborated here.

[0153] S404. Obtain parameters, i.e., the input parameters may include the cluster size threshold n, the number threshold N_min, and the iteration condition. The iteration condition is whether the current clustering count is greater than the maximum iteration count y.

[0154] S405. Determine n target cluster centers based on P first cluster centers, wherein if n is greater than P, the n target cluster centers are obtained by combining P first cluster centers and nP second cluster centers, and each of the nP second cluster centers is determined based on the sample characteristics in a supplementary sample group obtained from M samples; if n is less than or equal to P, the n target cluster centers are selected from P first cluster centers.

[0155] S406. For each of the R classification samples, calculate its distance to the n target cluster centers, and assign each classification sample to the class with the smallest distance. The n target cluster centers here can be the n target cluster centers obtained in S405, or the n updated cluster centers calculated in step S409. The computer device can cluster the R classification samples based on the classification features and the n target cluster centers to obtain n target cluster sets.

[0156] S407. Determine whether the number of classification samples included in each class is greater than N_min. If so, execute S409; if not, execute S408. Each class involved in step S407 corresponds to a clustering set. The clustering set corresponding to each class in step S407 can be the clustering set obtained after clustering each classification sample in S406. In some embodiments, it can also be the clustering set obtained after splitting in S415.

[0157] S408. Reassign the classification samples in the classes with less than N_min to other classes. Recalculate the sample clustering for the classification samples in the classes with less than N_min, assign the classification samples in the classes with less than N_min to the class where the clustering center with the smallest assignment distance is located, and delete the class with less than N_min.

[0158] S409. For each class, recalculate and update the clustering center, and execute step S410. The class mentioned in step S409 can be the class after clustering in S406; or, the clustering set corresponding to each class in step S409 can also be the class obtained after merging the two classes with the smallest distance in step S414, or each class in step S407 can also be the class obtained after randomly splitting the class with the largest variance (or sample feature variance value) under the sample feature type in step S415. Or, each class in step S409 can also be the class obtained after reassigning the classification samples in the classes with less than N_min to other classes and deleting the classes with less than N_min. The computer device can recalculate and update the clustering center for each class according to the classification features of the classification samples included in each class, and obtain the updated clustering center for each class. In some other embodiments, after recalculating and updating the clustering center for each class, S411 or S412 can also be directly executed.

[0159] S410. Determine whether the number of classes > n. If so, execute step S414; if not, execute step S411.

[0160] S411. Determine whether the number of classes < n. If so, execute step S415; if not (i.e., the number of classes is equal to n), execute step S412.

[0161] S412. Determine whether the current clustering count <= y. If so, use the updated clustering center calculated for each class in S409 as the n target clustering centers, and execute step S406; if not, execute step S413. The so-called current clustering count can be understood as the number of current iterations. The cumulative number of iterations for processing S406 to S412 needs to meet the iteration count threshold in the iteration condition. The iteration count threshold can be, for example, 2000 times.

[0162] S413. Obtain n target classes, which can also be understood as obtaining n cluster sets. In one embodiment, after repeatedly iterating through S406 to S412 above, n target classes or n target cluster sets will eventually be obtained.

[0163] S414. Merge the two classes with the smallest distance and proceed to step S409. In one embodiment, when the number of classes > n, the distance between the updated cluster centers of every two classes can be calculated, and the two classes with the smallest distance can be merged. In some embodiments, the three or four classes with the smallest distance can also be merged.

[0164] S415. Split the class with the largest variance value under the sample feature type, for example, by splitting it randomly, and then proceed to step S409. In one embodiment, the variance value under each sample feature type in each class can be calculated, and the class with the largest variance value under the sample feature type can be randomly split. For example, the classification samples in the class with the largest variance value under the sample feature type can be randomly split into two or three split classes.

[0165] Compared with the original K-means clustering, the data clustering processing method provided in this application has the following advantages: (1) The cluster center of the outlier sample group is calculated by the Z-score method, and some cluster centers can be pre-initialized to reduce the instability of kmeans clustering under different times and different random seeds, while improving the speed of clustering convergence; (2) The splitting and merging operations are introduced to improve the computational efficiency.

[0166] The data clustering method described above can be used to cluster R categorized samples. Then, a computer device can add cluster labels to some or all of the categorized samples in n target cluster sets, and use the labeled categorized samples to train the clustering model. Based on this, this application also provides a model training method; please refer to [link to relevant documentation]. Figure 5 The model training method may include the following steps S501-S503:

[0167] S501. Obtain m classifiers with cluster labels from R classifier samples. Where m is an integer greater than or equal to 1 and less than or equal to R.

[0168] After clustering the R classification samples using the data clustering processing method to obtain n target cluster sets, each target cluster set has a corresponding cluster label. The computer device can add the corresponding target cluster set cluster label to some or all of the classification samples in each target cluster set. Then, select m classification samples with cluster labels from the R classification samples. Among them, the selection of m classification samples with cluster labels from the R classification samples is divided into two cases: (1) Each classification sample in the R classification samples has a cluster label, and the computer device can randomly select m classification samples with cluster labels from the R classification samples, or since the R classification samples are clustered into n target cluster sets, the computer device can randomly select classification samples with cluster labels from each target cluster set according to the number of classification samples in each target cluster set to obtain m classification samples with cluster labels. For example, m=3, target cluster set 1 includes 2 classification samples with cluster labels, target cluster set 2 includes 4 classification samples with cluster labels, one classification sample with cluster labels can be selected from target cluster set 1, and two classification samples with cluster labels can be selected from target cluster set 2, finally resulting in 3 classification samples with cluster labels. (2) Among R classification samples, m classification samples have cluster labels, and the computer device can select m classification samples with cluster labels from R classification samples.

[0169] In one embodiment, after obtaining n target cluster sets, a corresponding cluster label can be assigned to each target cluster set. The cluster label for each target cluster set can be determined based on its cluster centers. For example, a computer device can number each cluster center and use that number as the cluster label for the corresponding target cluster set. For instance, the cluster centers of target cluster set 1 are labeled as label 1, and the cluster centers of target cluster set 2 are labeled as label 2. Alternatively, as mentioned earlier, each cluster center reflects the average distribution of sample characteristics among the samples in the corresponding target cluster set. Therefore, the cluster centers of each target cluster set can be compared, and the cluster label for each target cluster set can be determined based on the comparison results. If, by comparing the cluster centers of each target cluster set, the cluster centers of target cluster set 1 are very frequently active in the sample feature type "most frequent consumption scenario: transportation," then the cluster label for target cluster set 1 can be a travel group. If, by comparing the cluster centers of each target cluster set, the cluster centers of target cluster set 1 are in the middle range across multiple sample feature types, then the cluster label for target cluster set 1 can be a comprehensive group. The cluster centers of the target cluster sets can be the updated cluster centers or the target cluster centers obtained in the above embodiments.

[0170] S502. Call the initial clustering model to perform cluster analysis on m classified samples with clustering labels among R classified samples, and obtain the predicted labels of m classified samples.

[0171] S503. Optimize the initial clustering model based on the predicted label and clustering label of each of the m classification samples to obtain the target clustering model.

[0172] In one embodiment, after step S402, the computer device can obtain the predicted label of each classification sample, then calculate the model loss value of the initial clustering model based on the predicted label and cluster label of each classification sample, and adjust the model parameters in the initial clustering model in the direction of reducing the model loss value to obtain the target clustering model.

[0173] As can be seen from the foregoing, the clustering method provided in this application for clustering data to obtain the target cluster set by clustering R classification samples is relatively accurate. Since the m classification samples with cluster labels come from the target cluster set, in this application, the m classification samples with cluster labels among the R classification samples are used to train the initial clustering model, which can make the clustering performance of the trained target clustering model more reliable and stable.

[0174] Based on the description of the above-described data clustering processing method embodiments, this application also discloses a data clustering processing apparatus. This data clustering processing apparatus can be a computer program (including program code) running on the aforementioned computer device. This data clustering processing apparatus can execute... Figures 1c-5 The method shown. Please refer to [link / reference]. Figure 6 The data clustering processing device may include the following units:

[0175] The acquisition unit 601 is used to acquire sample feature data of M samples, wherein the sample feature data includes N sample features, each sample feature corresponds to a sample feature type, and M and N are positive integers greater than 1;

[0176] The processing unit 602 is used to determine multiple outlier sample groups based on the sample features of Q samples under P sample feature types, wherein each sample feature under the i-th sample feature type in the i-th outlier sample group satisfies the outlier condition; where P is an integer greater than 1 and less than or equal to N, i is an integer greater than 1 and less than or equal to P, and Q is an integer greater than 1 and less than or equal to M.

[0177] The processing unit 602 is further configured to calculate the cluster center of each outlier sample group based on the sample characteristics included in each outlier sample group, and obtain P first cluster centers.

[0178] The processing unit 602 is further configured to perform clustering processing on the classification features of R classification samples based on the P first cluster centers to obtain clustering results, wherein R is an integer greater than or equal to 1.

[0179] In one embodiment, when determining multiple outlier sample groups based on the sample features of Q samples under P sample feature types, the processing unit 602 can be specifically used for:

[0180] Differential analysis was performed on the sample characteristics of Q samples under P sample feature types to obtain the outlier feature reference value for each sample under each sample feature type.

[0181] The K samples corresponding to the outlier reference values ​​that meet the outlier conditions under the target sample feature type from the P sample feature types are determined as the outlier sample group corresponding to the target sample feature type, where K is an integer greater than or equal to 1 and less than or equal to Q.

[0182] In one embodiment, when clustering the classification features of R classification samples based on the P first cluster centers to obtain clustering results, the acquisition unit 601 is used to acquire a cluster number threshold n; the processing unit 602 is used to obtain n target cluster centers based on the P first cluster centers; and to cluster the R classification samples based on the classification features in the R classification samples and the n target cluster centers to obtain n target cluster sets; wherein, if n is greater than P, the n target cluster centers are obtained by combining the P first cluster centers and nP second cluster centers, and each of the nP second cluster centers is determined based on the sample features in the supplementary sample group obtained from the M samples; if n is less than or equal to P, the n target cluster centers are selected from the P first cluster centers.

[0183] In one embodiment, when the processing unit 601 clusters the R classification samples according to the classification features in the R classification samples and the n target cluster centers to obtain n target cluster sets, it can be specifically used for:

[0184] Based on the classification features of the R classification samples and the n target cluster centers, the R classification samples are clustered to obtain n initial cluster sets;

[0185] Based on the n initial cluster sets, the number of samples in each initial cluster set, the current number of clustering iterations, and the iteration conditions, n target cluster sets are determined.

[0186] In one embodiment, when determining n target cluster sets based on the n initial cluster sets, the number of samples in each initial cluster set, the current clustering count, and iteration conditions, the processing unit 602 may specifically be used for:

[0187] Determine whether the n initial cluster sets include the target initial cluster set, wherein the number of classified samples included in the target initial cluster set is less than a number threshold;

[0188] If the n initial cluster sets include the target initial cluster set, then the sample clustering calculation is performed again on each classification sample in the target initial cluster set to obtain L first updated cluster sets;

[0189] If the n initial cluster sets do not include the target initial cluster set, determine whether the current clustering count satisfies the iteration condition;

[0190] If the current number of clustering iterations satisfies the iteration condition, then the n initial clustering sets are determined as n target clustering sets.

[0191] In one embodiment, after the processing unit 602 re-clusters the samples in the target initial clustering set to obtain L first updated clustering sets, it is further configured to:

[0192] Compare L with n;

[0193] If L is greater than n, then the L first updated cluster sets are merged to obtain S1 second updated cluster sets;

[0194] If L is less than n, then the first updated cluster set with the largest sample feature variance under the sample feature type is split to obtain S2 third updated cluster sets.

[0195] In one embodiment, after the processing unit 602 re-calculates the sample clustering of each classification sample in the target initial clustering set to obtain L first updated clustering sets, it is further configured to: if S1 is equal to n, determine whether the current clustering count satisfies the iteration condition; if the current clustering count satisfies the iteration condition, determine the S1 second updated clustering sets as n target clustering sets.

[0196] Alternatively, if S2 equals n, then determine whether the current clustering count satisfies the iteration condition. If the current clustering count satisfies the iteration condition, then determine the S2 third updated clustering sets as n target clustering sets.

[0197] In one embodiment, before determining whether the current clustering count satisfies the iteration condition, the processing unit 602 is further configured to:

[0198] Get updated cluster centers;

[0199] The updated cluster centers are calculated based on L first updated cluster sets. The updated cluster centers are used to re-cluster the classification features of the R classification samples or to add cluster labels to the classification samples in the first updated cluster sets.

[0200] Alternatively, the updated cluster centers are calculated based on S1 second updated cluster sets. The updated cluster centers are used to re-cluster the classification features of the R classification samples or to add cluster labels to the classification samples in the second updated cluster sets.

[0201] Alternatively, the updated cluster centers are calculated based on S2 third updated cluster sets. These updated cluster centers are used to re-cluster the classification features of the R classification samples or to add cluster labels to the classification samples in the third updated cluster sets.

[0202] In one embodiment, the processing unit 602 is further configured to:

[0203] The initial clustering model is invoked to perform clustering analysis on m classification samples with clustering labels among the R classification samples, and the predicted labels of the m classification samples are obtained, where m is an integer greater than or equal to 1 and less than or equal to R;

[0204] The initial clustering model is optimized based on the predicted label and cluster label of each of the m classification samples to obtain the target clustering model.

[0205] In one embodiment, when the processing unit 602 clusters the R classification samples according to the classification features in the R classification samples and the n target cluster centers to obtain n target cluster sets, it can be specifically used for:

[0206] Based on the classification features included in each of the R classification samples, calculate the distance from each of the R classification samples to each of the n target cluster centers;

[0207] Each classified sample is recorded in the cluster set containing the target cluster center with the smallest distance, resulting in n target cluster sets.

[0208] In one embodiment, when the processing unit 602 calculates the cluster centers of each outlier sample group based on the sample characteristics included in each outlier sample group, and obtains P first cluster centers, it can be specifically used for:

[0209] The average value of each sample feature belonging to the same sample feature type in each outlier sample group is calculated to obtain the average value corresponding to each sample feature type in each outlier sample group.

[0210] The average values ​​corresponding to each sample feature type in each outlier sample group are combined to obtain P first cluster centers.

[0211] According to one embodiment of this application, Figures 1c-5 Each step involved in the method shown can be performed by... Figure 6 The clustering processing of data shown is performed by each unit in the apparatus. For example, Figure 1c The step S201 shown is by Figure 6 The acquisition unit 601 shown is used to execute steps S102-S104. Figure 6 The processing unit 602 shown is used to execute this. For example, Figure 2 The step S201 shown is by Figure 6 The acquisition unit 601 shown is used to execute steps S202-S203. Figure 6 The processing unit 602 shown is responsible for executing step 204. Figure 6 The acquisition unit 601 shown is used to execute steps S205-S206. Figure 6 The processing unit 602 shown is used to execute this. For example, Figure 3 Steps s11-s14 shown are by Figure 6 The processing unit 602 shown is used to execute this; for example, Figure 5 The step S501 shown is by Figure 6 The acquisition unit 601 shown is used to execute steps S502-S503. Figure 6 The processing unit 601 shown is used to execute this.

[0212] According to another embodiment of this application, Figure 6 The data clustering processing apparatus shown can be composed of individual or combined units into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. The above units are based on logical function division. In practical applications, the function of one unit can be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the data clustering processing apparatus may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0213] According to another embodiment of this application, processing elements and storage elements, such as a central processing unit (CPU), random access storage medium (RAM), and read-only storage medium (ROM), can be used. For example, a general-purpose computing device such as a computer can run on a device capable of performing tasks such as... Figures 1c-5 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 6 The diagram illustrates a data clustering processing apparatus and a data clustering processing method for implementing embodiments of this application. The computer program may be recorded on, for example, a computer-readable recording medium, loaded onto the aforementioned computer device via the same medium, and executed therein.

[0214] In this embodiment, after acquiring the sample feature data of M samples, the computer device can determine multiple outlier sample groups based on the sample features of Q samples under P sample feature types. Each sample feature under the i-th sample feature type in the i-th outlier sample group satisfies the outlier condition. Based on the sample features included in each outlier sample group, the cluster centers of each outlier sample group are calculated, resulting in P first cluster centers. The classification features of R categorized samples are then clustered based on the P first cluster centers to obtain clustering results. Samples corresponding to the outlier conditions under the same feature type can be grouped into the same outlier sample group. Then, the cluster centers of multiple outlier sample groups are calculated. The cluster centers obtained in this way are more accurate than randomly assigned cluster centers. Further clustering of categorized samples based on the obtained cluster centers improves the accuracy of the clustering results.

[0215] Based on the above description of the data clustering processing method embodiments, this application also discloses a computer device, please refer to... Figure 7 The computer device may include at least a processor 701, an input device 702, an output device 703, and a memory 704. The processor 701, input device 702, output device 703, and memory 704 within the computer device may be connected via a bus or other means.

[0216] The memory 704 is a memory device in a computer device used to store programs and data. It is understood that the memory 704 here can include the computer device's built-in storage medium, or it can include extended storage media supported by the computer device. The memory 704 provides storage space for the computer device's operating system. Furthermore, the computer program (including program code) is also stored in this storage space. It should be noted that the computer storage medium here can be a high-speed RAM memory; optionally, it can also be at least one computer storage medium located remotely from the aforementioned processor, which can be called a Central Processing Unit (CPU), the core and control center of the computer device, used to run the computer programs stored in the memory 704.

[0217] In one implementation, a processor 701 can load and execute a computer program stored in memory 704 to implement the corresponding steps of the method described in the above-described embodiments of the data clustering processing method; specifically, the processor 701 loads and executes the computer program stored in memory 704 for:

[0218] Obtain sample feature data for M samples, wherein the sample feature data includes N sample features, each sample feature corresponds to a sample feature type, and M and N are positive integers greater than 1;

[0219] Based on the sample features of Q samples under P sample feature types, multiple outlier sample groups are determined. In the i-th outlier sample group, each sample feature under the i-th sample feature type satisfies the outlier condition. Here, P is an integer greater than 1 and less than or equal to N, i is an integer greater than 1 and less than or equal to P, and Q is an integer greater than 1 and less than or equal to M.

[0220] Based on the characteristics of the samples included in each outlier group, the cluster center of each outlier group is calculated to obtain P first cluster centers.

[0221] Clustering is performed on the classification features of R classification samples based on the P first cluster centers to obtain the clustering result, where R is an integer greater than or equal to 1.

[0222] In one implementation, when determining multiple outlier sample groups based on the sample features of Q samples under P sample feature types, the processor 701 loads and executes the computer program stored in the memory 704, specifically for:

[0223] Differential analysis was performed on the sample characteristics of Q samples under P sample feature types to obtain the outlier feature reference value for each sample under each sample feature type.

[0224] The K samples corresponding to the outlier reference values ​​that meet the outlier conditions under the target sample feature type from the P sample feature types are determined as the outlier sample group corresponding to the target sample feature type, where K is an integer greater than or equal to 1 and less than or equal to Q.

[0225] In one implementation, when clustering the R classification samples based on the classification features of the P first cluster centers to obtain the clustering result, the processor 701 loads and executes the computer program stored in the memory 704, specifically for:

[0226] Obtain the threshold n for the number of clusters;

[0227] Based on P first cluster centers, n target cluster centers are obtained;

[0228] Based on the classification features of the R classification samples and the n target cluster centers, the R classification samples are clustered to obtain n target cluster sets;

[0229] Wherein, if n is greater than P, the n target cluster centers are obtained by combining P first cluster centers and nP second cluster centers, and each of the nP second cluster centers is determined based on the sample characteristics in the supplementary sample group obtained from the M samples; if n is less than or equal to P, the n target cluster centers are selected from P first cluster centers.

[0230] In one embodiment, when clustering the R classification samples according to the classification features in the R classification samples and the n target cluster centers to obtain n target cluster sets, the processor 701 loads and executes the computer program stored in the memory 704, specifically for:

[0231] Based on the classification features of the R classification samples and the n target cluster centers, the R classification samples are clustered to obtain n initial cluster sets;

[0232] Based on the n initial cluster sets, the number of samples in each initial cluster set, the current number of clustering iterations, and the iteration conditions, n target cluster sets are determined.

[0233] In one embodiment, when determining n target cluster sets based on the n initial cluster sets, the number of samples in each initial cluster set, the current clustering count, and iteration conditions, the processor 701 loads and executes the computer program stored in the memory 704, specifically for:

[0234] Determine whether the n initial cluster sets include the target initial cluster set, wherein the number of classified samples included in the target initial cluster set is less than a number threshold;

[0235] If the n initial cluster sets include the target initial cluster set, then the sample clustering calculation is performed again on each classification sample in the target initial cluster set to obtain L first updated cluster sets;

[0236] If the n initial cluster sets do not include the target initial cluster set, determine whether the current clustering count satisfies the iteration condition;

[0237] If the current number of clustering iterations satisfies the iteration condition, then the n initial clustering sets are determined as n target clustering sets.

[0238] In one embodiment, after re-clustering the samples in the initial target clustering set to obtain L first updated clustering sets, the processor 701 loads and executes the computer program stored in the memory 704, and is further configured to:

[0239] Compare L with n;

[0240] If L is greater than n, then the L first updated cluster sets are merged to obtain S1 second updated cluster sets;

[0241] If L is less than n, then the first updated cluster set with the largest sample feature variance under the sample feature type is split to obtain S2 third updated cluster sets.

[0242] In one embodiment, after re-clustering the samples in the initial target clustering set to obtain L first updated clustering sets, the processor 701 loads and executes the computer program stored in the memory 704, and is further configured to:

[0243] If S1 equals n, then determine whether the current clustering count satisfies the iteration condition. If the current clustering count satisfies the iteration condition, then determine the S1 second updated clustering sets as n target clustering sets.

[0244] Alternatively, if S2 equals n, then determine whether the current clustering count satisfies the iteration condition. If the current clustering count satisfies the iteration condition, then determine the S2 third updated clustering sets as n target clustering sets.

[0245] In one embodiment, before determining whether the current clustering count meets the iteration condition, the processor 701 loads and executes the computer program stored in the memory 704, and is further configured to:

[0246] Get updated cluster centers;

[0247] The updated cluster centers are calculated based on L first updated cluster sets. The updated cluster centers are used to re-cluster the classification features of the R classification samples or to add cluster labels to the classification samples in the first updated cluster sets.

[0248] Alternatively, the updated cluster centers are calculated based on S1 second updated cluster sets. The updated cluster centers are used to re-cluster the classification features of the R classification samples or to add cluster labels to the classification samples in the second updated cluster sets.

[0249] Alternatively, the updated cluster centers are calculated based on S2 third updated cluster sets. These updated cluster centers are used to re-cluster the classification features of the R classification samples or to add cluster labels to the classification samples in the third updated cluster sets.

[0250] In one embodiment, the processor 701 loads and executes a computer program stored in the memory 704, and is further configured to:

[0251] The initial clustering model is invoked to perform clustering analysis on m classification samples with clustering labels among the R classification samples, and the predicted labels of the m classification samples are obtained, where m is an integer greater than or equal to 1 and less than or equal to R;

[0252] The initial clustering model is optimized based on the predicted label and cluster label of each of the m classification samples to obtain the target clustering model.

[0253] In one embodiment, when clustering the R classification samples according to the classification features in the R classification samples and the n target cluster centers to obtain n target cluster sets, the processor 701 loads and executes the computer program stored in the memory 704, for:

[0254] Based on the classification features included in each of the R classification samples, calculate the distance from each of the R classification samples to each of the n target cluster centers;

[0255] Each classified sample is recorded in the cluster set containing the target cluster center with the smallest distance, resulting in n target cluster sets.

[0256] In one embodiment, when calculating the cluster centers of each outlier sample group based on the sample characteristics included in each outlier sample group to obtain P first cluster centers, the processor 701 loads and executes the computer program stored in the memory 704, and is further configured to:

[0257] The average value of each sample feature belonging to the same sample feature type in each outlier sample group is calculated to obtain the average value corresponding to each sample feature type in each outlier sample group.

[0258] The average values ​​corresponding to each sample feature type in each outlier sample group are combined to obtain P first cluster centers.

[0259] It should be understood that, in the embodiments of this application, the processor 701 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0260] In this embodiment, after acquiring the sample feature data of M samples, the computer device can determine multiple outlier sample groups based on the sample features of Q samples under P sample feature types. Each sample feature under the i-th sample feature type in the i-th outlier sample group satisfies the outlier condition. Based on the sample features included in each outlier sample group, the cluster centers of each outlier sample group are calculated, resulting in P first cluster centers. The classification features of R categorized samples are then clustered based on the P first cluster centers to obtain clustering results. Samples corresponding to the outlier conditions under the same feature type can be grouped into the same outlier sample group. Then, the cluster centers of multiple outlier sample groups are calculated. The cluster centers obtained in this way are more accurate than randomly assigned cluster centers. Further clustering of categorized samples based on the obtained cluster centers improves the accuracy of the clustering results.

[0261] This application provides a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor, can perform the steps performed in all the above embodiments.

[0262] This application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. When the computer instructions are executed by the processor of a computer device, they perform the methods described in all the above embodiments.

[0263] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0264] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Those skilled in the art will understand that implementing all or part of the processes of the above embodiments and making equivalent changes in accordance with the claims of this application are still within the scope of the invention.

Claims

1. A method for clustering data, characterized in that, Applications in cluster analysis equipment include: The system receives sample feature data of M samples collected and sent by a data acquisition device. The sample feature data includes N sample features, each of which corresponds to a sample feature type. M and N are positive integers greater than 1. The samples include objects that need to be clustered, and the objects include images. Based on the sample features of Q samples under P sample feature types, P outlier sample groups are determined, with each outlier sample group corresponding to a sample feature type. Each sample feature under the j-th sample feature type in the i-th outlier sample group satisfies the outlier condition. Here, P is an integer greater than 1 and less than or equal to N, i and j are integers greater than 1 and less than or equal to P, and Q is an integer greater than 1 and less than or equal to M. The condition that each sample feature under the j-th sample feature type in the i-th outlier sample group satisfies the outlier condition means that the difference between the sample feature value of each sample feature under the j-th sample feature type in the i-th outlier sample group and the reference feature value under the j-th sample feature type is greater than a preset difference threshold; or, the outlier feature reference value of each sample feature under the j-th sample feature type in the i-th outlier sample group is greater than a preset threshold. The outlier feature reference value represents the difference between the feature value corresponding to the sample feature and the mean of the sample feature under that sample feature type. Based on the characteristics of the samples included in each outlier group, the cluster center of each outlier group is calculated to obtain P first cluster centers. Clustering is performed on the classification features of R classification samples based on the P first cluster centers to obtain the clustering result, where R is an integer greater than or equal to 1; The clustering results are sent to the data recommendation device to trigger the data recommendation device to push information to the classification samples in the corresponding cluster set according to the clustering labels of each cluster set included in the clustering results.

2. The method as described in claim 1, characterized in that, The step of determining P outlier groups based on the sample characteristics of Q samples under P sample feature types includes: Differential analysis was performed on the sample characteristics of Q samples under P sample feature types to obtain the outlier feature reference value for each sample under each sample feature type. The K samples corresponding to the outlier reference values ​​that meet the outlier conditions under the target sample feature type from the P sample feature types are determined as the outlier sample group corresponding to the target sample feature type, where K is an integer greater than or equal to 1 and less than or equal to Q.

3. The method as described in claim 1, characterized in that, The step of clustering R classification samples based on the classification features of the P first cluster centers to obtain clustering results includes: Obtain the threshold n for the number of clusters; Based on P first cluster centers, n target cluster centers are obtained; Based on the classification features of the R classification samples and the n target cluster centers, the R classification samples are clustered to obtain n target cluster sets; Wherein, if n is greater than P, the n target cluster centers are obtained by combining P first cluster centers and nP second cluster centers, and each of the nP second cluster centers is determined based on the sample characteristics in the supplementary sample group obtained from the M samples; if n is less than or equal to P, the n target cluster centers are selected from P first cluster centers.

4. The method as described in claim 3, characterized in that, The step involves clustering the R classification samples based on their classification features and the n target cluster centers to obtain n target cluster sets, including: Based on the classification features of the R classification samples and the n target cluster centers, the R classification samples are clustered to obtain n initial cluster sets; Based on the n initial cluster sets, the number of categorized samples included in each initial cluster set, the current clustering count, and the iteration conditions, n target cluster sets are determined.

5. The method as described in claim 4, characterized in that, The process of determining n target cluster sets based on the n initial cluster sets, the number of samples in each initial cluster set, the current clustering count, and iteration conditions includes: Determine whether the n initial cluster sets include the target initial cluster set, wherein the number of classified samples included in the target initial cluster set is less than a number threshold; If the n initial cluster sets include the target initial cluster set, then the sample clustering calculation is performed again on each classification sample in the target initial cluster set to obtain L first updated cluster sets; If the n initial cluster sets do not include the target initial cluster set, determine whether the current clustering count satisfies the iteration condition; If the current number of clustering iterations satisfies the iteration condition, then the n initial clustering sets are determined as n target clustering sets.

6. The method as described in claim 5, characterized in that, After re-clustering each classification sample in the initial target cluster set to obtain L first updated cluster sets, the process further includes: Compare L with n; If L is greater than n, then the L first updated cluster sets are merged to obtain S1 second updated cluster sets; If L is less than n, then the first updated cluster set with the largest sample feature variance under the sample feature type is split to obtain S2 third updated cluster sets.

7. The method as described in claim 6, characterized in that, After re-clustering each classification sample in the initial target cluster set to obtain L first updated cluster sets, the process further includes: If S1 equals n, then determine whether the current clustering count satisfies the iteration condition. If the current clustering count satisfies the iteration condition, then determine the S1 second updated clustering sets as n target clustering sets. Alternatively, if S2 equals n, then determine whether the current clustering count satisfies the iteration condition. If the current clustering count satisfies the iteration condition, then determine the S2 third updated clustering sets as n target clustering sets.

8. The method as described in claim 5 or 7, characterized in that, Before determining whether the current clustering count meets the iteration condition, the process also includes: Get updated cluster centers; The updated cluster centers are calculated based on L first updated cluster sets. The updated cluster centers are used to re-cluster the classification features of the R classification samples or to add cluster labels to the classification samples in the first updated cluster sets. Alternatively, the updated cluster centers are calculated based on S1 second updated cluster sets. The updated cluster centers are used to re-cluster the classification features of the R classification samples or to add cluster labels to the classification samples in the second updated cluster sets. Alternatively, the updated cluster centers are calculated based on S2 third updated cluster sets. These updated cluster centers are used to re-cluster the classification features of the R classification samples or to add cluster labels to the classification samples in the third updated cluster sets.

9. The method as described in claim 8, characterized in that, Also includes: The initial clustering model is invoked to perform clustering analysis on m class samples with clustering labels among the R class samples, and the predicted labels of the m class samples are obtained, where m is an integer greater than or equal to 1 and less than or equal to R; The initial clustering model is optimized based on the predicted label and cluster label of each of the m classification samples to obtain the target clustering model.

10. The method as described in claim 3, characterized in that, The step involves clustering the R classification samples based on their classification features and the n target cluster centers to obtain n target cluster sets, including: Based on the classification features included in each of the R classification samples, calculate the distance from each of the R classification samples to each of the n target cluster centers; Each classified sample is recorded in the cluster set containing the target cluster center with the smallest distance, resulting in n target cluster sets.

11. The method as described in claim 1, characterized in that, The step involves calculating the cluster centers of each outlier group based on the characteristics of the samples included in each outlier group, resulting in P first cluster centers, including: The average value of each sample feature belonging to the same sample feature type in each outlier sample group is calculated to obtain the average value corresponding to each sample feature type in each outlier sample group. The average values ​​corresponding to each sample feature type in each outlier sample group are combined to obtain P first cluster centers.

12. A data clustering processing apparatus, characterized in that, The device is used in cluster analysis equipment and includes: The acquisition unit is used to receive sample feature data of M samples collected and sent by the data acquisition device. The sample feature data includes N sample features, each sample feature corresponds to a sample feature type, and M and N are positive integers greater than 1. The samples include objects that need to be clustered, and the objects include images. The processing unit is used to determine P outlier sample groups based on the sample features of Q samples under P sample feature types, where each outlier sample group corresponds to a sample feature type. Each sample feature under the j-th sample feature type in the i-th outlier sample group satisfies the outlier condition. Here, P is an integer greater than 1 and less than or equal to N, i and j are integers greater than 1 and less than or equal to P, and Q is an integer greater than 1 and less than or equal to M. The condition that each sample feature under the j-th sample feature type in the i-th outlier sample group satisfies the outlier condition means that the difference between the sample feature value of each sample feature under the j-th sample feature type in the i-th outlier sample group and the reference feature value under the j-th sample feature type is greater than a preset difference threshold; or, the outlier feature reference value of each sample feature under the j-th sample feature type in the i-th outlier sample group is greater than a preset threshold. The outlier feature reference value represents the difference between the feature value corresponding to the sample feature and the mean of the sample features under that sample feature type. The processing unit is also used to calculate the cluster center of each outlier sample group based on the sample characteristics included in each outlier sample group, and obtain P first cluster centers. The processing unit is further configured to perform clustering processing on the classification features of R classification samples based on the P first cluster centers to obtain clustering results, where R is an integer greater than or equal to 1; The processing unit is further configured to send the clustering results to the data recommendation device, thereby triggering the data recommendation device to push information to the classification samples in the corresponding cluster set according to the clustering labels of each cluster set included in the clustering results.

13. A computer device, characterized in that, include: Memory, used to store computer programs; The processor executes a computer program stored in the memory to implement the data clustering processing method as described in any one of claims 1-11.

14. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed by a processor, is used to implement the data clustering processing method as described in any one of claims 1-11.

15. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, is used to implement the data clustering processing method as described in any one of claims 1-11.

Images