Big model-based user clustering evaluation method and system, storage medium and computer device

Abstract

The application discloses a user clustering evaluation method and system based on a large model, a storage medium and a computer device. The method comprises the following steps: obtaining a user clustering scheme and original feature data and clustering index data of each clustering user; calculating statistical effectiveness index values and clustering discrimination index values of the user clustering scheme based on the obtained data; calling an analysis agent to aggregate the original feature data and the clustering index data of each clustering group in the user clustering scheme, and generating a text portrait of each clustering group based on the aggregated data summary and a feature metadata knowledge base; calling an evaluation agent to perform multi-dimensional scoring on the text portrait according to an evaluation rule prior knowledge base, and obtaining an interpretability index value according to the multi-dimensional scoring result; and obtaining an evaluation result of the user clustering scheme based on the statistical effectiveness index values, the clustering discrimination index values and the interpretability index value. The above method can improve the accuracy and interpretability of the user clustering scheme evaluation.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a user clustering evaluation method, system, storage medium, and computer device based on a large model. Background Technology

[0002] User clustering refers to the unsupervised learning process of dividing a massive number of users into several groups with different characteristics based on the similarity of user features. Currently, user clustering technology is widely used in various application scenarios such as personalized recommendation, product marketing, and product optimization. Accurate and reliable user clustering solutions can help operators deeply understand the behavioral characteristics and needs differences of different user groups, thereby formulating differentiated operational strategies.

[0003] Currently, after user clustering schemes are generated, they are mainly evaluated based on statistical indicators, posterior analysis, or human experts. However, evaluation based on statistical indicators is seriously out of touch with actual application scenarios, and the clustering results are not easy to understand. Evaluation schemes based on posterior analysis are time-consuming and costly, and it is difficult to quickly compare multiple alternative schemes. Human expert review schemes rely heavily on the time and experience of experts, and the evaluation results are highly subjective and cannot be reproduced. Summary of the Invention

[0004] In view of this, embodiments of this application provide a user clustering evaluation method, system, storage medium, and computer device based on a large model. The main purpose is to solve the technical problem that existing user clustering evaluation methods are out of touch with application scenarios, resulting in low evaluation accuracy and poor interpretability of evaluation results.

[0005] According to one aspect of this application, a user clustering evaluation method based on a large model is provided, the method comprising: Obtain the user clustering scheme to be evaluated, as well as the original feature data and clustering index data of each clustered user. The user clustering scheme includes user identifiers and clustering labels of multiple clustered users. Based on the user clustering scheme and the original feature data and clustering index data of each clustered user, the statistical validity index value and clustering discriminancy index value of the user clustering scheme are calculated. The analysis agent, which is pre-trained based on a large model, is invoked to aggregate the original feature data and clustering index data of each cluster in the user clustering scheme to obtain a data summary. Based on the data summary and the feature description information in the preset feature metadata knowledge base, a text profile of each cluster is generated. The evaluation agent, which is pre-trained based on a large model, is invoked to perform multi-dimensional scoring on the text profiles of each cluster group according to the evaluation rule information in the pre-set evaluation rule prior knowledge base. Based on the multi-dimensional scoring results of each cluster group, the interpretability index value of the user clustering scheme is obtained. The evaluation results of the user clustering scheme are obtained based on the statistical validity index, cluster discrimination index, and interpretability index of the scheme.

[0006] According to another aspect of this application, a user clustering evaluation system based on a large model is provided. This system includes a data input layer, a multidimensional evaluation layer, and a comprehensive decision layer. The multidimensional evaluation layer includes a statistical validity engine, a clustering discriminability engine, and an interpretability engine, wherein: The data input layer is used to obtain the user clustering scheme to be evaluated, as well as the original feature data and clustering index data of each clustered user. The user clustering scheme includes user identifiers and clustering labels of multiple clustered users. The statistical validity engine is used to calculate the statistical validity index value of the user clustering scheme based on the user clustering scheme and the original feature data of each clustered user. The clustering discriminant engine is used to calculate the clustering discriminant index value of the user clustering scheme based on the user clustering scheme and the original feature data and clustering index data of each clustered user. The interpretability engine is used to call an analytical agent based on a large model pre-training to aggregate the original feature data and clustering index data of each cluster in the user clustering scheme to obtain a data summary, and to generate a text profile of each cluster based on the data summary and feature description information in a preset feature metadata knowledge base. The interpretability engine is also used to call an evaluation agent based on a large model pre-training to perform multi-dimensional scoring on the text profiles of each cluster group according to the evaluation rule information in the preset evaluation rule prior knowledge base, and to obtain the interpretability index value of the user clustering scheme based on the multi-dimensional scoring results of each cluster group. The comprehensive decision-making layer is used to obtain the evaluation results of the user clustering scheme based on the statistical validity index, cluster discrimination index, and interpretability index of the user clustering scheme.

[0007] According to another aspect of this application, a storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described user clustering evaluation method based on a large model.

[0008] According to another aspect of this application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the program to implement the above-described user clustering evaluation method based on a large model.

[0009] By employing the above technical solutions, the user clustering evaluation method, system, storage medium, and computer device based on a large model provided in this application, through acquiring the original feature data and clustering index data of each clustered user, and calculating the statistical validity index value and clustering discriminancy index value of the user clustering scheme, can evaluate the user clustering scheme from the statistical perspective of data density and data separation, as well as the degree of discriminability of each cluster group on the clustering index data, thereby effectively improving the scientificity and accuracy of the evaluation results. Furthermore, by calling an analysis agent pre-trained based on a large model to generate text profiles, and calling an evaluation agent pre-trained based on a large model to perform multi-dimensional scoring on the text profiles, the difficult-to-understand clustering results can be transformed into natural language descriptions, and the concept of interpretability can be transformed into measurable and comparable engineering indicators, thereby improving the understandability of the evaluation results. Furthermore, by weighting and fusing the index values ​​of the three dimensions of statistical validity, cluster discrimination, and interpretability, the evaluation results of user clustering schemes can be obtained. This allows for a comprehensive quantitative evaluation of user clustering schemes from three aspects: data geometric structure, index discrimination ability, and result interpretability. Moreover, the evaluation process can be run offline, enabling rapid evaluation of multiple alternative user clustering schemes in a short period of time, thereby greatly improving iteration efficiency.

[0010] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0011] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This illustration shows a flowchart of a user clustering evaluation method based on a large model provided in an embodiment of this application; Figure 2 This illustration shows a schematic diagram of the structure of a user clustering evaluation system based on a large model, as provided in an embodiment of this application. Detailed Implementation

[0012] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0013] User clustering, also known as user grouping, refers to the unsupervised learning process of dividing a large number of users into several groups with different characteristics based on their feature similarity. In practical applications, it is often necessary to try multiple different clustering algorithms or the number of clusters to generate multiple candidate schemes, and then select one of these schemes as the final clustering scheme. Currently, the industry mainly uses the following methods to evaluate user clustering schemes.

[0014] The first type is the evaluation scheme based on statistical indicators. This is the most traditional and common evaluation scheme. When selecting a clustering scheme, it mainly relies on a certain statistical indicator, such as the elbow method (Sum of Squared Errors, SSE) or silhouette coefficient. By trying different numbers of clusters or algorithms, the scheme that makes the sum of squared errors curve show an inflection point or that makes the silhouette coefficient the highest is selected. However, this evaluation method is seriously out of touch with the actual application goals of clustering. A clustering scheme that performs well in terms of geometric structure may not have obvious discrimination between the groups it divides on key indicators, and this method does not consider the interpretability of the clustering results.

[0015] The second approach is an evaluation scheme based on posterior analysis. In this scheme, after generating multiple clustering schemes, these schemes are applied to real-world business scenarios for testing or by reviewing historical data to analyze the performance differences of the groups segmented under different clustering schemes on key metrics, ultimately selecting the best-performing scheme. However, this method has a long evaluation cycle and high cost, typically requiring online experiments or complex attribution analyses, which cannot meet the needs of rapid iteration. Furthermore, the posterior analysis results at different times and in different scenarios are affected by various factors such as the environment, making static comparisons among multiple candidate schemes difficult.

[0016] The third approach is manual review. This method requires a data science team to conduct preliminary profiling analysis of the clustering results, followed by a review meeting where a qualitative discussion and voting process determines the final adopted solution. However, this method heavily relies on the experts' time and experience, resulting in highly subjective and unreproducible evaluations. Each review may yield different results depending on the participants and the focus of the discussion.

[0017] In addition, some platforms also provide standardized evaluation schemes for user segmentation and profiling analysis. However, these evaluation methods usually only include basic statistical indicators and simple effect recovery analysis. They cannot automatically calculate the cluster discrimination, nor do they have the ability to quantitatively evaluate the interpretability of clustering results.

[0018] To address the above problems, in one embodiment, such as Figure 1 As shown, a user clustering evaluation method based on a large model is provided. Taking the application of this method to computer devices as an example, the method includes the following steps: Step 101: Obtain the user clustering scheme to be evaluated, as well as the original feature data and clustering index data of each clustered user. The user clustering scheme includes user identifiers and clustering labels of multiple clustered users.

[0019] The original feature data refers to the input data used to perform user clustering, including user profile features, behavioral features, spending power features, and fulfillment behavior features. The clustering index data refers to the data used to measure the user's performance in different dimensions, such as the unit price, number of orders, unit economic contribution value, average number of orders per person, and login conversion rate. Each clustering index data is calculated independently for each user.

[0020] Specifically, when evaluating a user clustering scheme, the first step is to receive the user clustering scheme to be evaluated. This scheme consists of a mapping relationship between user identifiers and cluster labels. Then, based on the user identifiers, the original feature data of each clustered user can be obtained from the feature database. The original feature data includes profile features such as age, gender, and city tier; behavioral features such as user activity level, login frequency, and access depth; spending power features such as spending amount and frequency; and fulfillment behavior features such as delivery address stability and return / exchange rate. Simultaneously, based on the user identifiers, clustering indicator data for each user can be obtained from the indicator database. The unit price is calculated by dividing the user's total payment amount by the total number of orders; the order quantity is the total number of orders placed by the user within a preset time range; the unit economic contribution value is the user's net contribution value; the average order quantity per user is the average number of orders placed by the user; and the login conversion rate is the user's conversion rate after logging in.

[0021] Step 102: Based on the user clustering scheme and the original feature data and clustering index data of each clustered user, calculate the statistical validity index value and cluster discrimination index value of the user clustering scheme.

[0022] Among them, the statistical validity index is used to characterize the data density within each cluster and the data separation between different clusters; the cluster discrimination index is used to characterize the degree of distinguishability of each cluster in the cluster index data.

[0023] Specifically, when calculating the statistical validity index, it can be calculated based on indicators such as the silhouette coefficient and the coefficient of variation. The silhouette coefficient can be calculated based on the average distance between the sample user and other users within the same group, as well as the average distance between the sample user and users in other groups. The coefficient of variation can be calculated based on the ratio of the standard deviation to the mean of the number of users in each cluster. Furthermore, the statistical validity index can also be calculated using statistical parameters such as the Davidson-Bolding index or the Dunn index, or it can be obtained by weighted aggregation of multiple statistical indicators, such as weighted aggregation of the silhouette coefficient and the coefficient of variation. This embodiment does not impose specific limitations on these methods. Furthermore, when calculating the cluster discrimination index value, multiple cluster discrimination sub-indices can be calculated based on different clustering index data or different dimensional preference information. These sub-indices are then aggregated to obtain the cluster discrimination index value of the user's clustering scheme. For example, a first cluster discrimination sub-indicator can be calculated based on the variance analysis results of different clustering groups using preset clustering index data. A second cluster discrimination sub-indicator can be calculated based on the Target Group Index (TGI) of each clustering group in different dimensions. The weighted sum of the first and second cluster discrimination sub-indicators yields the cluster discrimination index value, or a single sub-indicator can be used to obtain the cluster discrimination index value. Additionally, the cluster discrimination index value can also be calculated using indicators such as mutual information or adjusted RAND index, which measure the correlation between clustering results and external variables. This embodiment does not impose specific limitations on these methods.

[0024] Step 103: Call the analysis agent based on large model pre-training to aggregate the original feature data and clustering index data of each cluster group in the user clustering scheme to obtain a data summary, and generate a text profile of each cluster group based on the data summary and the feature description information in the preset feature metadata knowledge base.

[0025] Among them, the analytical agent is an agent instance pre-trained based on a large language model, which is given the role of a data analyst. The feature metadata knowledge base stores explanatory information such as the meaning, type and value range of each original feature. The text profile includes information such as group naming, core feature description and feature analysis information.

[0026] Specifically, after obtaining the statistical validity index and cluster discrimination index of the user clustering scheme, a textual profile of each cluster can be generated through an analytical agent. When generating the textual profile, each cluster in the user clustering scheme can be traversed. For the current cluster, the original feature data and clustering index data of all users in that cluster are aggregated to generate a data summary for that cluster. Then, this structured data summary is combined with feature description information from the feature metadata knowledge base to form a prompt message. This prompt message is then input into the analytical agent, which can perform semantic understanding and summarization of the cluster's features based on the data summary and feature description information in the prompt message, thereby generating a textual profile of that cluster. The textual profile can be output in a preset format such as JavaScript object notation and includes fields such as cluster name, core feature, and feature analysis. By calling an analytical agent pre-trained on a large model to generate textual profiles for each cluster, difficult-to-understand clustering results can be transformed into structured natural language descriptions.

[0027] Step 104: Call the evaluation agent based on the large model pre-training to perform multi-dimensional scoring on the text profiles of each cluster group according to the evaluation rule information in the pre-set evaluation rule prior knowledge base, and obtain the interpretability index value of the user clustering scheme based on the multi-dimensional scoring results of each cluster group.

[0028] Among them, the evaluation agent is another agent instance pre-trained based on a large language model. It is given the role of an evaluation expert. The prior knowledge base of evaluation rules stores structured evaluation criteria information, such as the average order value of high-value groups should not be too low, and the order frequency of high-frequency active groups should be high.

[0029] Specifically, after obtaining the textual profiles of each cluster, for each cluster, the textual profile, structured data summary, and evaluation rule information from the prior knowledge base of evaluation rules are combined into a prompt. This prompt is then input into the evaluation agent. The evaluation agent verifies the textual profile based on the evaluation rule information in the prompt. Based on the verification results, a pre-set, structured, and detailed scoring standard is used to score the textual profile in multiple dimensions. For example, the scoring includes whether the cluster naming accurately reflects the characteristics, whether the core feature description is consistent with the original data, whether the feature analysis logic is self-consistent, whether the textual profile reveals meaningful insights, and whether the textual expression is clear and professional. Then, the evaluation agent can perform a weighted calculation of the scores for each dimension of the cluster to obtain the interpretability score of the cluster. Finally, the evaluation agent can output the interpretability score of the cluster, along with the scores for each dimension and the evaluation reasons. Furthermore, after obtaining the multi-dimensional scores and interpretability scores for each cluster, the average interpretability scores of all clusters can be taken to obtain the interpretability index value of the user clustering scheme. By calling the evaluation agent to perform multi-dimensional scoring on the text profile, the concept of interpretability can be transformed into a measurable and comparable engineering metric.

[0030] In an optional implementation, a single agent pre-trained based on a large language model can be invoked. The evaluation rule information from the data summary, feature metadata knowledge base, and prior evaluation rule knowledge base is combined into a prompt message and input into the agent. This agent then simultaneously generates a text profile and performs a multi-dimensional evaluation of the text profile within a single prompt message, outputting a comprehensive evaluation result including the text profile and multi-dimensional scores. Alternatively, the text profiles of each cluster in the user clustering scheme can be presented to real users through questionnaires or interviews, and user feedback on each text profile can be collected. Based on this feedback, an interpretability score for each cluster can be calculated. Finally, the interpretability scores of each cluster are aggregated to obtain the interpretability index value of the user clustering scheme.

[0031] Step 105: Based on the statistical validity index, cluster discrimination index, and interpretability index of the user clustering scheme, obtain the evaluation results of the user clustering scheme.

[0032] Specifically, after obtaining the statistical validity index, cluster discrimination index, and interpretability index of the user clustering scheme, the three index values ​​can be weighted and summed according to preset weights to obtain the comprehensive evaluation score of the user clustering scheme. The evaluation result of the user clustering scheme is then generated based on the comprehensive evaluation score and each index value. Furthermore, textual profiles of each cluster group, along with corresponding multi-dimensional evaluation reasons and scores, can be obtained and included in the evaluation result. By weightedly fusing the index values ​​across the three dimensions and outputting an evaluation report, a comprehensive quantitative basis can be provided for the selection decision of the clustering scheme.

[0033] In a specific example, an evaluation of a user clustering scheme comprising 15 groups is required. The evaluation process begins by acquiring the user clustering scheme, followed by obtaining the original feature data and clustering index data for each user within the cluster. Then, the silhouette coefficient and coefficient of variation of the scheme are calculated based on the original feature data, and aggregated to obtain a statistical validity index of 0.72. Next, an analysis of variance (ANOVA) is performed on the clustering index data, including average order value, order volume, and unit economic contribution value for each user. The ANOVA results for each index are then logarithmically transformed and averaged to obtain a cluster discrimination index of 0.65. Next, the analysis agent is invoked to generate textual profiles for each of the 15 groups using feature description information from the feature metadata knowledge base. For example, the profile for group number three is "quality lifestyle consumers," with core features including high average order value, high repurchase rate, and preference for branded goods. Finally, the evaluation agent is invoked to perform multi-dimensional scoring on the profiles of each group based on evaluation rules from the prior knowledge base, and the aggregated score yields an interpretability index of 0.78. Finally, the three indicator values ​​are weighted and summed according to preset weighting coefficients, such as 0.2, 0.5, and 0.3, resulting in a comprehensive evaluation score of 0.709. Next, the same evaluation process is used to evaluate another user clustering scheme containing 13 clusters. The results show a statistical validity index of 0.74, a cluster discrimination index of 0.82, an interpretability index of 0.88, and a comprehensive evaluation score of 0.822. Based on these evaluation results, the second user clustering scheme has a higher comprehensive evaluation score than the first, therefore, the second user clustering scheme can be considered the preferred clustering scheme.

[0034] The above embodiments, by acquiring the original feature data and clustering index data of each clustered user, and calculating the statistical validity index and cluster discrimination index value of the user clustering scheme, can evaluate the user clustering scheme from the statistical perspective of data density and data separation, as well as the distinguishability of each cluster group on the clustering index data, thereby effectively improving the scientificity and accuracy of the evaluation results. Furthermore, by calling an analysis agent pre-trained on a large model to generate text profiles, and calling an evaluation agent pre-trained on a large model to perform multi-dimensional scoring on the text profiles, the difficult-to-understand clustering results can be transformed into natural language descriptions, and the concept of interpretability can be transformed into measurable and comparable engineering indicators, thereby improving the understandability of the evaluation results. In addition, by weighted fusion of the index values ​​of statistical validity, cluster discrimination, and interpretability to obtain the evaluation results of the user clustering scheme, a comprehensive quantitative evaluation of the user clustering scheme can be performed from three aspects: data geometric structure, index discrimination ability, and result interpretability. Moreover, the evaluation process can be run offline, allowing for rapid evaluation of multiple candidate user clustering schemes in a short time, thereby greatly improving iteration efficiency.

[0035] In one embodiment, in step 102, the statistical validity index of the user clustering scheme can be calculated as follows: Multiple clustered users are extracted from the clustered users of the user clustering scheme as sample users. Then, for each sample user, a first average distance is calculated between the original feature data of the sample user and the original feature data of other users within the same cluster, and a second average distance is calculated between the original feature data of the sample user and the original feature data of multiple users in other clusters. Based on the first and second average distances, the silhouette coefficient of the sample user is obtained. Based on the silhouette coefficients of each sample user, the silhouette coefficient of the user clustering scheme is obtained. Furthermore, the standard deviation and mean of the number of users in each cluster are calculated, and the coefficient of variation of the user clustering scheme is obtained based on the standard deviation and mean of the number of users in each cluster. Finally, the statistical validity index of the user clustering scheme is obtained based on the silhouette coefficient and / or the coefficient of variation of the user clustering scheme.

[0036] Among them, the silhouette coefficient is used to measure the closeness between an individual user and other users in its group, as well as the distance between users in other groups; the size variation coefficient is used to measure the dispersion of the number of users in each cluster.

[0037] Specifically, when calculating the silhouette coefficient, a predetermined proportion of sample users can be randomly selected from all clustered users in the user clustering scheme. For each sample user, a first average distance is calculated between the original feature data of that sample user and the original feature data of other users within the same cluster group. This first average distance represents the average similarity between that user and other users in the same group. Then, a second average distance is calculated between the original feature data of that sample user and the original feature data of multiple users in the nearest other cluster groups. This second average distance represents the average difference between that user and other groups. Next, the silhouette coefficient of that sample user is obtained by dividing the difference between the second average distance and the first average distance by the larger of the first and second average distances. Finally, the silhouette coefficient of the user clustering scheme is obtained by taking the arithmetic mean of the silhouette coefficients of all sample users.

[0038] Furthermore, when calculating the coefficient of variation (COP), the number of users in each cluster can be counted, and the standard deviation and average number of users across all clusters can be calculated. Then, the COP of the user clustering scheme is obtained by dividing the standard deviation by the average number of users. In this embodiment, the normalized value of the silhouette coefficient of the user clustering scheme can be used as the statistical validity index. Alternatively, the silhouette coefficient and COP can be normalized and then weighted and summed according to preset weights to obtain the statistical validity index. A higher silhouette coefficient indicates a closer user group within each cluster and a more distant user group between different clusters; a lower COP indicates a more balanced user group size.

[0039] The above embodiments can uniformly quantify the density within clusters and the separation between clusters by calculating the silhouette coefficients of multiple sample users in the user clustering scheme. By calculating the statistical value of the number of users in each cluster, the scale variation coefficient of the user clustering scheme can be obtained, which can uniformly quantify the scale balance of each cluster. In this way, the clustering scheme can be statistically and accurately evaluated from the two dimensions of group geometry and group scale balance.

[0040] In one embodiment, in step 102, the clustering discrimination index value of the user clustering scheme can be calculated by the following method: For each preset clustering index data, perform variance analysis on the index values ​​of the clustering index data of each cluster group to obtain the discrimination index value of the clustering index data, and perform numerical transformation on the discrimination index values ​​of each clustering index data to obtain a first clustering discrimination sub-index. In addition, obtain multiple preference dimensions of each cluster user, and calculate the target group index of each cluster group on multiple preference dimensions respectively. Based on the target group index of each cluster group on multiple preference dimensions, filter out the target preference dimensions whose target group index exceeds a preset threshold, and obtain a second clustering discrimination sub-index based on the statistical value of the target preference dimension. Finally, obtain the clustering discrimination index value of the user clustering scheme based on the first clustering discrimination sub-index and / or the second clustering discrimination sub-index.

[0041] Analysis of variance (ANOVA), also known as the ANOVA F-test, is a statistical test method used to measure whether the difference in the mean of an indicator is significant between different groups. The Target Group Index (TGI) is a measure of the ratio of the preference intensity of a specific group on a certain dimension to the preference intensity of all users on the same dimension.

[0042] Specifically, when calculating the cluster discrimination index value of the user clustering scheme, an ANOVA F-test can be performed on each preset clustering index data (such as unit price, order quantity, etc.). This involves performing an analysis of variance on the index value of each clustering index data within each cluster group, thereby obtaining the ANOVA FF value for each clustering index data as the discrimination index value of the current clustering index data. The ANOVA F-value is the ratio of the between-group mean square to the within-group mean square. A larger ANOVA FF value indicates a more significant difference between groups relative to the within-group difference, meaning a greater likelihood of significant differences in the means of different clustering groups on this clustering index. In this embodiment, a larger ANOVA FF value means a higher discrimination index between different groups. This value can be used to determine whether the grouping method of the user clustering scheme is reasonable. Furthermore, by performing a logarithmic transformation on the discrimination index value of each clustering index data and calculating the arithmetic mean of the logarithmically transformed discrimination index values, the first cluster discrimination sub-index can be obtained.

[0043] In this embodiment, analysis of variance (ANOVA) based on cluster index data can quantify the degree of difference in the mean values ​​of different cluster groups on a certain cluster index. When the clustering effect is good, the mean values ​​of the indexes of each cluster group differ significantly, and the differences within each group are small. At this time, the F-value obtained from the ANOVA is high, indicating that the cluster index can effectively distinguish different groups. Based on this, by taking the average of the F-values ​​of all preset cluster index data after logarithmic transformation, a quantitative score reflecting the overall distinguishing ability of the user's clustering scheme can be obtained.

[0044] For example, taking the average transaction price (ARP) as a clustering metric, in cases of good clustering, such as cluster A having an average ARP of 180 yuan, cluster B having an average ARP of 45 yuan, and cluster C having an average ARP of 95 yuan, the differences in average ARP among the three clusters are significant, and the distribution of ARP data within each cluster is relatively concentrated. In this case, the ANOVA FF value is high, indicating that ARP effectively distinguishes different clusters and proves that the clustering scheme does indeed effectively separate users in this dimension. Conversely, if the clustering effect is poor, for example, if the average ARP of the three clusters is between 90 and 110 yuan, and the distribution of ARP data within each cluster is scattered, then the between-group mean square is small while the within-group mean square is large, resulting in a low F value. This indicates that the clustering metric does not contribute to group differentiation, and the clustering scheme has poor discrimination in this dimension. Therefore, by comprehensively evaluating the F-values ​​of each preset clustering index data, the overall discriminative ability of the clustering scheme on continuous numerical indicators can be quantified.

[0045] Furthermore, when calculating the clustering discriminancy index of the user clustering scheme, the preference dimensions of each clustered user can be obtained. These preference dimensions can include category preferences and brand preferences, etc. For each cluster, the target group index (TGI index) for each preference dimension is calculated. The TGI index is the ratio of the proportion of users in the current cluster who have a preference for that preference dimension to the proportion of all users who have a preference for that preference dimension. Then, the preference dimensions can be sorted from high to low according to the target group index, thereby selecting the dimensions whose target group index exceeds a preset threshold as target preference dimensions. Finally, the number of target preference dimensions can be counted or the average value of the target group index of the target preference dimensions can be calculated to obtain the second clustering discriminancy sub-index.

[0046] In this embodiment, the evaluation based on the target group index can verify the clustering effect of the user clustering scheme from the discrete preference dimension. When the clustering effect is good, the target group index of each cluster will show a distinctly differentiated distribution across different categories or brands. Each cluster has multiple target group indices that are significantly higher than the overall average, and the preference dimensions corresponding to the high target group indices of different clusters are different. Based on this, by statistically analyzing the number and / or average value of high target group indices for each cluster, the distinguishing effect of the user clustering scheme on the preference dimension can be reflected.

[0047] For example, considering three clusters, in a well-defined clustering scenario, cluster A shows higher target group indices for certain high-end brands and imported foods, while cluster B shows higher indices for certain affordable and promotional items. Cluster C, on the other hand, exhibits relatively similar target group indices across all categories. This demonstrates that each cluster possesses distinct and differentiated preference dimensions. Furthermore, if each cluster has a large number of target group indices exceeding a preset threshold, and the average of these indices is high, it indicates that each cluster exhibits multiple significant and strong differentiated characteristics. Conversely, if the clustering is poor, the target group indices for each group are relatively similar across all categories and brands, suggesting a lack of clear differentiated preferences and indicating that the clustering scheme failed to capture meaningful user differences. Therefore, by counting the number of target group indicators above the preset threshold in each cluster, or by calculating the average value of these target group indicators, the differentiation effect of the clustering scheme on the discrete preference dimension can be quantified. The more target group indicators there are or the higher the average value, the better the differentiation effect of the user clustering scheme on the preference dimension.

[0048] In this embodiment, the normalized values ​​of either the first or second clustering discrimination sub-index can be used individually as the clustering discrimination index value of the user clustering scheme. Alternatively, the first and second clustering discrimination sub-indexes can be normalized and then weighted and summed according to preset weights to obtain the clustering discrimination index value of the user clustering scheme. The first and second clustering discrimination sub-indexes can verify the discrimination ability of the user clustering scheme from two perspectives: continuous numerical indicators and discrete preference dimensions. The former can demonstrate the hierarchical clarity of the clustered groups on value-related indicators, while the latter can demonstrate the distinctiveness of the clustered groups on interest preferences. Combining the two allows for a comprehensive evaluation of the clustering scheme's ability to effectively distinguish users across various dimensions.

[0049] The above embodiments obtain the first clustering discrimination sub-index by performing variance analysis on the clustering index data, calculate the target group index of each group on each preference dimension, and take the statistical value of the target preference dimension that exceeds the preset threshold as the second clustering discrimination sub-index. This can capture the differentiated preference characteristics of each cluster group on the sub-category or clustering index, thereby enabling a comprehensive quantitative evaluation of the user clustering scheme's ability to distinguish between continuous clustering indexes and discrete preferences.

[0050] In one embodiment, in step 102, the clustering discriminancy index of the user clustering scheme can also be calculated by the following method: First, a training dataset is constructed based on the clustering labels of each clustered user in the user clustering scheme and the original feature data of each clustered user. The training dataset is then split into a training set and a test set. A multi-classification model is trained using the data in the training set. The multi-classification model is used to predict the clustering labels of users based on their original feature data. Next, the trained multi-classification model is evaluated using the data in the test set, and the class balance index and / or model ranking discriminancy index of the multi-classification model on the test set are calculated. Finally, the clustering discriminancy index of the user clustering scheme is obtained based on the class balance index and / or model ranking discriminancy index of the multi-classification model on the test set.

[0051] Specifically, the category balance metric can be the macro-average F1 score, which is the harmonic mean of precision and recall, and can be used to measure the comprehensive recognition ability of a multi-class model across all categories. The model ranking discrimination metric can be the AUC value (Area Under the Receiver Operating Characteristic Curve), which can be used to measure the ability of a multi-class model to distinguish samples from different categories.

[0052] Specifically, when calculating the cluster discrimination index of a user clustering scheme, a multi-class classification model can be trained, and its classification ability can be evaluated to calculate the cluster discrimination index. Specifically, when calculating the cluster discrimination index, the original feature data and corresponding cluster labels of each user in the user clustering scheme can be obtained. Using the original feature data as input features and the cluster labels as prediction targets, a training dataset is constructed. Then, the training dataset is randomly split into a training set and a test set according to a preset ratio. The training set is used for model learning, and the test set is used to evaluate model performance. Next, a multi-class classification model is trained using the data in the training set. This model learns the mapping relationship between the original feature data and the cluster labels, gaining the ability to predict the cluster group to which a user belongs based on their original feature data. Then, the trained multi-class classification model is evaluated using the data in the test set, and the class balance index and / or model ranking discrimination index of the multi-class classification model on the test set are calculated.

[0053] In this embodiment, when calculating the class balance index, for each cluster, the precision and recall of that cluster are first calculated, and then the F1 score of that cluster is calculated based on the precision and recall. Next, the arithmetic mean of the F1 scores of all clusters is taken to obtain the macro-average F1 score as the class balance index. When calculating the model ranking discrimination index, a one-to-many strategy can be used to calculate the area under the receiver operating characteristic (AUC) curve. That is, each cluster is treated as a positive example, and all other clusters are treated as negative examples. The AUC curves for each cluster are calculated, and the average is taken as the model ranking discrimination index. In this embodiment, either the class balance index or the model ranking discrimination index can be used alone as the cluster discrimination index value of the user clustering scheme, or a weighted sum of the class balance index and the model ranking discrimination index can be used as the cluster discrimination index value of the user clustering scheme.

[0054] In this embodiment, the macro-average F1 score ranges from 0 to 1. A higher value indicates that the multi-classification model has better recognition capabilities across all categories. When the macro-average F1 score is close to 1, it means that the multi-classification model can accurately identify each cluster, and the clusters have clear boundaries and are compact within the feature space. When the macro-average F1 score is close to 0, it means that the multi-classification model cannot distinguish different groups based on feature data, and the clusters overlap in the feature space. In cluster evaluation scenarios, a higher macro-average F1 score indicates that the clusters created by the clustering scheme have high discriminability in terms of feature data. The AUC value measures the probability that the multi-classification model predicts a higher score for a randomly selected positive sample than for a randomly selected negative sample. When the AUC value is close to 1, it means that the model can effectively distinguish between positive and negative examples, and there is almost no feature overlap between the clusters. When the AUC value is 0.5, it means that the model's discriminative ability is equivalent to random guessing, and the clusters completely overlap in the feature space. In cluster evaluation scenarios, a high AUC value indicates that the different groups divided by the clustering scheme have good separation in feature data, and users between any two groups can be effectively distinguished.

[0055] The above embodiments train a multi-class classification model using the user clustering results of the user clustering scheme, and calculate the class balance index and / or model ranking discrimination index of the multi-class classification model on the test set as the cluster discrimination index value. This can effectively measure the comprehensive identifiability of each cluster group in the feature space. Furthermore, by indirectly quantifying the group separability of the clustering scheme in the feature space using the prediction accuracy of the multi-class classification model, a reliable alternative path can be provided for evaluating cluster discrimination.

[0056] In one embodiment, step 103 can be implemented by the following method: For each cluster group in the user clustering scheme, the original feature data and clustering index data of each cluster user in the cluster group are aggregated to obtain a data summary of the cluster group. Then, the data summary and feature metadata knowledge base of the cluster group are input into the analysis agent to generate the group name, core feature information and feature analysis conclusion of the cluster group. Finally, the analysis agent is used to integrate the group name, core feature information and feature analysis conclusion of each cluster group according to a preset format to obtain a text profile of each cluster group.

[0057] The data summary is a structured information set obtained by statistically summarizing the original feature data and clustering index data of all users in the cluster.

[0058] Specifically, when generating text profiles for each cluster, each cluster in the user clustering scheme can be traversed, and text profiles can be generated one by one. Specifically, for the current cluster, the original feature data of all users in the cluster is first aggregated, such as calculating the mean, median, and distribution of each original feature, and the clustering index data of all users in the cluster is also aggregated, such as calculating the mean of each clustering index. Then, the above statistical summary results are combined into a data summary for the cluster. Next, the data summary and feature metadata knowledge base are input into the analysis agent. The feature metadata knowledge base contains feature description information such as the meaning, data type, and value range of each original feature. The analysis agent can perform semantic understanding and summarization of the features of the cluster based on the statistical distribution of each feature in the data summary and the feature description information in the feature metadata knowledge base, thereby generating information such as the cluster name, core feature information, and feature analysis conclusions for the cluster. In this process, the group name summarizes the core characteristics of the group, the core characteristic information describes the typical value range of the group on key features, and the feature analysis conclusions explain the logical relationships between the features of the group. Finally, the analysis agent can integrate the group name, core characteristic information, and feature analysis conclusions according to a preset format, such as JavaScript object notation, to generate a textual profile of the current cluster group.

[0059] For example, when generating a text profile for group three in a user clustering scheme, the server first aggregates the original feature data and clustering index data of all users in this group, obtaining a data summary including that the group's average order value is 120 yuan, the average number of orders is eight per month, and the main consumption categories are food and beverages and daily necessities. Then, the data summary and feature metadata knowledge base are input into the analysis agent. The analysis agent, combining the meanings of each feature in the feature metadata knowledge base, names this group the "Quality Lifestyle Group" and generates core feature information as high average order value, medium-to-high purchase frequency, and preferred branded goods. The feature analysis conclusion is that this group has high requirements for product quality and stable consumption behavior. Finally, the analysis agent integrates the above information into a text profile in JavaScript object representation format and outputs it.

[0060] In this embodiment, by generating textual profiles of each cluster in the user clustering scheme, it is possible to verify whether the user clustering scheme can be clearly described and explained at the semantic level. Specifically, each cluster group divided by a high-quality user clustering scheme should have distinct and inductive characteristics, be given a reasonable name, and be able to describe its core features and typical behaviors in natural language. If a clustering scheme cannot generate clear textual profiles, or if the group characteristics described in the textual profiles are vague or overlapping, it indicates that the user clustering scheme may lack practical value in real-world applications. Therefore, textual profiles can be used to verify the quality of the user clustering scheme in terms of interpretability.

[0061] The above embodiments generate data summaries by aggregating the original feature data and clustering index data of each cluster group. The data summaries and feature metadata knowledge base are then input into the analysis agent. The analysis agent then performs semantic understanding and summarization of the features of the cluster groups to generate textual profiles containing group names, core feature information, and feature analysis conclusions. This can transform the multidimensional numerical features of each cluster group that are difficult to understand into highly readable and structurally consistent natural language descriptions, thereby making it easier for users to understand the meaning of the cluster groups.

[0062] In one embodiment, step 104 can be implemented by the following method: For each cluster, the text profile, data summary, and prior knowledge base of evaluation rules are input into the evaluation agent. The evaluation agent verifies the text profile based on the evaluation rule information in the prior knowledge base of evaluation rules. Then, based on the verification results and preset scoring rules, the text profile is scored in multiple dimensions to generate multi-dimensional scoring results for the cluster. The evaluation dimensions include at least one of the following: the matching degree between the cluster name and features, the rigor of feature description and feature analysis, the depth of insight, and the standardization of language structure. Finally, based on the multi-dimensional scoring results of each cluster, the interpretability score of each cluster is obtained. The interpretability scores of each cluster are aggregated to obtain the interpretability index value of the user clustering scheme.

[0063] The evaluation rule prior knowledge base stores evaluation rule information, which describes the constraints that clustered groups should meet in terms of feature data, such as "the average order value of high-value groups should not be too low." The scoring rules refer to a set of structured and detailed scoring criteria used to guide the evaluation process.

[0064] Specifically, when evaluating the text profiles of each cluster, each cluster in the user's clustering scheme can be traversed, and the text profile of each cluster can be evaluated from multiple dimensions. Specifically, for the current cluster, the text profile of that cluster, its data summary, and the evaluation rule information from the prior knowledge base of the evaluation rules are first combined into evaluation prompts, which are then input into the evaluation agent. The evaluation agent can perform multi-dimensional verification of the text profile based on the constraints in the evaluation rule information. For example, when the evaluation rule information includes the constraint "the average order value of high-value groups should not be too low," the evaluation agent, when evaluating clusters related to the group name of "high-value groups," can extract various information from the text profile of that cluster, obtain the average order value of that cluster from the data summary, and use this information to determine whether the description of the average order value in the core feature information of the cluster matches the average order value in the data summary, and whether the group name and core feature information match, generating corresponding verification results.

[0065] Furthermore, the evaluation agent can score the text profile in multiple dimensions based on the verification results across these dimensions, according to preset scoring rules. These scoring rules can specifically include scoring standards and ranges for various dimensions, such as the matching degree between group naming and features, the rigor of feature description and analysis, the depth of insight, and the standardization of language structure. Finally, the evaluation agent will output a multi-dimensional scoring result containing scores for each dimension and corresponding evaluation reasons. Further, after obtaining the multi-dimensional scores for each cluster, the scores for each dimension in the multi-dimensional scoring results for each cluster can be weighted and summed to obtain the interpretability score for each cluster. Finally, the arithmetic mean of the interpretability scores for all clusters is taken to obtain the interpretability index value of the user clustering scheme.

[0066] In this embodiment, the quality of the text profile reflects the interpretability level of the user clustering scheme. Interpretability is an independent dimension for evaluating the quality of a user clustering scheme, used to verify whether the clustering results are easy to understand and use, thus avoiding incomprehensible problems in subsequent use. A user clustering scheme with high interpretability should be able to generate accurately named, logically consistent, and insightful text profiles based on data summaries; conversely, if the generated text profiles are full of contradictions or too general, it indicates that the user clustering scheme may be too complex or lack practical significance. Based on this, this embodiment transforms the quality of the profile from a subjective judgment into a quantifiable score by scoring the text profiles from multiple dimensions. A high interpretability score indicates that the characteristics of each cluster group in the user clustering scheme are clear, explicit, and easy to understand; conversely, a low interpretability score indicates that the characteristics of each cluster group in the user clustering scheme are vague and difficult to understand.

[0067] The above embodiments input the text profile, data summary, and evaluation rule information from the prior knowledge base of evaluation rules into the evaluation agent. The evaluation agent then verifies and scores the text profile from multiple dimensions. This allows for an accurate determination of whether the content of the text profile and the statistical results in the data summary meet the preset constraints. This ensures that the multi-dimensional scoring fully reflects the comprehensive quality of the text profile in terms of accuracy of group naming, consistency of feature description, logical rigor, depth of insight, and standardization of language. It also provides a quantifiable basis for the understandability of results for different user clustering schemes.

[0068] In one embodiment, step 104 may further include the following method: For each cluster, a debate agent pre-trained based on a large model is invoked to generate questioning information based on the text profile of the cluster. The questioning information includes at least one of the following: logical contradictions between cluster naming and features; conflicts between feature analysis conclusions and evaluation rule information; and evidence of insufficient insight depth. Then, the text profile of the cluster and the questioning information are input into an evaluation agent, which responds to the questioning information at least once and generates debate information. After the debate agent and evaluation agent reach a preset number of debate rounds or meet a preset termination condition, a referee agent pre-trained based on a large model is invoked to update the multi-dimensional scoring results of the cluster based on the debate information, obtaining the updated multi-dimensional scoring results of the cluster.

[0069] Specifically, after the evaluation agent completes the initial scoring of the text profiles of each cluster, the debate agent can be invoked to challenge these text profiles. Specifically, for the current cluster, the text profile and data summary of the current cluster can be input into the debate agent. The debate agent can review the content of the text profile based on the statistical results in the data summary and generate challenge information. This challenge information can include various dimensions such as logical contradictions between the cluster naming and the feature statistics in the data summary, conflicts between the feature analysis conclusions and the evaluation rule information in the prior knowledge base of the evaluation rules, and evidence of insufficient insight depth. Then, the challenge information and text profile can be input into the evaluation agent, which responds to each challenge in the challenge information and generates a response. This process of invoking the debate agent to generate challenge information and the evaluation agent to generate response content is repeated until the preset debate round limit is reached or the debate agent cannot generate new valid challenges, at which point the debate process terminates. Finally, after the debate ends, the referee agent can be invoked, and the complete debate information can be input into the referee agent. The referee agent can then update the multi-dimensional score results of the current cluster group based on the quality of arguments and logical rigor of each party in the debate information, and use the updated multi-dimensional score results as the final interpretability score of the cluster group.

[0070] The above embodiments raise logical contradictions and insufficient evidence in the text profile by calling the debate agent, and allow the evaluation agent to fully demonstrate the scoring basis through multiple rounds of debate. Finally, the judge agent updates the scoring results based on the quality of the debate, which can effectively improve the rigor and credibility of interpretability assessment.

[0071] In one embodiment, step 105 can be implemented by the following method: weighting the statistical validity index, cluster discrimination index, and interpretability index of the user clustering scheme to obtain a comprehensive evaluation value of the user clustering scheme; then, generating an evaluation report of the user clustering scheme based on the comprehensive evaluation value, statistical validity index, cluster discrimination index, interpretability index, textual profiles of each cluster group in the user clustering scheme, and multi-dimensional scoring results.

[0072] Specifically, after calculating the statistical validity index, cluster discrimination index, and interpretability index of the user clustering scheme to be evaluated, these three indexes can be weighted according to preset weight parameters to obtain the comprehensive evaluation value of the user clustering scheme. Based on this, textual profiles of each cluster group in the user clustering scheme and the corresponding multi-dimensional scoring results can be obtained. Finally, the comprehensive evaluation value, statistical validity index, cluster discrimination index, interpretability index, textual profiles of each cluster group, and multi-dimensional scoring results of each cluster group are integrated, and an evaluation report is generated according to a preset evaluation report template. The evaluation report can include a scheme overview section, sub-scores, a comprehensive evaluation value section, and group details, and the evaluation report is output.

[0073] The above embodiments obtain a comprehensive evaluation value by weighting and summing the statistical validity index value, cluster discrimination index value, and interpretability index value, and condense the evaluation results of multiple dimensions into a single quantitative score. This provides an intuitive quantitative basis for comparing the merits of different clustering schemes. In addition, by integrating the comprehensive evaluation value, the scores of each sub-item, the textual profiles of each group, and the multi-dimensional scoring results into a structured evaluation report, a comprehensive and quantifiable reference basis can be provided for the selection decision of clustering schemes.

[0074] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. In addition, the labels corresponding to each step in the above embodiments are only for identification purposes and are not intended to limit the execution order of the steps. The execution order of the steps in each embodiment can be set according to the actual situation.

[0075] Furthermore, as Figure 1 In terms of specific implementation, this application provides a user clustering evaluation system based on a large model, such as... Figure 2 As shown, the system includes a data input layer, a multidimensional evaluation layer, and a comprehensive decision-making layer. The multidimensional evaluation layer includes a statistical validity engine, a clustering discriminative engine, and an interpretability engine, wherein: The data input layer is used to obtain the user clustering scheme to be evaluated, as well as the original feature data and clustering index data of each clustered user. The user clustering scheme includes user identifiers and clustering labels of multiple clustered users. The statistical validity engine is used to calculate the statistical validity index value of the user clustering scheme based on the user clustering scheme and the original feature data of each clustered user. The clustering discriminant engine is used to calculate the clustering discriminant index value of the user clustering scheme based on the user clustering scheme and the original feature data and clustering index data of each clustered user. The interpretability engine is used to call an analytical agent based on a large model pre-training to aggregate the original feature data and clustering index data of each cluster in the user clustering scheme to obtain a data summary, and to generate a text profile of each cluster based on the data summary and feature description information in a preset feature metadata knowledge base. The interpretability engine is also used to call an evaluation agent based on a large model pre-training to perform multi-dimensional scoring on the text profiles of each cluster group according to the evaluation rule information in the preset evaluation rule prior knowledge base, and to obtain the interpretability index value of the user clustering scheme based on the multi-dimensional scoring results of each cluster group. The comprehensive decision-making layer is used to obtain the evaluation results of the user clustering scheme based on the statistical validity index, cluster discrimination index, and interpretability index of the user clustering scheme.

[0076] In a specific application scenario, the statistical validity engine is specifically used to extract multiple sample users from the clustered users of the user clustering scheme; for each sample user, it calculates a first average distance between the original feature data of the sample user and the original feature data of other users in the same cluster, and a second average distance between the original feature data of the sample user and the original feature data of multiple users in other clusters, and obtains the silhouette coefficient of the sample user based on the first average distance and the second average distance; obtains the silhouette coefficient of the user clustering scheme based on the silhouette coefficient of each sample user; calculates the standard deviation and mean of the number of users in each cluster, and obtains the coefficient of variation of the user clustering scheme based on the standard deviation and mean of the number of users in each cluster; and obtains the statistical validity index value of the user clustering scheme based on the silhouette coefficient and / or the coefficient of variation of the user clustering scheme.

[0077] In specific application scenarios, the clustering discriminancy engine is specifically used to perform variance analysis on the index values ​​of the clustering index data for each preset clustering index data group to obtain the discriminancy index value of the clustering index data; to perform numerical transformation on the discriminancy index value of each clustering index data to obtain a first clustering discriminancy sub-index; to obtain multiple preference dimensions of each clustered user; to calculate the target group index of each clustered group on multiple preference dimensions respectively; and to filter out target preference dimensions whose target group index exceeds a preset threshold based on the target group index of each clustered group on multiple preference dimensions; to obtain a second clustering discriminancy sub-index based on the statistical value of the target preference dimension; and to obtain the clustering discriminancy index value of the user clustering scheme based on the first clustering discriminancy sub-index and / or the second clustering discriminancy sub-index.

[0078] In specific application scenarios, the clustering discriminability engine is further used to construct a training dataset based on the clustering labels of each clustered user in the user clustering scheme and the original feature data of each clustered user, and to split the training dataset into a training set and a test set; to train a multi-classification model using the data in the training set, wherein the multi-classification model is used to predict the clustering labels of the users based on the users' original feature data; to evaluate the trained multi-classification model using the data in the test set, and to calculate the class balance index and / or model ranking discriminability index of the multi-classification model on the test set; and to obtain the clustering discriminability index value of the user clustering scheme based on the class balance index and / or model ranking discriminability index of the multi-classification model on the test set.

[0079] In specific application scenarios, the interpretability engine is specifically used to aggregate the original feature data and clustering index data of each clustered user in the user clustering scheme to obtain a data summary of the clustered group; input the data summary of the clustered group and the feature metadata knowledge base into the analysis agent to generate the group name, core feature information and feature analysis conclusion of the clustered group; and use the analysis agent to integrate the group name, core feature information and feature analysis conclusion of the clustered group according to a preset format to obtain a text profile of the clustered group.

[0080] In specific application scenarios, the interpretability engine is further used to input the text profile, data summary, and preset evaluation rule prior knowledge base of each cluster into the evaluation agent. The evaluation agent then verifies the text profile based on the evaluation rule information in the prior knowledge base. Based on the verification results and preset scoring rules, the text profile is scored in multiple dimensions to generate a multi-dimensional scoring result for the cluster. The evaluation dimensions include at least one of the following: the matching degree between cluster naming and features; the rigor of feature description and analysis; the depth of insight; and the standardization of language structure. Based on the multi-dimensional scoring results of each cluster, an interpretability score is obtained for each cluster. The interpretability scores of each cluster are then aggregated to obtain the interpretability index value of the user clustering scheme.

[0081] In specific application scenarios, the interpretability engine is further used to, for each cluster, invoke a debate agent pre-trained on a large model to generate questioning information based on the text profile of the cluster. The questioning information includes at least one of the following: logical contradictions between the cluster name and features; conflicts between feature analysis conclusions and evaluation rule information; and evidence of insufficient insight depth. The text profile and questioning information of the cluster are input into the evaluation agent, which responds to the questioning information in at least one round and generates debate information. After the debate agent and the evaluation agent reach a preset number of debate rounds or meet a preset termination condition, a referee agent pre-trained on a large model is invoked to update the multi-dimensional scoring results of the cluster based on the debate information, thereby obtaining the updated multi-dimensional scoring results of the cluster.

[0082] In specific application scenarios, the comprehensive decision-making layer is specifically used to perform weighted calculations on the statistical validity index, cluster discrimination index, and interpretability index of the user clustering scheme to obtain a comprehensive evaluation value of the user clustering scheme; based on the comprehensive evaluation value, statistical validity index, cluster discrimination index, interpretability index, and textual profiles and multi-dimensional scoring results of each cluster group in the user clustering scheme, an evaluation report of the user clustering scheme is generated.

[0083] It should be noted that other corresponding descriptions of the functional units involved in the user clustering evaluation system based on a large model provided in this application embodiment can be found in the following references. Figure 1 The corresponding descriptions of the methods in the above embodiments will not be repeated here.

[0084] This application also provides a computer device, specifically a personal computer, server, network device, etc. The computer device includes a bus, processor, memory, and communication interface, and may also include input / output interfaces and a display device. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device stores location information. The network interface of the computer device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the various method embodiments.

[0085] Those skilled in the art will understand that the structure of the computer device described above is only a partial structure related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. A specific computer device may include more or fewer components, or combine certain components, or have different component arrangements.

[0086] In one embodiment, a computer-readable storage medium is provided, which may be non-volatile or volatile, having stored thereon a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0087] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0088] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, graphics processors, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0089] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0090] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A user clustering evaluation method based on a large model, characterized in that, The method includes: Obtain the user clustering scheme to be evaluated, as well as the original feature data and clustering index data of each clustered user. The user clustering scheme includes user identifiers and clustering labels of multiple clustered users. Based on the user clustering scheme and the original feature data and clustering index data of each clustered user, the statistical validity index value and clustering discriminancy index value of the user clustering scheme are calculated. The analysis agent, which is pre-trained based on a large model, is invoked to aggregate the original feature data and clustering index data of each cluster in the user clustering scheme to obtain a data summary. Based on the data summary and the feature description information in the preset feature metadata knowledge base, a text profile of each cluster is generated. The evaluation agent, which is pre-trained based on a large model, is invoked to perform multi-dimensional scoring on the text profiles of each cluster group according to the evaluation rule information in the pre-set evaluation rule prior knowledge base. Based on the multi-dimensional scoring results of each cluster group, the interpretability index value of the user clustering scheme is obtained. The evaluation results of the user clustering scheme are obtained based on the statistical validity index, cluster discrimination index, and interpretability index of the scheme.

2. The method according to claim 1, characterized in that, The method for calculating the statistical validity index value of the user clustering scheme includes: Multiple sample users are extracted from the clustered users in the user clustering scheme; For each sample user, calculate the first average distance between the original feature data of the sample user and the original feature data of other users in the same cluster, and the second average distance between the original feature data of the sample user and the original feature data of multiple users in other clusters, and obtain the profile coefficient of the sample user based on the first average distance and the second average distance. Based on the profile coefficients of each of the sample users, the profile coefficients of the user clustering scheme are obtained; Calculate the standard deviation and mean of the number of users in each cluster, and based on the standard deviation and mean of the number of users in each cluster, obtain the coefficient of variation of the user clustering scheme. The statistical validity index value of the user clustering scheme is obtained based on the silhouette coefficient and / or the scale variation coefficient of the user clustering scheme.

3. The method according to claim 1, characterized in that, The method for calculating the clustering discriminant index value of the user clustering scheme includes: For each preset clustering index data, variance analysis is performed on the index values ​​of the clustering index data for each cluster group to obtain the discrimination index value of the clustering index data. The discrimination index value of each clustering index data is then numerically transformed to obtain the first clustering discrimination sub-index. Multiple preference dimensions of users in each cluster are obtained, and the target group index of each cluster group on multiple preference dimensions is calculated respectively. Based on the target group index of each cluster group on multiple preference dimensions, the target preference dimensions with target group index exceeding a preset threshold are selected. Based on the statistical value of the target preference dimension, the second cluster discrimination sub-index is obtained. The clustering discrimination index value of the user clustering scheme is obtained based on the first clustering discrimination sub-index and / or the second clustering discrimination sub-index.

4. The method according to claim 1 or 3, characterized in that, The method for calculating the clustering discriminant index value of the user clustering scheme further includes: A training dataset is constructed based on the clustering labels of each clustered user in the user clustering scheme and the original feature data of each clustered user, and the training dataset is split into a training set and a test set; A multi-classification model is trained using the data in the training set, wherein the multi-classification model is used to predict the clustering label of the user based on the user's original feature data; The trained multi-class classification model is evaluated using data from the test set, and the class balance index and / or model ranking discrimination index of the multi-class classification model on the test set are calculated. The clustering discrimination index value of the user clustering scheme is obtained based on the class balance index and / or model ranking discrimination index of the multi-classification model on the test set.

5. The method according to claim 1, characterized in that, The process involves invoking an analytical agent pre-trained on a large model to aggregate the original feature data and clustering index data of each cluster in the user clustering scheme to obtain a data summary. Based on the data summary and feature description information in a pre-set feature metadata knowledge base, a textual profile of each cluster is generated, including: For each cluster in the user clustering scheme, the original feature data and clustering index data of each clustered user in the cluster are aggregated to obtain a data summary of the cluster. The data summary of the cluster and the feature metadata knowledge base are input into the analysis agent to generate the cluster name, core feature information and feature analysis conclusions of the cluster; The analytical agent integrates the group name, core feature information, and feature analysis conclusions of the clustered group according to a preset format to obtain a textual profile of the clustered group.

6. The method according to claim 1, characterized in that, The process involves calling an evaluation agent pre-trained on a large model to perform multi-dimensional scoring on the text profiles of each cluster group based on evaluation rule information in a pre-defined evaluation rule knowledge base. Based on the multi-dimensional scoring results of each cluster group, the interpretability index value of the user clustering scheme is obtained, including: For each cluster, the text profile, data summary, and preset evaluation rule prior knowledge base of the cluster are input into the evaluation agent, and the evaluation agent examines the text profile according to the evaluation rule information in the evaluation rule prior knowledge base. The text profile is scored in multiple dimensions based on the verification results and preset scoring rules to generate multi-dimensional scoring results for the clustered group. The evaluation dimensions include at least one of the following: the matching degree between the group name and the feature, the rigor of the feature description and feature analysis, the depth of insight, and the standardization of the language structure. Based on the multi-dimensional scoring results of each cluster group, the interpretability score of each cluster group is obtained. The interpretability scores of each cluster group are aggregated to obtain the interpretability index value of the user clustering scheme.

7. The method according to any one of claims 1, 5, and 6, characterized in that, The method further includes: For each cluster, a debate agent pre-trained on a large model is invoked to generate questioning information based on the text profile of the cluster. The questioning information includes at least one of the following: logical contradiction between the group name and features, conflict between feature analysis conclusions and evaluation rule information, and evidence of insufficient insight depth. The textual profiles and questioning information of the clustered groups are input into the evaluation agent, which then responds to the questioning information in at least one round and generates debate information. After the debate agent and the evaluation agent reach the preset number of debate rounds or meet the preset termination conditions, the referee agent, which is pre-trained based on a large model, is invoked to update the multi-dimensional scoring results of the cluster group according to the debate information, so as to obtain the updated multi-dimensional scoring results of the cluster group.

8. The method according to claim 1, characterized in that, The evaluation results of the user clustering scheme are obtained based on the statistical validity index, cluster discrimination index, and interpretability index values, including: The statistical validity index, cluster discrimination index, and interpretability index of the user clustering scheme are weighted and calculated to obtain the comprehensive evaluation value of the user clustering scheme. An evaluation report of the user clustering scheme is generated based on the comprehensive evaluation value, statistical validity index value, cluster discrimination index value, interpretability index value, and textual profiles and multi-dimensional scoring results of each cluster group in the user clustering scheme.

9. A user clustering evaluation system based on a large model, characterized in that, The system comprises a data input layer, a multidimensional evaluation layer, and a comprehensive decision-making layer. The multidimensional evaluation layer includes a statistical validity engine, a clustering discriminative engine, and an interpretability engine, wherein: The data input layer is used to obtain the user clustering scheme to be evaluated, as well as the original feature data and clustering index data of each clustered user. The user clustering scheme includes user identifiers and clustering labels of multiple clustered users. The statistical validity engine is used to calculate the statistical validity index value of the user clustering scheme based on the user clustering scheme and the original feature data of each clustered user. The clustering discriminant engine is used to calculate the clustering discriminant index value of the user clustering scheme based on the user clustering scheme and the original feature data and clustering index data of each clustered user. The interpretability engine is used to call an analytical agent based on a large model pre-training to aggregate the original feature data and clustering index data of each cluster in the user clustering scheme to obtain a data summary, and to generate a text profile of each cluster based on the data summary and feature description information in a preset feature metadata knowledge base. The interpretability engine is also used to call an evaluation agent based on a large model pre-training to perform multi-dimensional scoring on the text profiles of each cluster group according to the evaluation rule information in the preset evaluation rule prior knowledge base, and to obtain the interpretability index value of the user clustering scheme based on the multi-dimensional scoring results of each cluster group. The comprehensive decision-making layer is used to obtain the evaluation results of the user clustering scheme based on the statistical validity index, cluster discrimination index, and interpretability index of the user clustering scheme.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 8.

11. A computer device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 8.

Images