Time sequence sensing multi-view clustering method and system

Abstract

The invention discloses an order-aware multi-view clustering method and system. The method comprises the following steps: acquiring a data set of a certain electronic competition team or a certain player; an original model used for clustering analysis is constructed, the original model comprises a plurality of auto-encoder structures, and the views are input into different auto-encoder structures of the original model; the auto-encoder structure carries out dimension reduction processing and data reconstruction on data X contained in the view; processing the data of the views by adopting a KNN algorithm, calculating to obtain a similarity graph, and calculating to obtain a local geometric information regular term and a joint probability of any two views based on the similarity graph; and calculating the cosine similarity of the vectors of any two views after dimension reduction processing, and defining a hyper-parameter epsilon to represent a similarity threshold. According to the method, feature extraction and dimension reduction are carried out on each view, then the weight is introduced to measure the correlation between different views, and the correlation is applied to the clustering process, so that the accuracy and stability of clustering are improved.

Description

Technical Field

[0001] The present invention relates to a clustering method and system, and in particular to a time-series-aware multi-view clustering method and system. Background Art

[0002] In all kinds of e-sports competitions, the analysis of the performance of each team and player in the arena is a crucial part. It can dig out the potential game patterns and tactics of the players, and play a driving role in pre-match predictions and post-match reviews. At present, the analysis of the game is mostly done by experts. The experts have strong personal subjective factors and it is difficult to analyze the hidden patterns of the teams and players. With the development of e-sports competitions, some experts have proposed using clustering algorithms for analysis, but traditional clustering methods have weak modeling capabilities for nonlinear relationships, making it difficult to capture the complex potential patterns and relationships in e-sports data, and the reliability of clustering results is low. Summary of the invention

[0003] Purpose of the invention: The first purpose of the present invention is to provide a time-aware multi-view clustering method that can accurately mine potential patterns and relationships and has highly reliable clustering results.

[0004] A second object of the present invention is to provide a time-series-aware multi-view clustering system.

[0005] Technical solution: The present invention discloses a time-series-aware multi-view clustering method, comprising the following steps:

[0006] S1: Obtain a data set of an e-sports team or a player, wherein the data set includes multiple game characters used by the e-sports team or the player, wherein each game character includes multimodal data of multiple game matches, and the multimodal data of different matches of each game character are divided into different views according to image data, point sequence data and video data;

[0007] S2: construct an original model for cluster analysis, the original model includes multiple autoencoder structures, and input each view into a different autoencoder structure of the original model;

[0008] S3: The autoencoder structure performs dimensionality reduction processing on the data X contained in the view, extracts the feature representation H after dimensionality reduction, clusters the low-dimensional features to obtain the initial clustering center, and reconstructs the feature representation H to obtain the reconstructed data Calculate the reconstruction loss;

[0009] S4: Use the KNN algorithm to process the view data and calculate a similarity graph. Based on the similarity graph, the local geometric information regularization term and the joint probability of any two views are calculated.

[0010] S5: Calculate the cosine similarity of the vectors of any two views after dimensionality reduction, define a hyperparameter ε to represent the similarity threshold, calculate the similarity parameter of any two views based on the cosine similarity and the hyperparameter ε, introduce the time series information on the basis of the similarity parameter to calculate the consistency parameter of the corresponding two views, and calculate the consistency and complementarity regularization terms based on the joint probability and the consistency parameter;

[0011] S6: calculating the original view importance of each view according to the similarity parameter, and calculating the original view importance weight based on the original view importance;

[0012] S7: The low-dimensional features extracted from the view are integrated with the original view importance weight of the view, and then the prediction distribution matrix Q of the original model and the target distribution matrix P of the original model are calculated;

[0013] S8: Calculate the clustering loss of the autoencoder structure based on the target distribution matrix P and the prediction distribution matrix Q;

[0014] S9: Back-propagate according to the reconstruction loss and clustering loss, and iteratively update the parameters of the autoencoder structure under the correction of the local geometric information regularization term and the consistency and complementarity regularization term, and set the iterative update stop condition. The original model obtained after the iterative update stops is the final clustering model;

[0015] S10: Input the view of the data set to be processed into the clustering model, and the clustering model outputs the clustering result.

[0016] Furthermore, the steps for calculating the reconstruction loss in step S2 are as follows:

[0017] Calculate the penalty weight corresponding to each view. The calculation formula is as follows:

[0018]

[0019] in represents the jth element in the vector corresponding to the sth view of the ith game character;

[0020] The reconstruction loss of the corresponding view is calculated based on the penalty weight. The calculation formula is as follows:

[0021]

[0022] Where n represents the number of views, m represents the number of game characters, represents the reconstructed data of the s-th view of the ith game character, and a⊙b represents the Hadamard product of matrices a and b.

[0023] Furthermore, the steps of calculating the local geometric information regularization term and the joint probability of any two views based on the similarity graph in step S4 are as follows:

[0024] Based on the similarity graph, the local geometric information regularization term is calculated, and the calculation formula is as follows:

[0025]

[0026]

[0027] in The s-th view of the i-th game character corresponds to the low-dimensional feature representation output by the encoder, represents the joint probability between two different low-dimensional feature representations of the s-th view, represents the weights of the i-th sample and the j-th sample in the s-th view in the similarity graph;

[0028] Calculate the joint probability of each view, the calculation formula is as follows:

[0029]

[0030] Furthermore, the formula for calculating the cosine similarity of the vectors of any two views after the dimensionality reduction process in step S5 is:

[0031]

[0032] in represents the dimension-reduced Feature representation, represents the Sth 1 views, and ||·|| represents the L2 norm.

[0033] Furthermore, the step of calculating the consistency parameter in step S5 is as follows: the similarity parameter of the corresponding two views is calculated according to the cosine similarity, and the calculation formula is as follows:

[0034]

[0035] Two preliminary views with timing information and The consistency parameter calculation formula is:

[0036]

[0037] Two preliminary views without timing information and The consistency parameter calculation formula is:

[0038]

[0039] A preliminary view without timing information and a preliminary view with timing information The consistency parameter calculation formula is:

[0040]

[0041] The consistency parameter Representing low-dimensional features of two views and Does it represent the same sample? If so, If not then

[0042] Furthermore, the method of calculating the importance weight of the original view in step S6 is as follows:

[0043] The consistency and complementarity regularization terms are calculated based on the consistency parameter and joint probability. The calculation formula is as follows:

[0044]

[0045] The original view importance of each view is calculated based on the similarity parameter. The calculation formula is as follows:

[0046]

[0047] Among them, view-s represents the original view importance of the s-th view;

[0048] The original view importance weight is calculated based on the original view importance. The calculation formula is as follows:

[0049]

[0050] Where W view-s represents the original view importance weight of the s-th view, min(O view ) represents the minimum value of the original view importance weight.

[0051] Further, step S7 calculates the prediction distribution matrix Q of the original model and the target distribution matrix P of the original model as follows:

[0052] Calculate the prediction distribution matrix Q of the original model, the calculation formula is as follows:

[0053]

[0054] where q ij It is used to represent the probability that the i-th element obtained by clustering prediction belongs to the j-th cluster, and λ represents the free parameter of t distribution;

[0055] Calculate the target distribution matrix P of the original model. The calculation formula is as follows:

[0056]

[0057] Furthermore, the calculation formula of the clustering loss in step S8 is as follows:

[0058]

[0059] The KL divergence is used to measure the similarity between distributions P and Q.

[0060] Based on the same inventive concept, the present invention also provides a time-series-aware multi-view clustering system, comprising:

[0061] A data acquisition module is used to acquire a data set of an e-sports team or a player, where the data set includes multiple views divided into image data, point sequence data, and video data;

[0062] The original clustering module is used to construct the original model of clustering analysis. The original model includes multiple autoencoder structures. Each view of the data acquisition module inputs a different autoencoder structure of the original model.

[0063] Dimensionality reduction extraction and reconstruction module: The autoencoder structure of the original clustering module performs dimensionality reduction processing on the data X contained in the view, extracts the feature representation H after dimensionality reduction, and clusters the low-dimensional features to obtain the initial clustering center. The autoencoder structure reconstructs the feature representation H to obtain the reconstructed data And calculate the reconstruction loss;

[0064] A data processing module is used to calculate the local geometric information regularization term, consistency and complementarity regularization term and the importance weight of the original view of each view;

[0065] Fusion module: The low-dimensional features extracted from the view are fused with the original view importance weight of the view, and then the prediction distribution matrix Q of the original model and the target distribution matrix P of the original model are calculated. The clustering loss of the autoencoder structure is calculated based on the target distribution matrix P and the prediction distribution matrix Q.

[0066] The iterative update module back-propagates according to the reconstruction loss and clustering loss, and iteratively updates the parameters of the autoencoder structure under the correction of the local geometric information regularization term and the consistency and complementarity regularization term, and sets the iterative update stop condition. The original model obtained after the iterative update stops is the final clustering model;

[0067] The clustering module inputs the view of the data set to be processed into the clustering model, and the clustering model outputs the clustering results.

[0068] Furthermore, the data processing module includes:

[0069] In the first submodule, the KNN algorithm processes the view data and calculates a similarity graph. Based on the similarity graph, the local geometric information regularization term and the joint probability of any two views are calculated.

[0070] The second submodule calculates the cosine similarity of the vectors of any two views after dimensionality reduction, defines a hyperparameter ε to represent the similarity threshold, calculates the similarity parameters of any two views based on the cosine similarity and the hyperparameter ε, introduces time series information to calculate the consistency parameters of the two views based on the similarity parameters, and calculates the consistency and complementarity regularization terms based on the consistency parameters and the joint probability of the first submodule;

[0071] The third submodule calculates the original view importance of each view according to the similarity parameter of the second submodule, and calculates the original view importance weight based on the original view importance.

[0072] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: the present invention extracts features and reduces the dimension of each view, introduces weights to measure the correlation between different views, and applies them to the clustering process, which is beneficial to improving the accuracy and stability of clustering; the present invention also introduces a view similarity threshold to more accurately judge whether two views contain consistency information, and takes into account the timing information, so that the views with consistency information and later in time have similar features after encoding as the views earlier in time, without affecting the encoding features of the views earlier in time, and can accurately mine potential patterns and relationships, which is beneficial to further improving the clustering effect and clustering accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 It is a schematic diagram of the process structure of the method of the present invention;

[0074] Figure 2 It is a structural schematic diagram of the system of the present invention;

[0075] Figure 3 A comparison chart of experimental results of various algorithms of the embodiment of the system of the present invention on the LOLS9 data set;

[0076] Figure 4 A comparison chart of experimental results of various algorithms of the embodiment of the system of the present invention on the LOLS10 data set;

[0077] Figure 5 A comparison chart of experimental results of various algorithms of the embodiment of the system of the present invention on the LOLS11 data set;

[0078] Figure 6 A comparison chart of experimental results of various algorithms of the embodiment of the system of the present invention on the LOLS12 data set;

[0079] Figure 7 This is a visualization diagram of different clustering results of various algorithms of the embodiments of the system of the present invention on part of the data in the LOLS9 dataset. DETAILED DESCRIPTION

[0080] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0081] Example 1

[0082] The present invention provides a time-series-aware multi-view clustering method, comprising the following steps:

[0083] Obtain a data set of an e-sports team or a player, the data set includes multiple game characters used by the e-sports team or the player, wherein each game character contains multimodal data of multiple matches, and the multimodal data of different matches of each game character are divided into different views based on image data, point sequence data and video data. The multimodal data includes ban-pick game characters, participation rate of each player, equipment and runes of each player, kill image screen, action route, money, number of last hits, number of wards, economic difference of position, damage ratio, economic ratio, etc.

[0084] The data set is preprocessed, data cleaning and data supplement are performed on the data set, and the point sequence data is dimensionlessly quantized.

[0085] An original model for cluster analysis is constructed, the original model includes multiple autoencoder structures, and each preprocessed view is input into a different autoencoder structure of the original model, the autoencoder structure includes an encoder and a decoder.

[0086] The encoder performs dimensionality reduction on the data X protected by the view to d c dimension, and extract the feature representation H after dimensionality reduction. The decoder reconstructs the feature representation H to obtain the reconstructed data Compute the reconstruction loss.

[0087] Calculate the penalty weight corresponding to each view. The calculation formula is as follows:

[0088]

[0089] in The jth element in the vector corresponding to the sth view of the ith game character is represented by . The penalty weight is used to impose a higher penalty on the reconstruction error of non-zero elements and a lower penalty on the reconstruction error of zero elements. Its purpose is to alleviate the instability caused by sparse data reconstruction to a certain extent and distinguish some more important features.

[0090] The reconstruction loss of the corresponding view is calculated based on the penalty weight. The calculation formula is as follows:

[0091]

[0092] Where n represents the number of views, m represents the number of game characters, represents the reconstructed data of the s-th view of the ith game character, and a⊙b represents the Hadamard product of matrices a and b.

[0093] The KNN algorithm is used to process the view data and calculate the similarity graph. Based on the similarity graph, the local geometric information regularization term and the joint probability of any two views are calculated.

[0094] The formula for calculating the local geometric information regularization term based on the similarity graph is as follows:

[0095]

[0096] in The s-th view of the i-th game character corresponds to the low-dimensional feature representation output by the encoder. represents the joint probability between two different low-dimensional feature representations of the s-th view, which is part of the mechanism used to capture consistent and complementary information in multi-view clustering models. represents the weights of the i-th sample and the j-th sample in the s-th view in the similarity graph;

[0097] Calculate the joint probability of each view, the calculation formula is as follows:

[0098]

[0099] Calculate the cosine similarity of the vectors of any two views after dimensionality reduction, define the hyperparameter ε to represent the similarity threshold, calculate the similarity parameters of any two views based on the cosine similarity and the hyperparameter ε, introduce the time series information on the basis of the similarity parameters to calculate the consistency parameters of the corresponding two views, and calculate the consistency and complementarity regularization terms based on the joint probability and consistency parameters. Calculate the cosine similarity of the vectors of any two views after dimensionality reduction, the calculation formula is as follows:

[0100]

[0101] in represents the dimension-reduced Feature representation, represents the Sth 1 views, ·‖ represents the L2 norm, which is the square root of the sum of the squares of all elements in the vector, representing the Euclidean length of the vector, also known as the Euclidean distance.

[0102] Define the hyperparameter ε to represent the similarity threshold, and the real number ε satisfies ε∈[-1,1].

[0103] The similarity parameters of the two views are calculated based on the cosine similarity. The calculation formula is as follows:

[0104]

[0105] Based on the similarity parameter, the consistency parameter of the two views is calculated. Representing low-dimensional features of two views and Does it represent the same sample? If so, If not then

[0106] Two preliminary views with timing information and The consistency parameter calculation formula is:

[0107]

[0108] Two preliminary views without timing information and The consistency parameter calculation formula is:

[0109]

[0110] A preliminary view without timing information and a preliminary view with timing information The consistency parameter calculation formula is:

[0111]

[0112] This embodiment introduces a view similarity threshold to more accurately determine whether two views contain consistency information. At the same time, by introducing timing information, the encoded features of the view with consistency information and a later timing are similar to those of the view at the earlier timing, without affecting the encoding features of the view at the earlier timing, which is beneficial to improving the clustering effect and clustering accuracy.

[0113] The consistency and complementarity regularization terms are calculated based on the consistency parameter and joint probability. The calculation formula is as follows:

[0114]

[0115] The original view importance of each view is calculated based on the similarity parameter, and then the original view importance weight is calculated based on the original view importance. The calculation formula is as follows:

[0116]

[0117] Among them, view-s represents the original view importance of the s-th view, and the calculation formula is as follows:

[0118]

[0119] Where W view-s represents the original view importance weight of the s-th view, min(O view ) represents the minimum value of the original view importance weight. Since the view importance weight is often used to scale the contribution of the view to the effective information, negative values ​​and excessively large values ​​may lead to unreasonable results. Therefore, in order to ensure the effectiveness and accuracy of the view importance weight, the view importance weight must satisfy non-negativity and normalization.

[0120] The low-dimensional features extracted from the view are fused with the original view importance weight of the view. The fusion method is to multiply the low-dimensional features of each view with its corresponding original view importance weight; the view importance weight affects the input features of clustering by scaling the low-dimensional features of each view. Different views have different importance weights. More important view features will have a greater impact on the clustering process during clustering, while less important views will have a relatively small impact on clustering. This adjustment ensures that high-quality views have more weight in the clustering process, thereby improving the accuracy of clustering.

[0121] Calculate the prediction distribution matrix Q of the original model, the calculation formula is as follows:

[0122]

[0123] where q ij It is used to represent the probability that the i-th element obtained by clustering prediction belongs to the j-th cluster. λ represents the free parameter of t distribution, which is usually set to 1.

[0124] Calculate the target distribution matrix P of the original model. The calculation formula is as follows:

[0125]

[0126] The element p in the target distribution matrix P ij It is used to represent the probability that the i-th element in the auxiliary target distribution belongs to the j-th cluster.

[0127] The target distribution matrix P and the predicted distribution matrix Q calculate the clustering loss of the autoencoder structure. The calculation formula is as follows:

[0128]

[0129] The KL divergence is used to measure the similarity between distributions P and Q. The larger the KL divergence, the greater the gap between the two distributions.

[0130] Backpropagation is performed based on the reconstruction loss and clustering loss, and the parameters of the autoencoder structure are iteratively updated under the correction of the local geometric information regularization term and the consistency and complementarity regularization term. The clustering results output by the original model after each iterative update are recorded. For each game character, the cluster center corresponding to the maximum value in its predicted distribution matrix Q is taken as its label, and the clustering result is obtained.

[0131] The iterative update stopping condition is set. If the difference between the current clustering result and the previous iterative clustering result is less than the threshold δ, or the number of iterations reaches the preset maximum number of iterations, the original model obtained after the iterative update stops is the final clustering model.

[0132] The view of the data set to be processed is input into the clustering model, and the clustering model outputs the clustering results.

[0133] Example 2

[0134] In order to verify the beneficial effects of the method of the present invention, the method of the present invention is compared with the existing MVC-MAE algorithm, Co-train algorithm, MVGL algorithm and MCGC algorithm. The experiments in this implementation are carried out on a Windows system, the graphics card is NVIDIARTX2070super, the processor is Intel(R)Core(TM)i7-9700, and the memory is 32GB (in actual use, users can choose different hardware according to their own needs). The experimental results are obtained by averaging 10 epochs of pre-training and 3 experiments.

[0135] Since clustering is usually only one of the steps in data analysis, and the analysis methods of different e-sports analysis experts may also be different, it is difficult to conclude through direct experiments that the clustering algorithm can improve the prediction accuracy of e-sports analysis. This embodiment can verify the improvement of clustering effect and qualitatively analyze the impact of the improvement of clustering effect on the prediction accuracy of e-sports games.

[0136] This experiment selected game data from four seasons of League of Legends S9 to S12, that is, data from 2019 to 2022. Each data set contains data from different games of different game characters. According to the different seasons, this embodiment cleans them and divides them into four data sets. The detailed information of each data set is shown in the following table:

[0137] Table 1

[0138]

[0139] In order to verify the effectiveness of the method of the present invention, the MVC-MAE algorithm, Co-train algorithm, MVGL algorithm and MCGC algorithm are compared with the method proposed in the present invention in this experiment (the method of the present invention is Figure 3-Figure 7Marked as STVW-MVC in .

[0140] Since the purpose of this embodiment is to verify that the method of the present invention is better than the previous multimodal deep autoencoder clustering algorithm in the clustering effect of e-sports game mode analysis problems, and the clustering process is unsupervised clustering, this paper uses the silhouette coefficient (Silhouette coefficient) and the DB index (Davies-Bouldin) to measure the pros and cons of the experimental results. The silhouette coefficient can measure the compactness and separation of clusters by comparing the distance between the data point and its cluster and other clusters. The value range is between -1 and 1. The closer to 1, the tighter the cluster and the better it is separated from other clusters. The DB index is used to measure the quality of the clustering results. It is based on the compactness within the cluster and the separation between clusters, and quantifies the effect of clustering by calculating the average distance between clusters and the maximum distance within the cluster.

[0141] The silhouette coefficient is calculated based on the compactness of each sample within its cluster and the separation from the nearest cluster. The value range is between -1 and 1. The closer to 1, the better the sample clustering effect, and the closer to -1, the worse the sample clustering effect. When there are outliers in the clustering results, the outliers are often divided into separate clusters, or the compactness within the cluster is reduced, resulting in a decrease in the overall silhouette coefficient. Therefore, you can judge whether the clustering is affected by outliers by observing the silhouette coefficient of the clustering results.

[0142] The DB index quantifies the effect of clustering by calculating the average distance between clusters and the maximum distance within a cluster. The smaller the value, the better the clustering effect. The calculation formula of the DB index is as follows:

[0143]

[0144] The diameter i represents the diameter of the ith cluster, d ij represents the distance between the cluster centers of cluster i and cluster j, and C represents the number of clusters.

[0145] This paper compares the method of the present invention (STVW-MVC) with the aforementioned MVC-MAE algorithm, Co-train algorithm, MVGL algorithm and MCGC algorithm on the above four data sets through indicators such as silhouette coefficient and DB index. Figures 3 to 6 The experimental results on the above datasets are shown. Figures 3 to 6It can be seen that the method of the present invention is superior to the traditional multi-view deep autoencoder clustering algorithm. Specifically, on the LOLS9 dataset, compared with the method of the present invention, the method of the present invention increases the silhouette coefficient by 9 percentage points and reduces the DB index by 26%. In summary, the method of the present invention performs better in terms of inter-cluster density, intra-cluster variance, etc. in dealing with the clustering analysis problem of e-sports competitions.

[0146] In order to intuitively illustrate the clustering results of different algorithms, Figure 7 The comparison of different clustering results of some data in the LOLS9 dataset is shown. The experimental results show that the method of the present invention can effectively extract potential features from the data and improve the clustering accuracy.

[0147] This embodiment conducts a detailed analysis of the subclusters in the clustering results to illustrate that the method of the present invention can mine hidden information of players or teams during clustering, which is helpful for experts to perform game analysis.

[0148] In the LOLS9 dataset, this paper sets the number of clusters to 3. Through research and expert analysis, the clustering results of the method proposed in this paper are found that subcluster 1 ( Figure 7 In the game of cluster 2 (marked in gray), both sides tend to use high-damage game characters and choose more aggressive offensive strategies in the game. Figure 7 In the matches marked in yellow in the figure, both teams tend to adopt conservative styles of play, and both teams usually adopt control and counterattack strategies in these matches. Figure 7 The match data in Figure 1 (marked in red) show that one side tends to be more aggressive and the other side tends to be conservative and counter-attacking.

[0149] In summary, the method proposed in the present invention is conducive to optimizing the clustering effect, exploring the hidden patterns and tactics of teams or players, helping experts to analyze the game results more accurately, and providing technical support for e-sports competition data analysis.

[0150] Example 3

[0151] The time-series-aware multi-view clustering system described in the present invention includes a data acquisition module, an original clustering module, a dimensionality reduction extraction and reconstruction module, a data processing module, a fusion module, an iterative update module and a clustering module.

[0152] The data acquisition module is used to obtain a data set of an e-sports team or a player, wherein the data set includes multiple views divided according to image data, point sequence data, and video data; the data set includes multiple game characters used by the e-sports team or the player, wherein each game character contains multimodal data of multiple game matches, and the multimodal data of different matches of each game character are divided into different views according to image data, point sequence data, and video data. Multimodal data includes ban-pick game characters, participation rate of each player, equipment and runes of each player, kill image screen, action route, money, number of last hits, number of eye insertions, economic difference in position, damage ratio, economic ratio, etc. Preferably, the data acquisition module can pre-process the data set, clean and supplement the data set, and perform dimensionless quantization on the point sequence data.

[0153] The original clustering module is used to construct the original model of clustering analysis. The original model includes multiple autoencoder structures. Each view of the data acquisition module inputs a different autoencoder structure of the original model. The autoencoder structure includes an encoder and a decoder.

[0154] Dimensionality reduction extraction and reconstruction module: The autoencoder structure of the original clustering module performs dimensionality reduction processing on the data X contained in the view, extracts the feature representation H after dimensionality reduction, and clusters the low-dimensional features to obtain the initial clustering center. The autoencoder structure reconstructs the feature representation H to obtain the reconstructed data And calculate the reconstruction loss.

[0155] The encoder performs dimensionality reduction on the data X protected by the view to d c dimension, and extract the feature representation H after dimensionality reduction. The decoder reconstructs the feature representation H to obtain the reconstructed data Compute the reconstruction loss.

[0156] Calculate the penalty weight corresponding to each view. The calculation formula is as follows:

[0157]

[0158] in The jth element in the vector corresponding to the sth view of the ith game character is represented by . The penalty weight is used to impose a higher penalty on the reconstruction error of non-zero elements and a lower penalty on the reconstruction error of zero elements. Its purpose is to alleviate the instability caused by sparse data reconstruction to a certain extent and distinguish some more important features.

[0159] The reconstruction loss of the corresponding view is calculated based on the penalty weight. The calculation formula is as follows:

[0160]

[0161] Where n represents the number of views, m represents the number of game characters, represents the reconstructed data of the s-th view of the ith game character, and a⊙b represents the Hadamard product of matrices a and b.

[0162] The data processing module is used to calculate the local geometric information regularization term, consistency and complementarity regularization term and original view importance weight of each view. The data processing module includes a first submodule, a second submodule and a third submodule.

[0163] In the first submodule, the KNN algorithm processes the view data and calculates a similarity graph. Based on the similarity graph, the local geometric information regularization term and the joint probability of any two views are calculated.

[0164] The formula for calculating the local geometric information regularization term based on the similarity graph is as follows:

[0165]

[0166] in The s-th view of the i-th game character corresponds to the low-dimensional feature representation output by the encoder. represents the joint probability between two different low-dimensional feature representations of the s-th view, which is part of the mechanism used to capture consistent and complementary information in multi-view clustering models. represents the weights of the i-th sample and the j-th sample in the s-th view in the similarity graph;

[0167] Calculate the joint probability of each view, the calculation formula is as follows:

[0168]

[0169] The second sub-module calculates the cosine similarity of the vectors of any two views after dimensionality reduction processing, defines a hyperparameter ε to represent the similarity threshold, calculates the similarity parameters of any two views based on the cosine similarity and the hyperparameter ε, introduces time series information based on the similarity parameters to calculate the consistency parameters of the corresponding two views, and calculates the consistency and complementarity regularization terms based on the consistency parameters and the joint probability of the first sub-module.

[0170] Calculate the cosine similarity of the vectors of any two views after dimensionality reduction. The calculation formula is as follows:

[0171]

[0172] in represents the dimension-reduced Feature representation, represents the Sth 1views, ·‖ represents the L2 norm, which is the square root of the sum of the squares of all elements in the vector, representing the Euclidean length of the vector, also known as the Euclidean distance.

[0173] Define the hyperparameter ε to represent the similarity threshold, and the real number ε satisfies ε∈[-1,1].

[0174] The similarity parameters of the two views are calculated based on the cosine similarity. The calculation formula is as follows:

[0175]

[0176] Based on the similarity parameter, the consistency parameter of the two views is calculated. Representing low-dimensional features of two views and Does it represent the same sample? If so, If not then

[0177] Two preliminary views with timing information and The consistency parameter calculation formula is:

[0178]

[0179] Two preliminary views without timing information and The consistency parameter calculation formula is:

[0180]

[0181] A preliminary view without timing information and a preliminary view with timing information The consistency parameter calculation formula is:

[0182]

[0183] This embodiment introduces a view similarity threshold to more accurately determine whether two views contain consistency information. At the same time, by introducing timing information, the encoded features of the view with consistency information and a later timing are similar to those of the view at the earlier timing, without affecting the encoding features of the view at the earlier timing, which is beneficial to improving the clustering effect and clustering accuracy.

[0184] The consistency and complementarity regularization terms are calculated based on the consistency parameter and joint probability. The calculation formula is as follows:

[0185]

[0186] The third submodule calculates the original view importance of each view according to the similarity parameter of the second submodule, and calculates the original view importance weight based on the original view importance.

[0187] The original view importance of each view is calculated based on the similarity parameter, and then the original view importance weight is calculated based on the original view importance. The calculation formula is as follows:

[0188]

[0189] Among them, view-s represents the original view importance of the s-th view, and the calculation formula is as follows:

[0190]

[0191] Where W view-s represents the original view importance weight of the s-th view, min(O view ) represents the minimum value of the original view importance weight. Since the view importance weight is often used to scale the contribution of the view to the effective information, negative values ​​and excessively large values ​​may lead to unreasonable results. Therefore, in order to ensure the effectiveness and accuracy of the view importance weight, the view importance weight must satisfy non-negativity and normalization.

[0192] In the fusion module, the low-dimensional features extracted from the view are fused with the original view importance weights of the view, and then the prediction distribution matrix Q of the original model and the target distribution matrix P of the original model are calculated. The clustering loss of the autoencoder structure is calculated based on the target distribution matrix P and the prediction distribution matrix Q.

[0193] The low-dimensional features extracted from the view are fused with the original view importance weight of the view. The fusion method is to multiply the low-dimensional features of each view with its corresponding original view importance weight; the view importance weight affects the input features of clustering by scaling the low-dimensional features of each view. Different views have different importance weights. More important view features will have a greater impact on the clustering process during clustering, while less important views will have a relatively small impact on clustering. This adjustment ensures that high-quality views have more weight in the clustering process, thereby improving the accuracy of clustering.

[0194] Calculate the prediction distribution matrix Q of the original model, the calculation formula is as follows:

[0195]

[0196] where q ij It is used to represent the probability that the i-th element obtained by clustering prediction belongs to the j-th cluster. λ represents the free parameter of t distribution, which is usually set to 1.

[0197] Calculate the target distribution matrix P of the original model. The calculation formula is as follows:

[0198]

[0199] The element p in the target distribution matrix P ij It is used to represent the probability that the i-th element in the auxiliary target distribution belongs to the j-th cluster.

[0200] The target distribution matrix P and the predicted distribution matrix Q calculate the clustering loss of the autoencoder structure. The calculation formula is as follows:

[0201]

[0202] The KL divergence is used to measure the similarity between distributions P and Q. The larger the KL divergence, the greater the gap between the two distributions.

[0203] The iterative update module back-propagates based on the reconstruction loss and clustering loss, and iteratively updates the parameters of the autoencoder structure under the correction of the local geometric information regularization term and the consistency and complementarity regularization term, and records the clustering results output by the original model after each iterative update. For each game character, the cluster center corresponding to the maximum value in its predicted distribution matrix Q is taken as its label, and the clustering result is obtained.

[0204] The iterative update stopping condition is set, and the original model obtained after the iterative update stops is the final clustering model; preferably, the iterative update stopping condition is that the difference between the current clustering result and the previous iterative clustering result is less than a threshold δ, or the number of iterations reaches a preset maximum number of iterations.

[0205] The clustering module inputs the view of the data set to be processed into the clustering model, and the clustering model outputs the clustering results.

Claims

1. A time-series-aware multi-view clustering method, characterized by: The following steps are involved: S1: Obtain a data set of an e-sports team or a player, wherein the data set includes multiple game characters used by the e-sports team or the player, wherein each game character includes multimodal data of multiple game matches, and the multimodal data of different matches of each game character are divided into different views according to image data, point sequence data and video data; S2: construct an original model for cluster analysis, the original model includes multiple autoencoder structures, and input each view into a different autoencoder structure of the original model; S3: The autoencoder structure performs dimensionality reduction processing on the data X contained in the view, extracts the feature representation H after dimensionality reduction, clusters the low-dimensional features to obtain the initial clustering center, and reconstructs the feature representation H to obtain the reconstructed data Calculate reconstruction loss; S4: Use the KNN algorithm to process the view data and calculate a similarity graph. Based on the similarity graph, the local geometric information regularization term and the joint probability of any two views are calculated. S5: Calculate the cosine similarity of the vectors of any two views after dimensionality reduction, define a hyperparameter ε to represent the similarity threshold, calculate the similarity parameter of any two views based on the cosine similarity and the hyperparameter ε, introduce the time series information on the basis of the similarity parameter to calculate the consistency parameter of the corresponding two views, and calculate the consistency and complementarity regularization terms based on the joint probability and the consistency parameter; S6: calculating the original view importance of each view according to the similarity parameter, and calculating the original view importance weight based on the original view importance; S7: The low-dimensional features extracted from the view are integrated with the original view importance weight of the view, and then the prediction distribution matrix Q of the original model and the target distribution matrix P of the original model are calculated; S8: Calculate the clustering loss of the autoencoder structure based on the target distribution matrix P and the prediction distribution matrix Q; S9: Back-propagate according to the reconstruction loss and clustering loss, and iteratively update the parameters of the autoencoder structure under the correction of the local geometric information regularization term and the consistency and complementarity regularization term, and set the iterative update stop condition. The original model obtained after the iterative update stops is the final clustering model; S10: Input the view of the data set to be processed into the clustering model, and the clustering model outputs the clustering result.

2. The time-series-aware multi-view clustering method according to claim 1, characterized in that: The steps for calculating the reconstruction loss in step S2 are as follows: Calculate the penalty weight corresponding to each view. The calculation formula is as follows: in represents the jth element in the vector corresponding to the sth view of the ith game character; The reconstruction loss of the corresponding view is calculated based on the penalty weight. The calculation formula is as follows: Where n represents the number of views, m represents the number of game characters, represents the reconstructed data of the s-th view of the ith game character, and a⊙b represents the Hadamard product of matrices a and b.

3. The time-series-aware multi-view clustering method according to claim 2, characterized in that: The steps of calculating the local geometric information regularization term and the joint probability of any two views based on the similarity graph in step S4 are as follows: Based on the similarity graph, the local geometric information regularization term is calculated, and the calculation formula is as follows: in The s-th view of the i-th game character corresponds to the low-dimensional feature representation output by the encoder, represents the joint probability between two different low-dimensional feature representations of the s-th view, represents the weights of the i-th sample and the j-th sample in the s-th view in the similarity graph; Calculate the joint probability of each view, the calculation formula is as follows:

4. The time-series-aware multi-view clustering method according to claim 3, characterized in that: The formula for calculating the cosine similarity of the vectors of any two views after the dimensionality reduction process in step S5 is: in represents the dimension-reduced Feature representation, represents the S1th view of the i-th game character, and ||·|| represents the L2 norm.

5. The time-series-aware multi-view clustering method according to claim 4, characterized in that: The steps of calculating the consistency parameter in step S5 are as follows: Calculate the similarity parameter of the two views according to the cosine similarity, and the calculation formula is as follows: Two preliminary views with timing information and The consistency parameter calculation formula is: Two preliminary views without timing information and The consistency parameter calculation formula is: A preliminary view without timing information and a preliminary view with timing information The consistency parameter calculation formula is: The consistency parameter Representing low-dimensional features of two views and Does it represent the same sample? If so, If not then 6. The time-series-aware multi-view clustering method according to claim 5, characterized in that: The way to calculate the importance weight of the original view in step S6 is as follows: The consistency and complementarity regularization terms are calculated based on the consistency parameter and joint probability. The calculation formula is as follows: The original view importance of each view is calculated based on the similarity parameter. The calculation formula is as follows: Among them, view-s represents the original view importance of the s-th view; The original view importance weight is calculated based on the original view importance. The calculation formula is as follows: Where W view-s represents the original view importance weight of the s-th view, min(O view ) represents the minimum value of the original view importance weight.

7. The time-series-aware multi-view clustering method according to claim 6, characterized in that: Step S7: The steps of calculating the prediction distribution matrix Q of the original model and the target distribution matrix P of the original model are as follows: Calculate the prediction distribution matrix Q of the original model, the calculation formula is as follows: where q ij It is used to represent the probability that the i-th element obtained by clustering prediction belongs to the j-th cluster, and λ represents the free parameter of t distribution; Calculate the target distribution matrix P of the original model. The calculation formula is as follows:

8. The time-series-aware multi-view clustering method according to claim 7, characterized in that: The calculation formula of clustering loss in step S8 is as follows: The KL divergence is used to measure the similarity between distributions P and Q.

9. A time-series-aware multi-view clustering system, characterized by: include, A data acquisition module is used to acquire a data set of an e-sports team or a player, where the data set includes multiple views divided into image data, point sequence data, and video data; The original clustering module is used to construct the original model of clustering analysis. The original model includes multiple autoencoder structures. Each view of the data acquisition module inputs a different autoencoder structure of the original model. Dimensionality reduction extraction and reconstruction module: The autoencoder structure of the original clustering module performs dimensionality reduction processing on the data X contained in the view, extracts the feature representation H after dimensionality reduction, and clusters the low-dimensional features to obtain the initial clustering center. The autoencoder structure reconstructs the feature representation H to obtain the reconstructed data And calculate the reconstruction loss; A data processing module is used to calculate the local geometric information regularization term, consistency and complementarity regularization term and the importance weight of the original view of each view; Fusion module: The low-dimensional features extracted from the view are fused with the original view importance weight of the view, and then the prediction distribution matrix Q of the original model and the target distribution matrix P of the original model are calculated. The clustering loss of the autoencoder structure is calculated based on the target distribution matrix P and the prediction distribution matrix Q. The iterative update module back-propagates according to the reconstruction loss and clustering loss, and iteratively updates the parameters of the autoencoder structure under the correction of the local geometric information regularization term and the consistency and complementarity regularization term, and sets the iterative update stop condition. The original model obtained after the iterative update stops is the final clustering model; The clustering module inputs the view of the data set to be processed into the clustering model, and the clustering model outputs the clustering results.

10. A time-series-aware multi-view clustering system according to claim 9, characterized in that: The data processing module includes: In the first submodule, the KNN algorithm processes the view data and calculates a similarity graph. Based on the similarity graph, the local geometric information regularization term and the joint probability of any two views are calculated. The second submodule calculates the cosine similarity of the vectors of any two views after dimensionality reduction, defines a hyperparameter ε to represent the similarity threshold, calculates the similarity parameters of any two views based on the cosine similarity and the hyperparameter ε, introduces time series information to calculate the consistency parameters of the two views based on the similarity parameters, and calculates the consistency and complementarity regularization terms based on the consistency parameters and the joint probability of the first submodule; The third submodule calculates the original view importance of each view according to the similarity parameter of the second submodule, and calculates the original view importance weight based on the original view importance.

Images