Multivariate time series data classification method and device based on federated distillation learning system

Abstract

The present disclosure provides a multivariate time series data classification method and device based on a federated distillation learning system, including: a server sends an initial time series data classification model and a distillation model to each client, and provides a time series data knowledge base including a relationship directory and a knowledge queue; each client uses the initial model to classify and predict the local multivariate time series data training set, and uploads the predicted data set to the server after forming it; the server updates the relationship directory and the knowledge queue based on these data, and then feeds back the updated neighbor knowledge label vector set to the client; the client refines this knowledge through the distillation model, and evaluates the local model in combination with a preset loss function; if the loss value exceeds the threshold, the model is adjusted and retrained; otherwise, the optimization model is applied to classify the new time series data to be classified. The present disclosure improves the accuracy and robustness of multivariate time series data classification, and optimizes the information sharing and model updating process in federated learning.

Description

Technical Field

[0001] The present disclosure relates to the technical field of data classification, and in particular to a multivariate time series data classification method and device based on a federated distillation learning system. Background Art

[0002] Time series data, as a continuous and ordered data type recorded in time series, plays a vital role in many fields such as finance, medical care, transportation, meteorology, etc. The classification task of time series data aims to accurately classify these data into predefined categories by analyzing a series of input data. With the advancement of technology and the increase in data volume, the accuracy and efficiency of time series data classification have become a hot topic of research. Common time series data classification methods, such as distance metric-based methods, feature engineering methods, and deep learning methods, have met the classification needs to a certain extent, but they still face many challenges when processing large-scale, multi-dimensional time series data that require strict privacy protection.

[0003] As an emerging distributed machine learning method, federated learning provides a new approach to solving the privacy protection and large-scale data processing problems in time series data classification. By allowing multiple clients to collaborate on training models without centralizing the data on a central server, federated learning effectively protects data privacy and improves the generalization ability of the model. However, existing federated learning methods still have some drawbacks in time series data classification. For example, the existing weighted average classification method, although it has achieved remarkable results in the field of federated learning, assumes that the data of all clients are independent and identically distributed, which is often not true in the real world, especially in the field of time series data. Due to the heterogeneity of time series data, this method may lead to poor model performance. In addition, as the number of clients increases, each round of iteration requires frequent model parameter transmission, which not only increases the communication cost, but also may affect the training speed and convergence of the model. Summary of the invention

[0004] The disclosed embodiments at least provide a method and apparatus for classifying multivariate time series data based on a federated distillation learning system, which improves the accuracy and robustness of multivariate time series data classification, optimizes the information sharing and model updating process in federated learning, effectively protects user privacy, and reduces the risk of data leakage. At the same time, through federated distillation learning, the client can enhance local model performance by only sharing model distillation knowledge without uploading original data, thereby promoting cross-device collaborative learning. At the same time, without reducing the classification accuracy, the communication efficiency is further improved by transmitting distillation knowledge rather than model parameters, thereby achieving a balance between privacy protection and learning efficiency.

[0005] The embodiment of the present disclosure provides a multivariate time series data classification method based on a federated distillation learning system, wherein the federated distillation learning system includes a server and multiple clients, including:

[0006] Step 1: The server sends a pre-built initial time series data classification model and a distillation model to each of the multiple clients; wherein the server includes a time series data knowledge base, the time series data knowledge base includes a relationship directory and a knowledge queue, the relationship directory includes multiple knowledge label vector groups, the knowledge label vector group includes a data subset number corresponding to each client, a vector number in the data subset, and a knowledge label vector corresponding to the vector number; the knowledge queue includes multiple neighbor knowledge label vector sets corresponding to the knowledge label vector group; each of the knowledge label vector groups includes a knowledge label vector index, and the knowledge label vector index is used to point to the neighbor knowledge label vector set corresponding to the knowledge label vector group in the knowledge queue;

[0007] Step 2: Each client determines the initial time series data classification model as the local model corresponding to the client; and based on the local model corresponding to each client, classifies and predicts the multivariate time series data training set obtained by each client, to obtain a multivariate time series classification data prediction label set corresponding to each client;

[0008] Step 3: Each of the clients sends the multivariate time series classification data prediction label set obtained by each client to the server; for the multivariate time series classification data prediction label set corresponding to each client, the knowledge label vector group corresponding to each multivariate time series classification data prediction label label set in the relationship directory is updated based on the server, and the neighboring knowledge label vector set pointed to by the knowledge label vector index of the knowledge label vector group in the knowledge queue is updated based on the updated relationship directory, and the updated neighboring knowledge label vector set corresponding to each client in the knowledge queue is sent to each client respectively;

[0009] Step 4: The distillation model corresponding to each client performs knowledge distillation on the neighbor knowledge label vector set received by each client, and according to the knowledge distillation result corresponding to each client, the multivariate time series classification data prediction label set corresponding to each client and the preset loss function, each client calculates the local model loss value of the local model. If the local model loss value is greater than the preset loss threshold, the local model is adjusted, and the adjusted local model is used as the local model in step 2, and the process returns to step 2; otherwise, each client classifies the collected time series data set to be classified according to the adjusted local model to obtain the classification result of the time series data set to be classified.

[0010] The embodiment of the present disclosure provides a multivariate time series data classification device based on a federated distillation learning system, wherein the federated distillation learning system includes a server and multiple clients, including:

[0011] A model distribution module, used to execute step 1: the server sends the pre-built initial time series data classification model and the distillation model to each of the multiple clients; wherein the server includes a time series data knowledge base, the time series data knowledge base includes a relationship directory and a knowledge queue, the relationship directory includes multiple knowledge label vector groups, the knowledge label vector group includes a data subset number corresponding to each client, a vector number in the data subset, and a knowledge label vector corresponding to the vector number; the knowledge queue includes multiple neighbor knowledge label vector sets corresponding to the knowledge label vector group; each of the knowledge label vector groups includes a knowledge label vector index, and the knowledge label vector index is used to point to the neighbor knowledge label vector set corresponding to the knowledge label vector group in the knowledge queue;

[0012] The data classification module is used to execute step 2: each client determines the initial time series data classification model as the local model corresponding to the client; and based on the local model corresponding to each client, classifies and predicts the multivariate time series data training set obtained by each client, and obtains the multivariate time series classification data prediction label set corresponding to each client;

[0013] A data update module is used to execute step 3: each of the clients sends the multivariate time series classification data prediction label set obtained by each client to the server; for the multivariate time series classification data prediction label set corresponding to each client, the knowledge label vector group corresponding to each multivariate time series classification data prediction label label set in the relationship directory is updated based on the server, and the neighboring knowledge label vector set pointed to by the knowledge label vector index of the knowledge label vector group in the knowledge queue is updated based on the updated relationship directory, and the updated neighboring knowledge label vector set corresponding to each client in the knowledge queue is sent to each client respectively;

[0014] The loss calculation module is used to execute step 4: the distillation model corresponding to each client performs knowledge distillation on the neighbor knowledge label vector set received by each client, and according to the knowledge distillation result corresponding to each client, the multivariate time series classification data prediction label set corresponding to each client and the preset loss function, each client calculates the local model loss value of the local model; if the local model loss value is greater than the preset loss threshold, the local model is adjusted, and the adjusted local model is used as the local model in step 2, and step 2 is returned; otherwise, each client classifies the collected time series data set to be classified according to the adjusted local model to obtain the classification result of the time series data set to be classified.

[0015] An embodiment of the present disclosure provides a computer device, including: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, a multivariate time series data classification method based on a federated distillation learning system as described in any possible implementation manner described above is performed.

[0016] An embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the multivariate time series data classification method based on a federated distillation learning system as described in any possible implementation manner described above is implemented.

[0017] The multivariate time series data classification method and device based on the federated distillation learning system provided in the embodiments of the present disclosure continuously improve the performance of the local model of the client through the collaborative operation of the server and multiple clients. The trained model can accurately and efficiently process the time series data set to be classified, thereby improving the accuracy and robustness of the multivariate time series data classification, optimizing the information sharing and model updating process in federated learning, effectively protecting user privacy, and reducing the risk of data leakage; at the same time, through federated distillation learning, the client can enhance the local model performance by only sharing the model distillation knowledge without uploading the original data, thereby promoting cross-device collaborative learning, and at the same time, on the basis of not reducing the classification accuracy, by transmitting the distillation knowledge instead of the model parameters, the communication efficiency is further improved, thereby achieving a balance between privacy protection and learning efficiency.

[0018] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required to be cited in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A schematic diagram of an application environment of a multivariate time series data classification method based on a federated distillation learning system provided by an embodiment of the present disclosure is shown;

[0021] Figure 2 A flowchart of a multivariate time series data classification method based on a federated distillation learning system provided by an embodiment of the present disclosure is shown;

[0022] Figure 3 A schematic diagram of a multivariate time series data classification method based on a federated distillation learning system provided by an embodiment of the present disclosure is shown;

[0023] Figure 4 A schematic diagram of the structure of a multivariate time series data classification device based on a federated distillation learning system provided by an embodiment of the present disclosure is shown;

[0024] Figure 5 A schematic diagram of the structure of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0026] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0027] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.

[0028] Time series data is a type of continuous, ordered data recorded in time series, which is widely found in many fields such as finance, medical care, transportation, meteorology, etc. The goal of the time series data classification task is to accurately classify it into predefined categories given a series of input data. Common time series data classification methods include distance metric-based methods, feature engineering methods, and deep learning methods. However, with the increasing amount of data, the diversification of data dimensions, and the increase in data privacy protection requirements, especially in sensitive data fields such as medical health and finance, the traditional centralized model training method faces challenges. Centralizing all time series data on a single server for training is not only prone to data leakage, but also difficult to process large amounts of distributed data due to network bandwidth and storage limitations.

[0029] Federated learning, as an emerging distributed machine learning method, allows multiple clients to collaborate on training models without centralizing data on a central server. This method not only protects data privacy, but also improves the generalization ability of the model. Although federated learning has many advantages, it inevitably faces some challenges in terms of time series data, such as communication efficiency, model convergence speed, and how to maintain model performance on non-independent and identically distributed data. To solve these problems, researchers have proposed a variety of improvement methods, including model compression, differential privacy, and combining blockchain technology.

[0030] It has been found that the current classification method for weighted averaging of training model parameters for time series data mainly includes: first, the global model parameters are initialized on the server, and then the client uses local data to train the model and update the model parameters. The client sends the updated model parameters back to the server, and the server performs weighted averaging on the received model parameters to obtain the global model parameters, and distributes this global model to all clients for the next round of training. However, although this method has achieved remarkable results in the field of federated learning, it has some limitations in handling multivariate time series data classification. First, this method assumes that the data of all clients are independent and identically distributed, but in the real world, especially time series data, the data of clients is often not independent and identically distributed. In this case, this method may lead to poor model performance because it cannot effectively handle the heterogeneity of data. Secondly, this method has limitations in communication efficiency, especially when there are a large number of clients, each round of iteration requires communication with the server to transmit model parameters, which leads to high communication costs.

[0031] Based on the above research, a method and device for classifying multivariate time series data based on a federated distillation learning system is provided in an embodiment of the present disclosure, including: a server sends an initial time series data classification model and a distillation model to each client, and provides a time series data knowledge base including a relationship directory and a knowledge queue; each client uses the initial model to classify and predict the local multivariate time series data training set, and uploads the formed prediction data set to the server; the server updates the relationship directory and the knowledge queue based on these data, and then feeds back the updated neighbor knowledge label vector set to the client; the client refines the knowledge through the distillation model, and evaluates the local model in combination with a preset loss function; if the loss value exceeds the threshold, the model is adjusted and retrained; otherwise, the optimization model is applied to classify the new time series data to be classified.

[0032] In the disclosed embodiments, the performance of the local model of the client is continuously improved through the collaborative operation of the server and multiple clients. The trained model can accurately and efficiently process the time series data set to be classified, improve the accuracy and robustness of the classification of multivariate time series data, optimize the information sharing and model update process in federated learning, effectively protect user privacy, and reduce the risk of data leakage; at the same time, through federated distillation learning, the client can enhance the local model performance by only sharing the model distillation knowledge without uploading the original data, promote cross-device collaborative learning, and at the same time, on the basis of not reducing the classification accuracy, by transmitting the distillation knowledge instead of the model parameters, the communication efficiency is further improved, and a balance between privacy protection and learning efficiency is achieved.

[0033] To facilitate understanding of this embodiment, the execution subject of the multivariate time series data classification method based on the federated distillation learning system provided by the embodiment of the present disclosure is first introduced in detail. The multivariate time series data classification method based on the federated distillation learning system provided by the embodiment of the present disclosure can be applied to Figure 1 In an application environment, the client communicates with the server through a network. The client can be a mobile device, a user terminal, a terminal, a handheld device, a computing device, etc. The server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, big data and artificial intelligence platforms.

[0034] The following is a detailed description of the multivariate time series data classification method based on the federated distillation learning system provided in the embodiment of the present application in conjunction with the accompanying drawings. Figure 2 As shown, it is a flowchart of a multivariate time series data classification method based on a federated distillation learning system provided by an embodiment of the present disclosure. The federated distillation learning system includes a server and multiple clients. The method includes the following S201-S204:

[0035] S201, the server sends a pre-built initial time series data classification model and a distillation model to each of the multiple clients.

[0036] It can be understood that the federated distillation learning system proposed in the present disclosure includes a server and multiple clients. The server serves as a central node and is responsible for coordinating and managing the entire learning process. The clients are usually multiple nodes distributed on different devices or systems, which have local data and independently perform learning tasks.

[0037] Specifically, the server sends a pre-built initial time series data classification model and a distillation model for the distillation process to each of the multiple clients. Here, the server includes a time series data knowledge base, which includes a relationship directory and a knowledge queue. The relationship directory stores multiple knowledge label vector groups, each of which represents the representation of a data subset of a certain client in the time series data model. Specifically, the knowledge label vector group includes a data subset sequence number corresponding to each client, a vector sequence number in the data subset, and a knowledge label vector corresponding to the vector sequence number. The knowledge queue is a collection that stores neighboring knowledge label vector sets related to the knowledge label vector group, including multiple neighboring knowledge label vector sets corresponding to the knowledge label vector group; each knowledge label vector group includes a knowledge label vector index, which is used to point to the neighboring knowledge label vector set corresponding to the knowledge label vector group in the knowledge queue. Through this index, the server can quickly retrieve the knowledge label vector related to the data of a certain client.

[0038] Exemplarily, the server also sends the pre-built feature extraction model to each of the multiple clients, and after the server also sends the pre-built feature extraction model to each of the multiple clients, the following (1) to (2) may be included:

[0039] (1) obtaining a global data set, and dividing the global data set into a plurality of non-independent and identically distributed multivariate time series data training sets based on the Dirichlet partitioning method, wherein the server distributes the plurality of non-independent and identically distributed multivariate time series data training sets to the plurality of clients in sequence according to the sequence number of each data training set, so as to ensure that each client can receive a multivariate time series data training set corresponding to the client;

[0040] (2) Each of the clients performs feature extraction on the multivariate time series data training set based on the feature extraction model to obtain a data subset corresponding to each of the clients, and each of the clients uploads the data subset to the relational directory of the time series database of the server; wherein the subset number of each of the data subsets is the same as the number of the multivariate time series data training set corresponding to the data subset, and the data subset includes multiple knowledge label vectors, and each knowledge label vector corresponds to a vector number.

[0041] It is understandable that the global data set contains multivariate time series data that may be involved in all clients. Multivariate time series data refers to a data set that contains multiple variables that change over time, which can involve financial analysis, medical health monitoring, industrial process control and other fields. After obtaining the global data set, the server can preprocess it, including cleaning and normalizing abnormal data, so that the data fits the input requirements of the model. Then, in order to simulate the data distribution in the real world, the server uses the Dirichlet Partition Method to divide this global data set into multiple non-independent and identically distributed multivariate time series data training sets. Among them, non-independent and identically distributed (Non-IID) means that the data between data points are not independent of each other, and their statistical characteristics (such as mean, variance, etc.) may also change over time or other factors. For example, some data may come from different sources or regions, or different time periods, causing the distribution of data to change over time or other factors. Here, for multivariate time series data (such as financial data, medical health monitoring data, industrial control data, etc.), each data point not only includes multiple variables (such as temperature, humidity, pressure, etc.), but also these variables change over time. Time series data has time correlation, which means that the data at one moment may be dependent on the data at the previous moment. This is the non-independence of the data.

[0042] Here, the Dirichlet partitioning method is a random partitioning technique that can ensure that the partitioned subsets can show differences and diversity in data distribution based on the complex probability mechanism of the Dirichlet distribution, while making each partitioned subset have its own data characteristics. After the partitioning is completed, the server will distribute these non-independent and identically distributed multivariate time series data training sets to multiple clients in sequence according to the sequence number of each data training set, for model training and model testing of the client's local model, so as to reduce the privacy risks that may be caused by data transmission.

[0043] For example, assuming that the global data set belongs to the health monitoring data set of the hospital, which includes the health monitoring data of different patients, the global data set is divided into n non-independent and identically distributed health monitoring time series data training sets using the Dirichlet partitioning method. After the partitioning is completed, the server will distribute these non-independent and identically distributed health monitoring time series data training sets to multiple hospitals (i.e., multiple clients) in sequence according to the sequence number of each data training set. In this way, the health monitoring time series data training sets received by each hospital have their own data characteristics (i.e., health monitoring data from different patients).

[0044] This distribution method ensures that each client can obtain a unique and corresponding multivariate time series data training set, which not only maintains the characteristics of each data, but also reflects the difference and diversity in data distribution. In this way, each client can use these data training sets with different characteristics to train and test local models, which not only reduces the privacy risks that may arise during data transmission, but also helps to improve the generalization ability of the model.

[0045] Specifically, after receiving the multivariate time series data training set, each client will use the feature extraction model sent by the server to extract features from the training set. After the feature extraction is completed, each client will obtain a corresponding data subset, which contains multiple knowledge label vectors. Each knowledge label vector corresponds to a vector serial number, which together constitute the feature representation of the client's local data. In order to integrate these feature representations into the server's time series database, each client will upload its data subset to the relational directory of the server's time series database. During the upload process, the subset serial number of each data subset is consistent with the serial number of the multivariate time series data training set corresponding to the data subset.

[0046] Exemplarily, after each client uploads its respective data subset to the relational directory of the server's time series database, for the knowledge label vector corresponding to each knowledge label vector group in the relational directory, the neighboring knowledge label vector set corresponding to each knowledge label vector in the relational directory is determined based on a neighbor search algorithm, and the neighboring knowledge label vector set is stored in the knowledge queue.

[0047] Among them, in order to achieve efficient neighbor search, in this disclosure, the server's neighbor search algorithm uses the HNSW (Hierarchical Navigable Small World graphs) algorithm to calculate R neighbor samples for each client's knowledge label vector, extract personalized knowledge, and further improve the generalization ability of the model. HNSW is an approximate nearest neighbor search algorithm that constructs a hierarchical graph structure to quickly find the most similar point to the query point by gradually narrowing the search range. Here, the server uses the HNSW algorithm to find the R neighbor knowledge labels of each label from the relationship directory based on cosine similarity. Cosine similarity is used to measure the similarity between knowledge label vectors, thereby determining which labels are "similar" and can be stored in the knowledge queue as "neighbors". The value of cosine similarity is between -1 and 1. The larger the value, the more similar the two vectors are.

[0048] It can be understood that after calculating the similarity between each knowledge tag vector, the server uses the HNSW algorithm to find the R nearest neighbor tags of each knowledge tag based on cosine similarity, and its formula can be expressed as:

[0049] ;

[0050] in, Represented as the i-th knowledge vector label, R is The number of neighbor knowledge label vectors in the neighbor knowledge label vector set; It is expressed as the number of candidate neighbors found during a certain search process.

[0051] S202, each client determines the initial time series data classification model as the local model corresponding to the client; and based on the local model corresponding to each client, classification prediction is performed on the multivariate time series data training set obtained by each client respectively, to obtain a multivariate time series classification data prediction label set corresponding to each client.

[0052] It is understandable that, unlike the traditional centralized model training method, the model deployment and initialization of the present disclosure is performed independently on each client, and each client will determine the initial time series data classification model (which can be a CNN data classification model) received by each client as their respective corresponding local model. In this way, the computing resources and data privacy protection requirements of each client can be fully utilized to avoid the risk of data leakage during transmission.

[0053] Specifically, based on the determined local model, each client will use the multivariate time series data training set it has obtained to perform classification prediction and obtain the corresponding multivariate time series classification data prediction label set. Each multivariate time series classification data prediction label set includes multiple multivariate time series classification data prediction labels. The multivariate time series classification data prediction label set corresponding to each client reflects the data distribution and characteristics of the client, and can be used to evaluate the performance of each client's local model under the current data distribution.

[0054] S203, each of the clients sends the multivariate time series classification data prediction label set obtained by each client to the server; for the multivariate time series classification data prediction label set corresponding to each client, the knowledge label vector group corresponding to each multivariate time series classification data prediction label label set in the relationship directory is updated based on the server, and the neighboring knowledge label vector set pointed to by the knowledge label vector index of the knowledge label vector group in the knowledge queue is updated based on the updated relationship directory, and the updated neighboring knowledge label vector set corresponding to each client in the knowledge queue is sent to each client respectively.

[0055] It can be understood that after each client completes the classification prediction of its local multivariate time series data and obtains the multivariate time series classification data prediction label set, each client sends the multivariate time series classification data prediction label set obtained by each client to the server. After the server receives the multivariate time series classification data prediction label set from each client, the server will perform a series of update operations on the current relationship directory according to the multivariate time series classification data prediction label set corresponding to each client. Specifically, for each multivariate time series classification data prediction label in the multivariate time series classification data prediction label set corresponding to each client, the server first finds the corresponding data subset serial number through the prediction set of each client, and determines the corresponding knowledge label vector group according to the serial number; then, the server finds the vector serial number corresponding to the multivariate time series data prediction label in the knowledge label vector group corresponding to the data subset serial number; then the server will update the knowledge label vector corresponding to the vector serial number according to the multivariate time series data prediction label, so as to update the knowledge label vector group corresponding to the multivariate time series classification data prediction label set in the relationship directory.

[0056] Exemplarily, after the server completes updating the relationship directory based on the multivariate time series classification data prediction label set corresponding to each client, the knowledge queue can be updated based on the updated relationship directory. Specifically, the server processes each knowledge label vector group in the updated relationship directory, using the same neighbor knowledge label determination method as the aforementioned step S201, that is, using a neighbor search algorithm to find the neighbor label vector set corresponding to each knowledge label vector, and updates the knowledge queue based on this. After the server completes the update of the knowledge queue, it sends the updated neighbor knowledge label vector set corresponding to each client in the knowledge queue to each client.

[0057] In this way, the server ensures that the updated knowledge label vector maintains a high degree of consistency and correlation with its neighboring vectors, making the information in the knowledge queue more complete and accurate; at the same time, by compressing the data transmitted by the server and the client into a knowledge vector label each time, the amount of data required for each communication can be reduced, thereby improving communication efficiency.

[0058] S204, the distillation model corresponding to each client performs knowledge distillation on the neighbor knowledge label vector set received by each client, and according to the knowledge distillation result corresponding to each client, the multivariate time series classification data prediction label set corresponding to each client and the preset loss function, each client calculates the local model loss value of the local model; if the local model loss value is greater than the preset loss threshold, the local model is adjusted, and the adjusted local model is used as the local model in step S202, and the process returns to step S202; otherwise, each client classifies the collected time series data set to be classified according to the adjusted local model to obtain the classification result of the time series data set to be classified.

[0059] Specifically, after each client receives its corresponding neighbor knowledge label vector set, each client uses its own distillation model to perform knowledge distillation on it to obtain the knowledge distillation result corresponding to each client. Here, knowledge distillation is a model compression technology that transfers the knowledge in a larger and complex model (usually a label vector set generated by a global model on the server side) to a smaller and lighter local model so that the local model can remain efficient without sacrificing performance. In the present disclosure, the client's distillation model uses these neighbor label vector sets as soft labels for training, extracting useful knowledge from them to improve the prediction ability of its local model.

[0060] Exemplarily, after each client obtains its corresponding knowledge distillation results, each client will combine the knowledge distillation results and the multivariate time series classification data prediction label set corresponding to each client itself using a preset loss function to calculate the loss value of each client's local model to determine whether the performance of the local model meets expectations.

[0061] It can be understood that the preset loss function includes a cross-entropy loss function and a distillation loss function. For the local model corresponding to each client, the distillation loss value of the local model is determined based on the distillation loss function according to the knowledge distillation result corresponding to the client and the multivariate time series classification data prediction label set, and the cross-entropy loss value of the local model is determined based on the cross-entropy loss function according to the multivariate time series classification data prediction label set corresponding to the client and the multivariate time series classification data target set.

[0062] Here, the distillation loss function can be expressed as:

[0063] ;

[0064] in, It is expressed as the distillation loss value; t is expressed as the temperature coefficient; Represented as the softmax output result of the distilled knowledge result; Represents the softmax output of predicting the label set for multivariate time series classification data. Here, the softmax function is a multi-classification function that converts a real vector (usually the output of a neural network) into a probability distribution.

[0065] Here, the cross entropy loss function can be expressed as:

[0066] ;

[0067] in, It is represented by the cross entropy loss value; N is represented by the number of multivariate time series training data in the multivariate time series data training set; It is represented as the multivariate time series classification real data of the i-th multivariate time series training data in the multivariate time series classification data target set of the k-th client; Predict the multivariate time series classification data of the k-th client for the i-th multivariate time series training data in the multivariate time series classification data prediction label set.

[0068] For example, after obtaining the distillation loss value and the cross entropy loss value of the local model, the local model loss value can be obtained according to the distillation loss value and the cross entropy loss value, and the formula can be expressed as:

[0069] ;

[0070] in, Denoted as a balance parameter, it is used to control the weight of the two losses.

[0071] It is understandable that if the local model loss value calculated by the client is greater than the preset loss threshold, it means that the performance of the local model has not yet met expectations, and the client will enter the model adjustment stage. At this time, the client needs to adjust the local model to further optimize its prediction accuracy. Model adjustment may involve steps such as changing the model structure or adjusting hyperparameters. In the embodiment of the present disclosure, the client uses the Adam optimizer for model training and optimization. Through back propagation and gradient descent algorithms, the Adam optimizer can effectively adjust the model weights, thereby continuously reducing the loss value and improving the prediction accuracy of the model. After each round of communication, the client will update the parameters of the local model based on the personalized knowledge provided by the server (i.e., the neighbor knowledge label vector set), use the adjusted local model as the new local model, return to step S202, and continue the same process until the loss value of the local model is lower than the preset threshold.

[0072] In this way, through this iterative learning method, each client can not only use its own local data for effective learning, but also benefit from the knowledge of other clients, thereby further improving the generalization ability and global performance of the model. In this way, it is ensured that the model of each client is not only highly adaptable to local data, but also can achieve better results at the global level.

[0073] It is understandable that if the local model loss value calculated by the client is less than or equal to the preset loss threshold, it means that the local model is accurate enough to meet the requirements of the classification task. At this time, the client will classify the collected time series data set to be classified according to the adjusted local model and generate corresponding classification results. These classification results can provide decision support for each client in practical applications. For example: in the financial field, through the classification and prediction of historical transaction data, it can help investors identify potential investment opportunities or risks; in the medical and health field, it may be used for early diagnosis of diseases or prediction of disease development trends.

[0074] In some other embodiments, when the local model loss value calculated by the client is less than or equal to a preset loss threshold and meets the requirements of the classification task, the average accuracy of the calculated model can also be used to evaluate the performance indicators of the model, thereby judging the pros and cons of the model in the multivariate time series data classification task, especially in terms of data privacy protection and communication efficiency improvement.

[0075] Exemplarily, in order to avoid over-training, the present disclosure further proposes that after each client calculates the local model loss value of the local model, the following steps are included (a) to (b):

[0076] (a) determining whether the difference between the local model loss value and the historical local model loss value is greater than a preset value, and if not, recording the number of times the model is not trained; wherein the historical local model loss value includes the local model loss value of the most recent local model training;

[0077] (b) When the number of training times of the local model without model improvement reaches a preset number, the early stopping mechanism is triggered to exit the model training of the local model.

[0078] Specifically, it is determined whether the difference between the currently calculated local model loss value and the historical local model loss value exceeds a preset threshold. If the difference is not greater than the preset value, a "number of training times in which the model has not been improved" is recorded. The historical local model loss value refers to the loss value obtained during the most recent local model training, which serves as a reference for comparing the current loss value. The purpose of this step is to monitor the progress of model training and avoid continuing to train the model without obvious improvement, which wastes computing resources and time.

[0079] It is understandable that when the model fails to improve after a predetermined number of trainings (for example, 50 or 80 times), the early stopping mechanism will be triggered to stop further training of the local model. The early stopping mechanism is an effective training control strategy that can prevent the model from falling into an overfitting state and ensure the efficiency and stability of the training process. By setting appropriate trigger conditions and times, it can ensure that the model stops in time when performance stagnation occurs, avoiding the negative impact of overtraining on model performance.

[0080] In this way, the present disclosure effectively combines the loss value change monitoring and early stopping mechanism, which not only helps to improve the efficiency of training, but also reduces unnecessary computing overhead and provides a more intelligent optimization strategy for the model training process of each client.

[0081] Next, combine Figure 3 , the multivariate time series data classification method based on the federated distillation learning system proposed in the present disclosure is described in detail. Specifically, Figure 3 In the federated distillation learning system shown in the figure, a server and two clients (Client 1 and Client 2) are set up. Each client is equipped with a local model, and its architecture is shown in the figure. The server contains a relationship directory and a knowledge queue. First, the server preprocesses and partitions the global dataset to generate multiple non-independent and identically distributed multivariate time series data training sets. Figure 3For example, two multivariate time series data training sets are generated here. Subsequently, the server distributes these two training sets to client 1 and client 2 respectively. After each client receives its own data set (sample data 1 and sample data 2), it performs feature extraction. After the feature extraction is completed, the client uploads the obtained feature label vector set to the server. This process can be regarded as "sending knowledge". After receiving the knowledge label vector set from the client, the server will build a relationship directory and a knowledge queue based on this information. Next, the local model of each client begins to classify and predict the multivariate time series data training set it receives. After the prediction is completed, the client uploads the prediction results to the server. The server updates the relationship directory and knowledge queue based on these prediction results, and sends the updated neighbor knowledge label vector set corresponding to each client in the knowledge queue to each client. In this way, the client can obtain the knowledge information closest to its current data, so as to perform more accurate knowledge distillation. After receiving the neighbor knowledge label vector set, each client will perform knowledge distillation. Through this process, the client can further refine and integrate knowledge information, thereby optimizing the performance of the local model. Subsequently, the client will calculate the model loss of the local model to determine whether the current model meets the preset requirements and whether the model needs to be adjusted. After iteration and optimization of the above series of steps, each client can finally obtain a trained local model, which can not only accurately classify multivariate time series data, but also realize knowledge sharing and collaborative optimization in the federated learning system while protecting data privacy.

[0082] The multivariate time series data classification method and device based on the federated distillation learning system provided in the embodiments of the present disclosure continuously improve the performance of the local model of the client through the collaborative operation of the server and multiple clients. The trained model can accurately and efficiently process the time series data set to be classified, thereby improving the accuracy and robustness of the multivariate time series data classification, optimizing the information sharing and model updating process in federated learning, effectively protecting user privacy, and reducing the risk of data leakage; at the same time, through federated distillation learning, the client can enhance the local model performance by only sharing the model distillation knowledge without uploading the original data, thereby promoting cross-device collaborative learning, and at the same time, on the basis of not reducing the classification accuracy, by transmitting the distillation knowledge instead of the model parameters, the communication efficiency is further improved, thereby achieving a balance between privacy protection and learning efficiency.

[0083] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0084] Based on the same inventive concept, the embodiments of the present disclosure also provide a multivariate time series data classification device based on a federated distillation learning system corresponding to the multivariate time series data classification method based on a federated distillation learning system. Since the principle of solving the problem by the device in the embodiments of the present disclosure is similar to the multivariate time series data classification method based on a federated distillation learning system in the embodiments of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0085] Reference Figure 4 FIG. 4 is a schematic diagram of a multivariate time series data classification device 400 based on a federated distillation learning system provided in an embodiment of the present disclosure, wherein the device includes:

[0086] The model distribution module 401 is used to execute step 1: the server sends the pre-built initial time series data classification model and the distillation model to each of the multiple clients; wherein the server includes a time series data knowledge base, the time series data knowledge base includes a relationship directory and a knowledge queue, the relationship directory includes multiple knowledge label vector groups, the knowledge label vector group includes a data subset sequence number corresponding to each client, a vector sequence number in the data subset, and a knowledge label vector corresponding to the vector sequence number; the knowledge queue includes multiple neighbor knowledge label vector sets corresponding to the knowledge label vector group; each of the knowledge label vector groups includes a knowledge label vector index, and the knowledge label vector index is used to point to the neighbor knowledge label vector set corresponding to the knowledge label vector group in the knowledge queue;

[0087] The data classification module 402 is used to execute step 2: each client determines the initial time series data classification model as the local model corresponding to the client; and based on the local model corresponding to each client, classifies and predicts the multivariate time series data training set obtained by each client, and obtains the multivariate time series classification data prediction label set corresponding to each client;

[0088] The data update module 403 is used to execute step 3: each of the clients sends the multivariate time series classification data prediction label set obtained by each client to the server; for the multivariate time series classification data prediction label set corresponding to each client, the knowledge label vector group corresponding to each multivariate time series classification data prediction label label set in the relationship directory is updated based on the server, and the neighboring knowledge label vector set pointed to by the knowledge label vector index of the knowledge label vector group in the knowledge queue is updated based on the updated relationship directory, and the updated neighboring knowledge label vector set corresponding to each client in the knowledge queue is sent to each client respectively;

[0089] The loss calculation module 404 is used to execute step 4: the distillation model corresponding to each client performs knowledge distillation on the neighbor knowledge label vector set received by each client, and according to the knowledge distillation result corresponding to each client, the multivariate time series classification data prediction label set corresponding to each client and the preset loss function, each client calculates the local model loss value of the local model; if the local model loss value is greater than the preset loss threshold, the local model is adjusted, and the adjusted local model is used as the local model in step 2, and step 2 is returned; otherwise, each client classifies the collected time series data set to be classified according to the adjusted local model to obtain the classification result of the time series data set to be classified.

[0090] In some possible embodiments, the model distribution module 401 is further used for:

[0091] The server sends the pre-built feature extraction model to each of the multiple clients;

[0092] The model distribution module 401 is also used for:

[0093] Acquire a global data set, and divide the global data set into a plurality of non-independent and identically distributed multivariate time series data training sets based on the Dirichlet partitioning method, wherein the server distributes the plurality of non-independent and identically distributed multivariate time series data training sets to the plurality of clients in sequence according to the sequence number of each data training set, so as to ensure that each client can receive a multivariate time series data training set corresponding to the client;

[0094] Each of the clients performs feature extraction on the multivariate time series data training set based on the feature extraction model to obtain a data subset corresponding to each of the clients, and each of the clients uploads the data subset to the relational directory of the time series database of the server; wherein the subset sequence number of each of the data subsets is the same as the sequence number of the multivariate time series data training set corresponding to the data subset, and the data subset includes multiple knowledge label vectors, and each knowledge label vector corresponds to a vector sequence number.

[0095] In some possible embodiments, the model distribution module 401 is further used for:

[0096] For each knowledge label vector group in the relationship directory, determine a neighbor knowledge label vector set corresponding to each knowledge label vector in the relationship directory based on a neighbor search algorithm, and store the neighbor knowledge label vector set in the knowledge queue;

[0097] The data updating module 403 is specifically used for:

[0098] For the knowledge label vector corresponding to each knowledge label vector group in the updated relationship directory, a neighboring knowledge label vector set corresponding to each updated knowledge label vector in the updated relationship directory is determined based on a neighboring search algorithm, and the knowledge queue is updated based on the neighboring knowledge label vector set corresponding to each updated knowledge label vector.

[0099] In some possible embodiments, the multivariate time series classification data prediction label set includes a plurality of multivariate time series classification data prediction labels, and the data updating module 403 is specifically used to:

[0100] For each multivariate time series classification data prediction label in the multivariate time series classification data prediction label set corresponding to each client, the server determines the data subset number corresponding to the multivariate time series classification data prediction label set corresponding to each client in the relational directory, and determines the vector number corresponding to the multivariate time series data prediction label in the knowledge label vector group corresponding to the data subset number based on the server; and the server updates the knowledge label vector corresponding to the vector number based on the multivariate time series data prediction label, so as to update the knowledge label vector group corresponding to the multivariate time series classification data prediction label set in the relational directory.

[0101] In some possible embodiments, the preset loss function includes a cross entropy loss function and a distillation loss function, and the loss calculation module 404 is specifically used to:

[0102] For the local model corresponding to each client, the distillation loss value of the local model is determined based on the distillation loss function according to the knowledge distillation result corresponding to the client and the multivariate time series classification data prediction label set; and the cross entropy loss value of the local model is determined based on the cross entropy loss function according to the multivariate time series classification data prediction label set corresponding to each client and the multivariate time series classification data target set; the local model loss value is determined based on the distillation loss value and the cross entropy loss array.

[0103] In some possible embodiments, the distillation loss function includes:

[0104] ;

[0105] in, It is expressed as the distillation loss value; t is expressed as the temperature coefficient; Represented as the softmax output result of the distilled knowledge result; Represents the softmax output result of the predicted label set for multivariate time series classification data;

[0106] The cross entropy loss function includes:

[0107] ;

[0108] in, It is represented by the cross entropy loss value; N is represented by the number of multivariate time series training data in the multivariate time series data training set; It is represented as the multivariate time series classification real data of the i-th multivariate time series training data in the multivariate time series classification data target set of the k-th client; Predict the multivariate time series classification data of the k-th client for the i-th multivariate time series training data in the multivariate time series classification data prediction label set.

[0109] In some possible embodiments, the loss calculation module 404 is further used to:

[0110] Determine whether the difference between the local model loss value and the historical local model loss value is greater than a preset value, and if not, record the number of times the model is not improved; wherein the historical local model loss value includes the local model loss value of the most recent local model training;

[0111] When the number of consecutive training times without model improvement of the local model reaches a preset number, the early stopping mechanism is triggered to exit the model training of the local model.

[0112] Based on the same technical concept, the embodiment of the present disclosure also provides a computer device. Figure 5 , which is a schematic diagram of the structure of a computer device 500 provided in an embodiment of the present disclosure, including a processor 501, a memory 502, and a bus 503. The memory 502 is used to store execution instructions, including a memory 5021 and an external memory 5022; the memory 5021 is also called an internal memory, which is used to temporarily store the operation data in the processor 501 and the data exchanged with the external memory 5022 such as a hard disk. The processor 501 exchanges data with the external memory 5022 through the memory 5021.

[0113] In the embodiment of the present application, the memory 502 is specifically used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 501. That is, when the computer device 500 is running, the processor 501 communicates with the memory 502 through the bus 503, so that the processor 501 executes the application code stored in the memory 502, and then executes the method described in any of the above embodiments.

[0114] Among them, the memory 502 can be, but is not limited to, random access memory (Random Access Memory, RAM), read only memory (Read Only Memory, ROM), programmable read-only memory (Programmable Read-Only Memory, PROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM), electrically erasable read-only memory (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.

[0115] Processor 501 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present invention may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0116] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the computer device 500. In other embodiments of the present application, the computer device 500 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0117] The present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the multivariate time series data classification method based on the federated distillation learning system described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0118] The present disclosure embodiment also provides a computer program product, which carries a program code. The instructions included in the program code can be used to execute the steps of the multivariate time series data classification method based on the federated distillation learning system described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.

[0119] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0120] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0121] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0122] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0123] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0124] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A multivariate time series data classification method based on a federated distillation learning system, wherein the federated distillation learning system includes a server and multiple clients, characterized in that: The multivariate time series data classification method comprises: Step 1: The server sends a pre-built initial time series data classification model and a distillation model to each of the multiple clients; wherein the server includes a time series data knowledge base, the time series data knowledge base includes a relationship directory and a knowledge queue, the relationship directory includes multiple knowledge label vector groups, the knowledge label vector group includes a data subset number corresponding to each client, a vector number in the data subset, and a knowledge label vector corresponding to the vector number; the knowledge queue includes multiple neighbor knowledge label vector sets corresponding to the knowledge label vector group; each of the knowledge label vector groups includes a knowledge label vector index, and the knowledge label vector index is used to point to the neighbor knowledge label vector set corresponding to the knowledge label vector group in the knowledge queue; Step 2: Each client determines the initial time series data classification model as the local model corresponding to the client; and classifies and predicts the multivariate time series data training sets obtained by each client based on the local model corresponding to each client, to obtain a multivariate time series classification data prediction label set corresponding to each client; wherein each client obtains its own multivariate time series data training set in the following manner: obtains a global data set, and divides the global data set into multiple non-independent and identically distributed multivariate time series data training sets based on the Dirichlet partition method, and the server distributes the multiple non-independent and identically distributed multivariate time series data training sets to the multiple clients in sequence according to the sequence number of each data training set, so as to ensure that each client can receive a multivariate time series data training set corresponding to the client; Step 3: Each of the clients sends the multivariate time series classification data prediction label set obtained by each client to the server; for the multivariate time series classification data prediction label set corresponding to each client, the knowledge label vector group corresponding to each multivariate time series classification data prediction label label set in the relationship directory is updated based on the server, and the neighboring knowledge label vector set pointed to by the knowledge label vector index of the knowledge label vector group in the knowledge queue is updated based on the updated relationship directory, and the updated neighboring knowledge label vector set corresponding to each client in the knowledge queue is sent to each client respectively; Step 4: The distillation model corresponding to each client performs knowledge distillation on the neighbor knowledge label vector set received by each client, and according to the knowledge distillation result corresponding to each client, the multivariate time series classification data prediction label set corresponding to each client and the preset loss function, each client calculates the local model loss value of the local model. If the local model loss value is greater than the preset loss threshold, the local model is adjusted, and the adjusted local model is used as the local model in step 2, and the process returns to step 2; otherwise, each client classifies the collected time series data set to be classified according to the adjusted local model to obtain the classification result of the time series data set to be classified.

2. The method according to claim 1, characterized in that: The server sends the pre-built initial time series data classification model and the distillation model to each of the multiple clients, and further includes: The server sends the pre-built feature extraction model to each of the multiple clients; Each of the clients performs feature extraction on the multivariate time series data training set based on the feature extraction model to obtain a data subset corresponding to each of the clients, and each of the clients uploads the data subset to the relational directory of the time series database of the server; wherein the subset sequence number of each of the data subsets is the same as the sequence number of the multivariate time series data training set corresponding to the data subset, and the data subset includes multiple knowledge label vectors, and each knowledge label vector corresponds to a vector sequence number.

3. The method according to claim 2, characterized in that After each client uploads the data subset to the relational directory of the time series database of the server, the method includes: For each knowledge label vector group in the relationship directory, determine a neighbor knowledge label vector set corresponding to each knowledge label vector in the relationship directory based on a neighbor search algorithm, and store the neighbor knowledge label vector set in the knowledge queue; The updating of the neighbor knowledge label vector set corresponding to each client in the knowledge queue based on the updated relationship directory includes: For the knowledge label vector corresponding to each knowledge label vector group in the updated relationship directory, a neighboring knowledge label vector set corresponding to each updated knowledge label vector in the updated relationship directory is determined based on a neighboring search algorithm, and the knowledge queue is updated based on the neighboring knowledge label vector set corresponding to each updated knowledge label vector.

4. The method according to claim 2, characterized in that: The multivariate time series classification data prediction label set includes a plurality of multivariate time series classification data prediction labels, and the multivariate time series classification data prediction label set corresponding to each client, based on the server updating the knowledge label vector group corresponding to each multivariate time series classification data prediction label in the multivariate time series classification data prediction label set in the relationship directory, includes: For each multivariate time series classification data prediction label in the multivariate time series classification data prediction label set corresponding to each client, the server determines the data subset number corresponding to the multivariate time series classification data prediction label set corresponding to each client in the relational directory, and determines the vector number corresponding to the multivariate time series data prediction label in the knowledge label vector group corresponding to the data subset number based on the server; and the server updates the knowledge label vector corresponding to the vector number based on the multivariate time series data prediction label, so as to update the knowledge label vector group corresponding to the multivariate time series classification data prediction label set in the relational directory.

5. The method according to claim 1, characterized in that The preset loss function includes a cross entropy loss function and a distillation loss function, and the calculation of the local model loss value of the local model by each client according to the knowledge distillation result corresponding to each client, the multivariate time series classification data prediction label set corresponding to each client and the preset loss function includes: For the local model corresponding to each client, the distillation loss value of the local model is determined based on the distillation loss function according to the knowledge distillation result corresponding to the client and the multivariate time series classification data prediction label set; and the cross entropy loss value of the local model is determined based on the cross entropy loss function according to the multivariate time series classification data prediction label set corresponding to each client and the multivariate time series classification data target set; the local model loss value is determined based on the distillation loss value and the cross entropy loss array.

6. The method according to claim 5, characterized in that The distillation loss function includes: ； in, It is expressed as the distillation loss value; t is expressed as the temperature coefficient; Represented as the softmax output result of the distilled knowledge result; Represents the softmax output result of the predicted label set for multivariate time series classification data; The cross entropy loss function includes: ； in, It is represented by the cross entropy loss value; N is represented by the number of multivariate time series training data in the multivariate time series data training set; It is represented as the multivariate time series classification real data of the i-th multivariate time series training data in the multivariate time series classification data target set of the k-th client; Predict the multivariate time series classification data of the k-th client for the i-th multivariate time series training data in the multivariate time series classification data prediction label set.

7. The method according to claim 1, characterized in that After each client calculates the local model loss value of the local model, the method further includes: Determine whether the difference between the local model loss value and the historical local model loss value is greater than a preset value, and if not, record the number of times the model is not improved; wherein the historical local model loss value includes the local model loss value of the most recent local model training; When the number of consecutive training times without model improvement of the local model reaches a preset number, the early stopping mechanism is triggered to exit the model training of the local model.

8. A multivariate time series data classification device based on a federated distillation learning system, wherein the federated distillation learning system comprises a server and a plurality of clients, characterized in that: include: A model distribution module, used to execute step 1: the server sends the pre-built initial time series data classification model and the distillation model to each of the multiple clients; wherein the server includes a time series data knowledge base, the time series data knowledge base includes a relationship directory and a knowledge queue, the relationship directory includes multiple knowledge label vector groups, the knowledge label vector group includes a data subset number corresponding to each client, a vector number in the data subset, and a knowledge label vector corresponding to the vector number; the knowledge queue includes multiple neighbor knowledge label vector sets corresponding to the knowledge label vector group; each of the knowledge label vector groups includes a knowledge label vector index, and the knowledge label vector index is used to point to the neighbor knowledge label vector set corresponding to the knowledge label vector group in the knowledge queue; A data classification module, used to execute step 2: each client determines the initial time series data classification model as the local model corresponding to the client; and classifies and predicts the multivariate time series data training sets obtained by each client based on the local model corresponding to each client, and obtains the multivariate time series classification data prediction label set corresponding to each client; wherein each client obtains its own multivariate time series data training set in the following manner: obtains a global data set, and divides the global data set into multiple non-independent and identically distributed multivariate time series data training sets based on the Dirichlet partition method, and the server distributes the multiple non-independent and identically distributed multivariate time series data training sets to the multiple clients in sequence according to the sequence number of each data training set, so as to ensure that each client can receive a multivariate time series data training set corresponding to the client; A data update module is used to execute step 3: each of the clients sends the multivariate time series classification data prediction label set obtained by each client to the server; for the multivariate time series classification data prediction label set corresponding to each client, the knowledge label vector group corresponding to each multivariate time series classification data prediction label label set in the relationship directory is updated based on the server, and the neighboring knowledge label vector set pointed to by the knowledge label vector index of the knowledge label vector group in the knowledge queue is updated based on the updated relationship directory, and the updated neighboring knowledge label vector set corresponding to each client in the knowledge queue is sent to each client respectively; The loss calculation module is used to execute step 4: the distillation model corresponding to each client performs knowledge distillation on the neighbor knowledge label vector set received by each client, and according to the knowledge distillation result corresponding to each client, the multivariate time series classification data prediction label set corresponding to each client and the preset loss function, each client calculates the local model loss value of the local model; if the local model loss value is greater than the preset loss threshold, the local model is adjusted, and the adjusted local model is used as the local model in step 2, and the process returns to step 2; otherwise, each client classifies the collected time series data set to be classified according to the adjusted local model to obtain the classification result of the time series data set to be classified.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images