Data processing method and device, storage medium and processor

Abstract

The application discloses a data processing method and device, a storage medium and a processor. The method comprises the following steps: obtaining a plurality of time series data generated in the process of executing a target task; inputting the plurality of time series data into a target model for clustering processing to obtain a clustering result, wherein the target model at least comprises an encoding module and a clustering module, the encoding module is used for extracting features of the plurality of time series data, and the clustering module is used for clustering the features to obtain the clustering result; and analyzing the execution of the target task according to the clustering result. The application solves the technical problem of low accuracy of the clustering method for clustering time series data in the related art.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data processing, in particular to a data processing method and device, a storage medium and a processor. BACKGROUND

[0002] Time series data refers to a sequence formed by arranging values of a certain phenomenon or a certain statistical index at different times in chronological order. The time series data can be widely applied in various fields. For example, in the field of Internet of Things, a result obtained by clustering time series data can be applied in device operation state monitoring, index correlation analysis and fault diagnosis.

[0003] However, in the clustering method in the related art, the features corresponding to the time series data for clustering have little data meaning, and are only a dimension reduction representation of the time series data. Moreover, the clustering method in the related art needs to determine an initial clustering center artificially, but different initial clustering centers will result in different clustering results, thereby reducing the accuracy of the clustering results.

[0004] At present, there is no effective solution to the problem of low accuracy of the clustering method in the related art in clustering time series data. SUMMARY

[0005] The embodiments of the present application provide a data processing method and device, a storage medium and a processor to at least solve the technical problem of low accuracy of the clustering method in the related art in clustering time series data.

[0006] According to an aspect of the embodiments of the present application, a data processing method is provided, including: obtaining a plurality of time series data generated in the process of executing a target task; inputting the plurality of time series data into a target model for clustering processing to obtain a clustering result, wherein the target model at least includes an encoding module and a clustering module, the encoding module is used to extract features of the plurality of time series data, and the clustering module is used to cluster the features to obtain the clustering result; and analyzing the execution of the target task according to the clustering result.

[0007] According to an aspect of the embodiments of the present application, another data processing method is provided, including: a cloud server obtains a plurality of time series data; the cloud server processes the plurality of time series data by using a target model to obtain a clustering result, wherein the target model at least includes an encoding module and a clustering module, the encoding module is used to extract features of the plurality of time series data, and the clustering module is used to cluster the features to obtain the clustering result; and the cloud server returns the clustering result to a client.

[0008] According to another aspect of the embodiments of the present application, a data processing apparatus is also provided, comprising: a first obtaining unit, configured to obtain a plurality of time series data generated in a process of executing a target task; a first processing unit, configured to input the plurality of time series data into a target model for clustering processing to obtain a clustering result, wherein the target model at least comprises an encoding module and a clustering module, the encoding module is configured to extract features of the plurality of time series data, and the clustering module is configured to cluster the features to obtain the clustering result; and a first analyzing unit, configured to analyze an execution situation of the target task according to the clustering result.

[0009] According to another aspect of the embodiments of the present application, a storage medium is also provided, comprising a stored program, wherein the program, when executed, controls a device where the storage medium is located to perform any of the data processing methods.

[0010] According to another aspect of the embodiments of the present application, a processor is also provided, configured to execute a program, wherein the program, when executed, performs any of the data processing methods.

[0011] In the embodiments of the present application, the plurality of time series data generated in the process of executing the target task is obtained; the plurality of time series data is input into the target model for clustering processing to obtain the clustering result, wherein the target model at least comprises the encoding module and the clustering module, the encoding module is configured to extract the features of the plurality of time series data, and the clustering module is configured to cluster the features to obtain the clustering result; and the execution situation of the target task is analyzed according to the clustering result. The encoding module and the clustering module are combined into the target model, the time series data is encoded by the encoding module to obtain the features of the plurality of time series data, and the clustering module is used to cluster the features to obtain the clustering result, so that the technical effect of improving the accuracy of time series data clustering is achieved, and the technical problem of low accuracy of clustering method for clustering time series data in the related art is solved. BRIEF DESCRIPTION OF DRAWINGS

[0012] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and serve to explain the principles of the application. In the drawings:

[0013] Figure 1 is a hardware structure block diagram of a computer terminal according to an embodiment of the present application;

[0014] Figure 2 is an interactive schematic diagram of an optional computer terminal according to an embodiment of the present application;

[0015] Figure 3 is a flowchart of a data processing method provided according to an embodiment of the present application;

[0016] Figure 4 is a flow chart of an optional data processing method according to an embodiment of the present application;

[0017] Figure 5 is a flow chart of another data processing method according to an embodiment of the present application;

[0018] Figure 6 is a schematic diagram of a data processing device according to an embodiment of the present application;

[0019] Figure 7 is a schematic diagram of another data processing device according to an embodiment of the present application;

[0020] Figure 8 is a structural block diagram of an optional computer terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the personnel in the technical field better understand the present application scheme, the technical scheme in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0022] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0023] First, some of the nouns or terms that appear in the description of the embodiments of the present application are applicable to the following explanations:

[0024] Time series data: refers to the sequence data formed by arranging the values of a certain phenomenon or a certain statistical indicator at different times in chronological order.

[0025] Embodiment 1

[0026] According to the embodiments of the present application, a method embodiment of data processing is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0027] The method embodiment provided by the embodiment one of the present application can be executed in a mobile terminal, a computer terminal or similar computing device. Figure 1 is a hardware structure block diagram of a computer terminal according to the embodiments of the present application. As shown in Figure 1 , the computer terminal 10 (or mobile device 10) can include one or more processors (the processor can include but not limited to a microprocessor MCU or a programmable logic device FPGA processing device, etc.), a memory 104 for storing data, and a transmission module 106 for communication function. In addition, it can also include a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports in the bus), a network interface, a power supply and / or a camera. Those skilled in the art can understand that Figure 1 The structure shown is only schematic, which does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 can also include more or less components than those shown in Figure 1 , or have a different configuration from that shown in Figure 1 .

[0028] It should be noted that the one or more processors and / or other data processing circuits described above can be referred to as "data processing circuits" herein. The data processing circuit can be embodied in whole or in part as software, hardware, firmware or any other combination. In addition, the data processing circuit can be a single independent processing module, or any one of the other elements combined into the computer terminal 10 (or mobile device) in whole or in part. As referred to in the embodiments of the present application, the data processing circuit as a kind of processor control (for example, the selection of variable resistance terminal path connected with the interface).

[0029] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the data processing method in this embodiment of the invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the aforementioned data processing method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0030] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0031] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0032] Figure 1 The hardware structure block diagram shown can serve not only as an exemplary block diagram of the aforementioned computer terminal 10 (or mobile device), but also as an exemplary block diagram of the aforementioned server. In one optional embodiment, Figure 2 The use of the above is illustrated in a block diagram. Figure 1 The computer terminal 10 (or mobile device) shown is one embodiment of the receiving end. For example... Figure 2 As shown, computer terminal 10 (or mobile device) can be connected to one or more servers 108 via a data network connection or electronically. In one alternative embodiment, the computer terminal 10 (or mobile device) can be a mobile phone or a PC. The data network connection can be a local area network (LAN) connection, a wide area network (WAN) connection, an Internet connection, or other types of data network connection. Computer terminal 10 (or mobile device) can execute network services 110 to connect to a server (e.g., a security server) or a group of servers. Network services 110 are network-based user services such as social networks, cloud resources, email, online payments, or other online applications.

[0033] In the above operating environment, the application provides a data processing method as shown in Figure 3 Figure 3 is a flowchart of the data processing method according to an embodiment of the application.

[0034] S31, obtaining a plurality of time series data generated in the process of executing a target task.

[0035] Specifically, the target task can be a sensor monitoring task in the Internet of Things field. When a certain target task is being executed, multiple data can be generated, and each data can be accumulated according to the progress of the task, so as to obtain time series data corresponding to each data. For example, in the process of performing a certain experiment, a plurality of sensors can detect monitoring data including vibration intensity, temperature, humidity, etc., and as the experiment progresses, time series data of each experimental data corresponding to the plurality of experimental data of vibration intensity, temperature, humidity, etc. can be generated.

[0036] S32, inputting the plurality of time series data into a target model for clustering processing to obtain a clustering result, wherein the target model at least includes an encoding module and a clustering module, the encoding module is used to extract features of the plurality of time series data, and the clustering module is used to cluster the features to obtain the clustering result.

[0037] Specifically, the target model can be a trained machine learning model, wherein the machine learning model can be divided into two parts in the process of data processing. The first part is to obtain feature information of the time series data by encoding the time series data, wherein the feature information can include positive sequence features and inverse sequence features, and two feature information in the latent space: mean and variance are obtained by the positive sequence features and the inverse sequence features, so as to obtain a plurality of feature information. The second part is to obtain the clustering result by clustering the feature information of the plurality of time series data through the clustering module after obtaining the feature information corresponding to the plurality of time series data.

[0038] S33, analyzing the execution of the target task according to the clustering result.

[0039] Specifically, after obtaining the clustering result corresponding to the plurality of time series data, the execution of the target task is analyzed according to the clustering result.

[0040] Optionally, in the data processing method of the embodiment of the application, analyzing the execution of the target task according to the clustering result includes: in the case that the target task appears an execution failure, determining a fault monitoring quantity sequence data; obtaining data belonging to the same class as the fault monitoring quantity sequence data, and determining the obtained data as fault associated data.

[0041] ​For example, in the process of performing an experiment, monitoring data such as vibration intensity, temperature, humidity, etc. can be detected by various sensors, and as the experiment progresses, time series data of each experimental data corresponding to various experimental data such as vibration intensity, temperature, humidity, etc. can be generated, and after clustering processing, temperature and humidity are classified into a category, that is, it can be considered that there is a correlation between the two characteristics of temperature and humidity. When temperature is abnormal, humidity can be detected according to the correlation to determine whether the temperature abnormality is caused by humidity abnormality.

[0042] In the embodiment of the present application, by acquiring a plurality of time series data generated in the process of performing a target task; inputting the plurality of time series data into a target model for clustering processing to obtain a clustering result, wherein the target model at least includes an encoding module and a clustering module, the encoding module is used to extract features of the plurality of time series data, and the clustering module is used to cluster the features to obtain the clustering result; analyzing the execution of the target task according to the clustering result, combining the encoding module and the clustering module into the target model, and encoding the time series data according to the encoding module to obtain the features of the plurality of time series data, and using the clustering module to cluster the features to obtain the clustering result, thereby realizing the technical effect of improving the accuracy of time series data clustering, and further solving the technical problem of low accuracy of clustering method for clustering time series data in related technologies.

[0043] In order to make the clustering result of the target model more accurate, optionally, in the data processing method of the embodiment of the present application, before the plurality of time series data is input into the target model for clustering processing to obtain the clustering result, the method further includes: acquiring a plurality of sample time series data; training a preset encoding module through the plurality of sample time series data, and determining a first loss value corresponding to the process of training the preset encoding module, wherein the preset encoding module processes the plurality of sample time series data to obtain a plurality of sample features when training the preset encoding module; acquiring a plurality of sample features generated in the process of training the preset encoding module, training a preset clustering module through the plurality of sample features, and determining a second loss value corresponding to the process of training the preset clustering module; determining a target loss value according to the first loss value and the second loss value, and acquiring the encoding module and the clustering module corresponding to the case that the target loss value is less than a loss threshold; combining the acquired encoding module and the clustering module into the target model.

[0044] Specifically, before clustering processing is performed using the target model, the target model needs to be trained to ensure that the target loss value is less than a loss threshold, so that the clustering result is more accurate. The plurality of sample time series data can be time series data of known categories, and each sample time series data and its corresponding category are input into the target model. The preset encoding module in the target model is trained by the plurality of sample time series data, and encoded sample feature information is obtained.

[0045] On the one hand, the first loss value in the process of training the preset encoding module can be determined by decoding the sample time series data that has been encoded and comparing the decoded data with the corresponding sample time series data after obtaining the decoded data. On the other hand, the clustering module is trained by the sample feature information, and the category prediction result corresponding to each sample time series data is obtained by the clustering module. The category prediction result is compared with the category corresponding to the sample time series data, and the second loss value is determined according to the comparison result.

[0046] Further, the target loss value can be determined according to the first loss value and the second loss value. For example, the target loss value can be determined by the formula Loss3=Loss1+w*Loss2, where Loss3 is the target loss value, Loss1 is the first loss value, Loss2 is the second loss value, and w is a preset weight. In the case where the target loss value is greater than the loss threshold, the training parameters of the target model need to be adjusted until the target loss value is less than the loss threshold, so as to improve the accuracy of the clustering result.

[0047] Optionally, in the data processing method of the embodiment of the present application, the type of the encoding module is at least one of the following: sparse autoencoder, variational autoencoder.

[0048] Specifically, the target encoding module can be a sparse autoencoder, wherein the sparse autoencoder is basically the same as the autoencoder structure, and the difference lies in that the hidden layer vector is sparse, that is, as many zero elements as possible, which can reduce the risk of model overfitting. However, the feature sequence extracted by the sparse autoencoder does not have much data meaning, and is only a kind of dimensionality reduction representation of the time series data.

[0049] Further, in order to better encode, the target encoding model can also use a variational autoencoder. The variational autoencoder is an improvement of the autoencoder, which returns multiple probability models in the latent space and uses them to describe the time series data as features. It should be noted that the variational autoencoder returns a distribution in the latent space rather than a single point, and a regularization term for the returned distribution can be added to the loss function to solve the problem of irregularity in the latent space to ensure better organization of the latent space. The type of encoding module in the embodiment is not limited to the above two types of encoders, and other types of encoders can also be used.

[0050] In order to enhance the effect of training the target model and make the clustering result obtained by the trained target model more accurate, optionally, in the data processing method of the embodiment of the application, the preset encoding module is trained by the plurality of sample time series data, and the corresponding first loss value in the process of training the preset encoding module is determined by: inputting the plurality of sample time series data into the preset encoding module to obtain a plurality of sample features of the plurality of sample time series data; restoring the plurality of sample features by the preset decoding module to obtain restored plurality of time series data; and determining the first loss value according to the difference between the plurality of sample time series data and the restored plurality of time series data.

[0051] Specifically, after obtaining the sample features corresponding to the plurality of sample time series data by the preset encoding module, the sampling layer in the preset decoding module can be used to sample the normal distribution, wherein the normal distribution is represented by the mean and variance in the sample features, and the data is mapped to the dimension of the original data through three one-dimensional convolution layers and one oversampling layer to obtain the restored time series data. Further, the restored time series data is compared with the sample time series data, and the first loss value is determined according to the comparison result.

[0052] In order to obtain more accurate feature information of the time series data, optionally, in the data processing method of the embodiment of the application, when the type of the encoding module is a variational autoencoder, the encoding module comprises: a convolution layer for extracting first features of the time series data; a pooling layer connected with the convolution layer for reducing the dimension of the first features; a first long short-term memory model connected with the pooling layer for extracting positive sequence features in the first features; a second long short-term memory model connected with the pooling layer for extracting inverse sequence features in the first features; a first full connection layer connected with the first long short-term memory model and the second long short-term memory model for determining first distribution parameters according to the positive sequence features and the positive sequence features; a second full connection layer connected with the first long short-term memory model and the second long short-term memory model for determining second distribution parameters according to the positive sequence features and the inverse sequence features; and the positive sequence features, the inverse sequence features, the first distribution parameters and the second distribution parameters are determined as the output of the encoder.

[0053] Specifically, the convolution layer can be an activation function layer obtained by one-dimensional convolution in the convolutional neural network plus an activation function, used to extract the first feature of the input time series data and send the first feature to the pooling layer. The pooling layer usually acts on each input feature respectively and reduces its size, thereby reducing the dimension of the first feature of the time series data.

[0054] After obtaining the reduced first feature, the first long short-term memory model and the second long short-term memory model can constitute a bidirectional LSTM layer (Long Short-Term Memory, long short-term memory network), which extracts the forward sequence feature and the reverse sequence feature of the reduced first feature through the bidirectional LSTM layer, determines the first distribution parameter of the time series data according to the forward sequence feature and the reverse sequence feature in the first full connection layer, wherein the first distribution parameter can be the mean, and determines the second distribution parameter of the time series data according to the forward sequence feature and the reverse sequence feature in the second full connection layer, wherein the second distribution parameter can be the variance, and takes the variance, the mean, the forward sequence feature and the reverse sequence feature as the feature information of the time series data.

[0055] Optionally, in the data processing method of the embodiment of the application, the clustering algorithm used by the clustering module is at least one of the following: K-means clustering algorithm, hierarchical clustering algorithm.

[0056] Specifically, the clustering module can use the K-means clustering algorithm, wherein the K-means clustering algorithm selects K points as initial clustering cluster centers, respectively calculates the distance of each sample point to the K cluster centers, finds the nearest cluster center to the point, and attributes it to the corresponding cluster, and after all points are attributed to the cluster, the center of gravity (average distance center) of each cluster is recalculated and is defined as the new "cluster center". The above steps are iterated until the preset condition is met to complete clustering and obtain the clustering result. However, the K-means clustering algorithm needs to determine the initial clustering center artificially, but selecting different initial clustering centers will obtain different clustering results, thereby reducing the accuracy of the clustering result.

[0057] Further, in order to better cluster, the clustering module can also use the hierarchical clustering algorithm.

[0058] To improve the accuracy of the polymerization result, optionally, in the data processing method of the embodiment of the application, in the case where the clustering algorithm used by the clustering module is a hierarchical clustering algorithm, the hierarchical clustering algorithm is used to cluster the features, including: determining each feature as a class to obtain a plurality of first-level classes; calculating the minimum distance between each two first-level classes, and merging the two first-level classes with the shortest minimum distance to obtain a plurality of second-level classes; calculating the minimum distance between each two second-level classes, and merging the two first-level classes with the shortest minimum distance until the plurality of first-level classes are merged into one class.

[0059] Specifically, in the case where the clustering algorithm is a hierarchical clustering algorithm, each feature corresponding to each time series data in the clustering module can be taken as a class to obtain a plurality of first-level classes, the distance between each two features is calculated, the minimum distance feature corresponding to each feature is obtained, each feature is merged with the corresponding minimum distance feature, and is classified into a class to obtain a plurality of second-level classes.

[0060] Further, after obtaining the plurality of second-level classes, the distance between each two classes is calculated again according to the coordinates of each class, the minimum distance class corresponding to each class is obtained, each class is merged with the corresponding minimum distance class, and is classified into a class to obtain a plurality of third-level classes. The above process is repeated until all classes are classified into one class, thereby obtaining the relationship between the classes of each feature. Thus, different time series data can be classified according to a preset level.

[0061] Compared with the K-means clustering algorithm, the hierarchical clustering algorithm has the characteristics of not needing to predefine the number of clusters, can obviously display the hierarchical relationship between the classes, can be clustered into any shape, and is not affected by a single outlier, thereby achieving the effect of improving the clustering accuracy. The clustering method that can be used by the clustering module in the embodiment is not limited to the above two clustering algorithms, and other types of clustering algorithms can also be used.

[0062] To improve the accuracy of time series data clustering, the distance calculation method between different coordinates in the clustering algorithm needs to be set. Optionally, in the data processing method of the embodiment of the application, the clustering algorithm clusters by the distance between each two time series data, wherein the distance between each two time series data is determined by the following formula:

[0063] CID(x, y) = ED(x, y) · CF(x, y);

[0064] Wherein, x and y are the features of two time series data, ED(x, y) is the Euclidean distance between x and y, and CF(x, y) is the time series complexity factor.

[0065] Wherein, N is the number of time series data, and t is the serial number of time series data;

[0066] wherein, CF(x, y) = max(CE(x), CE(y)) / min(CE(x), CE(y)), CE(x) is a time series complexity factor corresponding to x, and CE(y) is a time series complexity factor corresponding to y;

[0067] wherein,

[0068] In addition, in order to reduce the amount of calculation, the distance between each two time series data can also be directly determined by calculating the Euclidean distance.

[0069] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the present application.

[0070] Through the description of the above embodiments, those skilled in the art can clearly understand that the data processing method according to the above embodiments can be realized by means of software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, or network device, etc.) execute the method described in each embodiment of the present application.

[0071] Embodiment 2

[0072] According to the embodiments of the present application, an optional data processing method is also provided, Figure 4 is a flow chart of an optional data processing method provided according to the first embodiment of the present application, as Figure 4 shown:

[0073] After the target model receives the time series data, the first feature of the input time series data is extracted in the activation function layer obtained by the one-dimensional convolution in the encoding module and the activation function, and the first feature is sent to the pooling layer. The pooling layer usually acts on each input feature respectively and reduces its size, thereby reducing the dimension of the first feature of the time series data.

[0074] After obtaining the first feature after dimension reduction, the first long short-term memory model and the second long short-term memory model can constitute a bidirectional LSTM layer (Long Short-Term Memory, long short-term memory network), the bidirectional LSTM layer extracts the forward sequence feature and the reverse sequence feature of the first feature with reduced latitude, and the first full connection layer determines the first distribution parameter of the time sequence data according to the forward sequence feature and the reverse sequence feature, wherein the first distribution parameter can be the mean, and the second full connection layer determines the second distribution parameter of the time sequence data according to the forward sequence feature and the reverse sequence feature, wherein the second distribution parameter can be the variance, and the variance, the mean, the forward sequence feature and the reverse sequence feature are used as the feature information of the time sequence data.

[0075] Further, after obtaining the variance and the mean, the two full connection layers can sample the normal distribution through the sampling layer in the preset decoding module, wherein the normal distribution is represented by the mean and the variance in the sample feature, and the data is mapped to the dimension of the original data through three one-dimensional convolution layers and one oversampling layer, to obtain the restored time sequence data. Further, the restored time sequence data is compared with the sample time sequence data, and the first loss value is determined according to the comparison result.

[0076] It should be noted that after the encoding module obtains the feature information of the time sequence data, the feature information is sent to the clustering module, and the multiple feature information is clustered by the clustering module, so as to obtain the clustering result corresponding to the time sequence data, and after obtaining the clustering result corresponding to the multiple time sequence data, the execution of the target task is analyzed according to the clustering result.

[0077] Embodiment 3

[0078] According to the embodiments of the present application, a data processing method is also provided. Figure 5 The flowchart of the data processing method provided by the embodiment of the present application is shown in FIG. 1, which includes the following steps: Figure 5

[0079] S51, the cloud server obtains multiple time sequence data.

[0080] Specifically, when a certain target task is being executed, multiple data can be generated, and each data can be accumulated according to the progress of the task, so that the cloud server can obtain the time sequence data corresponding to each data. The target task can be a sensor monitoring task in the field of Internet of Things. For example, in the process of performing a certain experiment, multiple sensors can detect monitoring data including vibration intensity, temperature, humidity, etc., and as the experiment progresses, multiple experimental data corresponding to each experimental data of vibration intensity, temperature, humidity, etc. can be generated. ​

[0081] S52, the cloud server processes the plurality of time series data by using a target model to obtain a clustering result, wherein the target model at least includes an encoding module and a clustering module, the encoding module is used to extract features of the plurality of time series data, and the clustering module is used to cluster the features to obtain the clustering result.

[0082] Specifically, the target model used by the cloud server can be a trained machine learning model, wherein the machine learning model can be divided into two parts in the process of data processing. The first part encodes the time series data to obtain feature information of the time series data, wherein the feature information can include positive sequence features and inverse sequence features, and two feature information in the latent space: mean and variance are obtained through the positive sequence features and the inverse sequence features, so as to obtain a plurality of feature information. The second part, after obtaining the feature information corresponding to the plurality of time series data, the clustering module is used to cluster the feature information of the plurality of time series data, so as to obtain the clustering result.

[0083] S53, the cloud server returns the clustering result to the client.

[0084] Specifically, after obtaining the clustering result corresponding to a certain time series data, the cloud server can return the clustering result to the client, and the client can analyze the target task execution according to the rules corresponding to the determined category.

[0085] For example, in the process of performing an experiment, a plurality of sensors can be used to detect monitoring data including vibration intensity, temperature, humidity, etc., and as the experiment proceeds, time series data corresponding to each experimental data of the experimental data of vibration intensity, temperature, humidity, etc. can be generated, and after clustering processing, temperature and humidity are classified into one category, that is, it can be considered that there is a correlation between the two characteristics of temperature and humidity. When the temperature is abnormal, the humidity can be detected according to the correlation to determine whether the temperature abnormality is caused by the humidity abnormality.

[0086] Embodiment 4

[0087] According to the embodiment of the application, a data processing device for implementing the above-mentioned data processing method is also provided, as shown in the figure, the device comprises: Figure 6

[0088] The first acquisition unit 61 is used to acquire a plurality of time series data generated in the process of executing a target task.

[0089] ​The first processing unit 62 is configured to input the plurality of time series data into a target model to perform clustering processing, and obtain a clustering result, wherein the target model at least includes an encoding module and a clustering module, the encoding module is configured to extract features of the plurality of time series data, and the clustering module is configured to perform clustering on the features to obtain the clustering result.

[0090] The first analysis unit 63 is configured to analyze an execution situation of a target task according to the clustering result.

[0091] It should be noted that the first obtaining unit 61, the first processing unit 62 and the first analysis unit 63 correspond to the step S31, the step S32 and the step S33 in the embodiment 1, and the two modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in the above embodiment 1. It should be noted that the above modules can run in the computer terminal 10 provided in the embodiment 1 as a part of the device.

[0092] Optionally, in the data processing device provided by the embodiment of the present application, the device further includes: a second obtaining unit configured to obtain a plurality of sample time series data; a first determining unit configured to train a preset encoding module by using the plurality of sample time series data, and determine a first loss value corresponding to a process of training the preset encoding module, wherein the preset encoding module processes the plurality of sample time series data to obtain a plurality of sample features when training the preset encoding module; a third obtaining unit configured to obtain the plurality of sample features generated in the process of training the preset encoding module, train a preset clustering module by using the plurality of sample features, and determine a second loss value corresponding to a process of training the preset clustering module; a second determining unit configured to determine a target loss value according to the first loss value and the second loss value, and obtain an encoding module and a clustering module corresponding to a case that the target loss value is less than a loss threshold; and a combination unit configured to combine the obtained encoding module and the clustering module into a target model.

[0093] Optionally, in the data processing device provided by the embodiment of the present application, the first determining unit includes: an input module configured to input the plurality of sample time series data into the preset encoding module to process a plurality of sample features of the plurality of sample time series data; a restoration module configured to restore the plurality of sample features by using a preset decoding module to obtain a plurality of restored time series data; and a first determining module configured to determine the first loss value according to a difference between the plurality of sample time series data and the plurality of restored time series data.

[0094] Optionally, in the data processing device provided by the embodiment of the present application, the type of the encoding module is at least one of the following: a sparse autoencoder and a variational autoencoder.

[0095] Optionally, in the data processing apparatus according to the embodiment of the present application, when the type of the encoding module is a variational autoencoder, the encoding module comprises: a convolutional layer configured to extract first features of the time series data; a pooling layer connected to the convolutional layer and configured to reduce the dimension of the first features; a first long short-term memory model connected to the pooling layer and configured to extract positive sequence features in the first features; a second long short-term memory model connected to the pooling layer and configured to extract negative sequence features in the first features; a first fully connected layer connected to the first long short-term memory model and the second long short-term memory model and configured to determine first distribution parameters according to the positive sequence features and the positive sequence features; a second fully connected layer connected to the first long short-term memory model and the second long short-term memory model and configured to determine second distribution parameters according to the positive sequence features and the negative sequence features; and the positive sequence features, the negative sequence features, the first distribution parameters and the second distribution parameters are determined as the output of the encoder.

[0096] Optionally, in the data processing apparatus according to the embodiment of the present application, the clustering algorithm used by the clustering module is at least one of the following: a K-means clustering algorithm, a hierarchical clustering algorithm.

[0097] Optionally, in the data processing apparatus according to the embodiment of the present application, the clustering algorithm clusters by the distance between each two time series data, wherein the distance between each two time series data is determined by the following formula:

[0098] CID(x,y) = ED(x,y) · CF(x,y);

[0099] wherein x and y are features of two time series data, ED(x,y) is the Euclidean distance between x and y, and CF(x,y) is a time series complexity factor;

[0100] wherein, N is the number of time series data, and t is the sequence number of the time series data.

[0101] wherein, CF(x,y) = max(CE(x), CE(y)) / min(CE(x), CE(y)), CE(x) is a time series complexity factor corresponding to x, and CE(y) is a time series complexity factor corresponding to y.

[0102] wherein,

[0103] Optionally, in the data processing apparatus according to the embodiments of the present application, when the clustering algorithm used by the clustering module is a hierarchical clustering algorithm, the hierarchical clustering algorithm is used to cluster the features, including: a second determination module configured to determine each feature as a class to obtain a plurality of first-level classes; a first calculation module configured to calculate the minimum distance between each two first-level classes, and combine the two first-level classes with the shortest minimum distance to obtain a plurality of second-level classes; and a second calculation module configured to calculate the minimum distance between each two second-level classes, and combine the two first-level classes with the shortest minimum distance until the plurality of first-level classes are combined into one class.

[0104] Optionally, in the data processing apparatus according to the embodiments of the present application, the first analysis unit 63 includes: a third determination module configured to determine the fault monitoring quantity sequence data when the target task occurs a fault; and a fourth determination module configured to obtain data belonging to the same class as the fault monitoring quantity sequence data, and determine the obtained data as the fault associated data.

[0105] Embodiment 5

[0106] According to the embodiments of the present application, a data processing apparatus for implementing the above data processing method is also provided, as shown in Figure 7 The apparatus includes:

[0107] A fourth obtaining unit 71 is configured to obtain a plurality of time series data through a cloud server.

[0108] A second processing unit 72 is configured to process the plurality of time series data through the cloud server using a target model to obtain a clustering result, wherein the target model at least includes an encoding module and a clustering module, the encoding module is configured to extract features of the plurality of time series data, and the clustering module is configured to cluster the features to obtain the clustering result.

[0109] A second analysis unit 73 is configured to return the clustering result to a client through the cloud server.

[0110] It should be noted that the fourth obtaining unit 71, the second processing unit 72 and the second analysis unit 73 correspond to steps S51, S52 and S53 in Embodiment 3, and the two modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 3. It should be noted that the above modules as part of the apparatus can run in the computer terminal 10 provided in Embodiment 1.

[0111] Embodiment 6

[0112] The embodiment of the present application can provide a computer terminal, which can be any one of computer terminal devices in a computer terminal group. Alternatively, in the embodiment, the computer terminal can be replaced by a mobile terminal or other terminal device.

[0113] Alternatively, in the embodiment, the computer terminal can be located in at least one of a plurality of network devices in a computer network.

[0114] In the embodiment, the computer terminal can execute program codes of the following steps in the data processing method: obtaining a plurality of time series data generated in the process of executing a target task; inputting the plurality of time series data into a target model for clustering processing to obtain a clustering result, wherein the target model at least includes an encoding module and a clustering module, the encoding module is used to extract features of the plurality of time series data, and the clustering module is used to cluster the features to obtain the clustering result; and analyzing the execution of the target task according to the clustering result.

[0115] Alternatively, Figure 8 is a structural block diagram of an optional computer terminal provided by the embodiment of the present application. As shown in Figure 8 the computer terminal 10 can include one or more (only one is shown in the figure) processors, a memory, and a transmission device.

[0116] The memory can be used to store software programs and modules, such as program instructions / modules corresponding to the data processing method and device in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implements the above-mentioned data processing method. The memory can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory can further include a memory remotely arranged with respect to the processor, which can be connected to the terminal A through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0117] The processor can call information and application programs stored in the memory through the transmission device to execute the following steps: obtaining a plurality of time series data generated in the process of executing a target task; inputting the plurality of time series data into a target model for clustering processing to obtain a clustering result, wherein the target model at least includes an encoding module and a clustering module, the encoding module is used to extract features of the plurality of time series data, and the clustering module is used to cluster the features to obtain the clustering result; and analyzing the execution of the target task according to the clustering result.

[0118] The embodiment of the present application provides a data processing scheme. The scheme comprises the following steps: obtaining a plurality of time series data generated in the process of executing a target task; inputting the plurality of time series data into a target model for clustering processing to obtain a clustering result, wherein the target model at least comprises an encoding module and a clustering module, the encoding module is used for extracting features of the plurality of time series data, and the clustering module is used for clustering the features to obtain the clustering result; and analyzing the execution of the target task according to the clustering result. The technical effect of improving the accuracy of time series data clustering is realized, and the technical problem of low accuracy of the clustering method for clustering time series data in the related art is solved.

[0119] Those skilled in the art can understand that, Figure 8 The structure shown is only schematic, and the computer terminal 10 can also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a mobile Internet device (MID), a PAD, or the like. Figure 8 The above electronic device is not limited in structure. For example, the computer terminal 10 can further include more or less components (such as a network interface, a display device, etc.) than those shown in the figure, or have a different configuration from that shown in the figure. Figure 8 The above electronic device is not limited in structure. For example, the computer terminal 10 can further include more or less components (such as a network interface, a display device, etc.) than those shown in the figure, or have a different configuration from that shown in the figure. Figure 8 The above electronic device is not limited in structure. For example, the computer terminal 10 can further include more or less components (such as a network interface, a display device, etc.) than those shown in the figure, or have a different configuration from that shown in the figure.

[0120] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by programs instructing the related hardware of the terminal device, and the programs can be stored in a computer readable storage medium, which can include a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0121] The embodiment of the present application further provides a storage medium. Optionally, in the embodiment, the above storage medium can be used to save the program code executed by the data processing method provided in the first embodiment.

[0122] Optionally, in the embodiment, the above storage medium can be located in any one of the computer terminals in a computer terminal group in a computer network, or in any one of the mobile terminals in a mobile terminal group.

[0123] Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: obtaining a plurality of time series data generated in the process of performing the target task; inputting the plurality of time series data into a target model for clustering processing to obtain a clustering result, wherein the target model at least includes an encoding module and a clustering module, the encoding module is used to extract features of the plurality of time series data, and the clustering module is used to cluster the features to obtain the clustering result; and analyzing the execution of the target task according to the clustering result.

[0124] The above-mentioned serial numbers of the embodiments of the application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0125] In the above-mentioned embodiments of the application, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0126] In several embodiments provided in the present application, it should be understood that the disclosed technical contents can be implemented by other ways. Among them, the above-mentioned device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division way, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or modules shown or discussed can be indirect coupling or communication connection through some interfaces, units or modules, which can be electrical or other forms.

[0127] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment.

[0128] In addition, each functional unit in each embodiment of the application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0129] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to make a computer device (which can be a personal computer, a server or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0130] The above is only the preferred embodiment of the present application, and it should be pointed out that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. A data processing method, characterized by, The method comprises: obtaining a plurality of time series data generated in the process of executing a target task; inputting the plurality of time series data into a target model for clustering processing to obtain a clustering result, wherein the target model at least comprises an encoding module and a clustering module, the encoding module is used for extracting features of the plurality of time series data, and the clustering module is used for clustering the features to obtain the clustering result; analyzing the execution of the target task according to the clustering result, comprising: in the case that the target task appears to have an execution failure, determining a fault monitoring quantity sequence data; obtaining data belonging to the same class as the fault monitoring quantity sequence data, and determining the obtained data as fault associated data; detecting the fault associated data to determine whether the target task appears to have an execution failure due to the abnormality of the fault associated data; in the case that the clustering algorithm used in the clustering module is a hierarchical clustering algorithm, the hierarchical clustering algorithm is used to cluster the features, comprising: taking the features corresponding to each time series data as a class to obtain a plurality of first-level classes, calculating the distance between each two features, obtaining the minimum distance feature corresponding to each feature, merging each feature and the corresponding minimum distance feature, and classifying them into a class to obtain a plurality of second-level classes, calculating the distance between each two classes again according to the coordinates of each class, obtaining the minimum distance class corresponding to each class, merging each class and the corresponding minimum distance class, and classifying them into a class to obtain a plurality of third-level classes, until all classes are classified into one class.

2. The data processing method according to claim 1, characterized in that, Before inputting the plurality of time series data into the target model for clustering processing to obtain the clustering result, the method further comprises: obtaining a plurality of sample time series data; training a preset encoding module through the plurality of sample time series data, and determining a first loss value corresponding to the process of training the preset encoding module, wherein the preset encoding module processes the plurality of sample time series data to obtain a plurality of sample features when training the preset encoding module; obtaining the plurality of sample features generated in the process of training the preset encoding module, training a preset clustering module through the plurality of sample features, and determining a second loss value corresponding to the process of training the preset clustering module; determining a target loss value according to the first loss value and the second loss value, and obtaining a corresponding encoding module and clustering module in the case that the target loss value is less than a loss threshold value; combining the obtained encoding module and clustering module into the target model.

3. The data processing method according to claim 2, characterized in that, Training a preset encoding module through the plurality of sample time series data, and determining a first loss value corresponding to the process of training the preset encoding module comprises: inputting the plurality of sample time series data into the preset encoding module to process and obtain the plurality of sample features of the plurality of sample time series data; restoring the plurality of sample features through a preset decoding module to obtain restored plurality of time series data; determining the first loss value according to the difference between the plurality of sample time series data and the restored plurality of time series data.

4. The data processing method of claim 1, wherein, The type of the encoding module is at least one of a sparse autoencoder and a variational autoencoder.

5. The data processing method according to claim 4, characterized in that, In a case where the type of the encoding module is the variational autoencoder, the encoding module comprises: a convolutional layer configured to extract first features of the time series data; a pooling layer connected to the convolutional layer and configured to reduce dimensions of the first features; a first long short-term memory model connected to the pooling layer and configured to extract positive sequence features in the first features; a second long short-term memory model connected to the pooling layer and configured to extract inverse sequence features in the first features; a first fully connected layer connected to the first long short-term memory model and the second long short-term memory model and configured to determine first distribution parameters according to the positive sequence features and the positive sequence features; a second fully connected layer connected to the first long short-term memory model and the second long short-term memory model and configured to determine second distribution parameters according to the positive sequence features and the inverse sequence features; the positive sequence features, the inverse sequence features, the first distribution parameters, and the second distribution parameters are determined as outputs of the encoder.

6. The data processing method of claim 1, wherein, The clustering algorithm used by the clustering module is at least one of a K-means clustering algorithm and a hierarchical clustering algorithm.

7. The data processing method according to claim 6, characterized in that, The clustering algorithm clusters by distances between every two time series data, wherein the distance between every two time series data is determined by the following formula: ； where x and y are features of two time series data, is the Euclidean distance between x and y, is the time complexity factor; wherein, N is the number of time series data, and t is the serial number of time series data. wherein , is a timing complexity factor corresponding to x, is a timing complexity factor corresponding to y; wherein , .

8. A data processing method, characterized by, comprises: a cloud server configured to obtain a plurality of time series data, wherein the plurality of time series data are generated in a process of executing a target task; the cloud server is configured to process the plurality of time series data by using a target model to obtain a clustering result, wherein the target model at least comprises an encoding module and a clustering module, the encoding module is configured to extract features of the plurality of time series data, and the clustering module is configured to cluster the features to obtain the clustering result, in a case where a clustering algorithm used by the clustering module is a hierarchical clustering algorithm, the hierarchical clustering algorithm is used to cluster the features, including: taking features corresponding to each time series data as a class to obtain a plurality of first-level classes, calculating distances between every two features, obtaining a minimum distance feature corresponding to each feature, merging each feature and the corresponding minimum distance feature, and classifying them into a class to obtain a plurality of second-level classes, calculating distances between every two classes again according to coordinates of each class, obtaining a minimum distance class corresponding to each class, merging each class and the corresponding minimum distance class, and classifying them into a class to obtain a plurality of third-level classes, and the process is repeated until all classes are classified into one class; the cloud server is configured to return the clustering result to a client, wherein the clustering result is used to analyze an execution situation of the target task, in a case where the target task has an execution fault, a fault monitoring quantity sequence data is determined, data belonging to the same class as the fault monitoring quantity sequence data is obtained, and the obtained data is determined as fault associated data, and the fault associated data is detected to determine whether the target task has an execution fault due to an abnormality of the fault associated data.

9. A data processing apparatus, characterized by, comprises: The first obtaining unit is configured to obtain a plurality of time series data generated in a process of executing a target task. The first processing unit is configured to input the plurality of time series data into a target model for clustering processing to obtain a clustering result, wherein the target model at least includes an encoding module and a clustering module, the encoding module is configured to extract features of the plurality of time series data, and the clustering module is configured to cluster the features to obtain the clustering result. The first analysis unit is configured to analyze an execution condition of the target task according to the clustering result, and the first analysis unit includes: a third determining module configured to determine a fault monitoring quantity sequence data in a case where the target task has an execution fault; and a fourth determining module configured to obtain data belonging to a same class as the fault monitoring quantity sequence data, and determine the obtained data as fault associated data, and determine whether the target task has the execution fault due to an abnormality of the fault associated data. The first processing unit is further configured to: in a case where a clustering algorithm used by the clustering module is a hierarchical clustering algorithm, cluster the features by using the hierarchical clustering algorithm, including: taking the features corresponding to each time series data as a class to obtain a plurality of first-level classes, calculating distances between each two features, obtaining a minimum distance feature corresponding to each feature, merging each feature and the corresponding minimum distance feature, and classifying them into a class to obtain a plurality of second-level classes, calculating distances between each two classes according to coordinates of each class, obtaining a minimum distance class corresponding to each class, merging each class and the corresponding minimum distance class, and classifying them into a class to obtain a plurality of third-level classes, and repeating the above steps until all classes are classified into one class.

10. A storage medium, characterized by The storage medium includes a stored program, wherein the program controls a device where the storage medium is located to execute the data processing method in any one of claims 1 to 7, or the data processing method in claim 8 when the program is running.

11. A processor, comprising: The processor is configured to run a program, wherein the program executes the data processing method in any one of claims 1 to 7, or the data processing method in claim 8 when the program is running.

Images