Data processing method and computer device

Abstract

Provided in the present application are a data processing method and a computer device, which can be used for interpolating missing values of time series data. The method comprises: fusing n types of data to be processed on m nodes spatially and temporally, so as to obtain a fused representation code; on this basis, constructing a first dynamic spatial relationship graph between the m nodes; and finally, on the basis of the constructed first dynamic spatial relationship graph, using a learned denoising function to implement interpolation of missing values of target data. The n types of data used in the present application are actually acquired data, and there is no constraint on distribution of said data, thus better representing actual application scenarios; the first dynamic spatial relationship graph is data driven, such that implicit spatial correlation (which can reflect a complex dependency relationship of a data spatio-temporal sequence between nodes) between nodes can be learned on this basis; the first dynamic spatial relationship graph can change along with changes of input data, thus overcoming limitations of a static adjacency matrix in traditional methods.

Description

Data processing method and computer device

[0001] The present application claims priority to the Chinese Patent Application No. 202411333709.3, filed on September 23, 2024, and entitled "A data processing method and computer device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of data processing, and in particular, to a data processing method and computer device. BACKGROUND

[0003] Many businesses need data to maintain integrity in order to better visualize and analyze, however, in real scenarios, due to acquisition capacity or network transmission, etc., time series data often has missing conditions. For example, the fields of traffic management, environmental detection, weather forecasting, power load prediction, etc. involve a large amount of time series data, and data missing conditions are common.

[0004] Taking traffic management as an example, due to device failure, network transmission failure, road reconstruction during, etc., the data information of traffic flow, speed, etc. of the highway / urban road has a high missing rate, which seriously affects the traffic state judgment and the effect of downstream algorithms. Specifically, during the reconstruction / expansion of the highway, part of the sensor data is missing for a long time, and only the flow sensor set on the individual gantry provides data, which is difficult to meet the operation supervision demand; and for other non-reconstruction / expansion roads, due to device failure and network transmission problems, random data missing often occurs. In order to solve the problems of random missing, long-term missing, and complete missing of data of traffic flow, speed, etc. at part of the nodes, the current methods generally include the following: method 1, a time-space matrix decomposition-based interpolation method, which represents the traffic flow data as a time-space matrix, and uses matrix decomposition technology to decompose the matrix into the sum of a low-rank matrix and a sparse matrix, wherein the low-rank matrix represents the global trend of the data, and the sparse matrix represents the local fluctuation of the data. By filling in the missing values in the sparse matrix, the interpolation of traffic flow is realized. Method 2, a graph convolutional neural network (GCN)-based interpolation method, which models the traffic network as a graph structure, uses graph convolutional neural network technology to learn the time-space dependency between nodes, and according to the learned dependency to interpolate the missing values; method 3, a generative adversarial network (GAN)-based interpolation method, which uses GAN to learn the latent distribution of the data, and according to the learned distribution to generate missing values.

[0005] The above interpolation methods have the following disadvantages: method 1 usually assumes that the traffic flow data obeys a certain specific distribution, such as Gaussian distribution, but the actual traffic flow data often presents complex characteristics such as nonlinearity and non-stationarity, which are difficult to describe by a simple distribution. Moreover, this method usually assumes that the spatial correlation is static and uniformly distributed, but the spatial correlation of actual traffic flow is often dynamically changing, for example, traffic congestion during rush hours will affect the traffic flow of surrounding road segments. Moreover, this method can only process single modal (i.e., single type) traffic data, and it is difficult to fuse multi-source heterogeneous information, that is, it is difficult to effectively fuse multiple types of traffic data; method 2 usually relies on local domain information for prediction, and for long-term missing data, the interpolation accuracy will decrease due to the lack of sufficient domain information. Moreover, this method usually uses a pre-defined topological adjacency matrix to describe the spatial dependence, but the spatial correlation of actual traffic flow may be affected by various factors such as road grade and traffic signals, and it is difficult to accurately describe the complex dependence between nodes (especially between nodes beyond geographical adjacency) in the data spatio-temporal sequence using a simple topological adjacency matrix, so this method is difficult to capture the complex dependence between nodes. The model training process of method 3 is unstable, and problems such as mode collapse and gradient disappearance may occur, which makes it difficult for the model to converge to an optimal solution, and most existing GAN models are designed for single modal data, which is difficult to handle multi-source heterogeneous information. SUMMARY

[0006] The present application provides a data processing method and a computer device, which are used for aligning and fusing the actual collected multi-source heterogeneous data (i.e., n types of data to be processed) in space and time, mapping the data of different modalities to the same feature space to obtain a fusion representation code, and constructing a dynamic spatial relationship graph (i.e., a first dynamic spatial relationship graph) between each pair of nodes (i.e., m nodes) based on the fusion representation code. Finally, based on the constructed first dynamic spatial relationship graph, the learned denoising function is used to interpolate the missing values of the target data. In this method, the n types of data used by the present application are actual collected data, which have no constraints on the distribution of the data to be processed, and can better represent the actual application scenario. Moreover, the first dynamic spatial relationship graph is data-driven, so the spatial correlation between the nodes can be learned based on the fusion representation code, and the implicit spatial correlation can reflect the complex dependence of the data spatio-temporal sequence between the nodes. In addition, the dynamic spatial relationship graph can change according to the input data, so it overcomes the limitations of the static adjacency matrix in traditional methods.

[0007] Based on this, the embodiments of the present application provide the following technical solutions:

[0008] In a first aspect, the present application provides a data processing method, which specifically comprises: first, obtaining a fusion representation code of n types of to-be-processed data on m nodes, the n types of to-be-processed data being synchronous in space-time, m≥2, n≥2. In the embodiments of the present application, the data source of the to-be-processed data required to be processed can be different according to different application scenarios. Then, a dynamic spatial relationship graph (which can be referred to as a first dynamic spatial relationship graph) of the m nodes is determined according to the obtained fusion representation code, wherein the first dynamic spatial relationship graph is used to represent that the constructed spatial relationship graph changes with the change of the n types of to-be-processed data in space-time, and is used to represent the spatial correlation between any two nodes in the m nodes. Then, the first dynamic spatial relationship graph and target data are processed by using a learned denoising function to obtain recovered data, wherein the target data is to-be-processed data of a target type in the n types of to-be-processed data, the target data is time series data, the target data has a data missing condition within a preset time length (for example, traffic flow data in the past year), and the recovered data is the target data after the data missing condition is interpolated.

[0009] In the above-mentioned embodiments of the present application, the actual collected multi-source heterogeneous data (i.e. n types of to-be-processed data) are aligned and fused in space-time, the data of different modalities are mapped to the same feature space to obtain a fusion representation code, and a dynamic spatial relationship graph (i.e. a first dynamic spatial relationship graph) between each two nodes (i.e. m nodes) is constructed based on the fusion representation code. Finally, the learned denoising function is used to interpolate the missing values of the target data based on the constructed first dynamic spatial relationship graph. In this method, the n types of data used by the present application are actual collected data, there is no constraint condition on the distribution of the to-be-processed data, and there is no need to require that the to-be-processed data should conform to a certain distribution rule, which can better represent the actual application scenario, and realizes the space-time alignment and fusion representation of multi-source heterogeneous data, and improves the interpolation accuracy. Moreover, the first dynamic spatial relationship graph is data-driven, so that the spatial correlation between the nodes can be learned based on the fusion representation code, and the hidden spatial correlation can reflect the complex dependence relationship of the space-time sequence of the data between the nodes. In addition, the dynamic spatial relationship graph can change according to the input data, so as to overcome the limitation of the static adjacency matrix in the traditional method, and can more accurately capture the hidden functional association between the nodes, thereby further improving the interpolation accuracy.

[0010] In a possible implementation of the first aspect, an implementation of obtaining the fusion representation code of the n types of to-be-processed data on the m nodes can be: constructing a training set for training the contrast learning model according to the n types of historical data, which can be referred to as a first training set, and n≥2. In the embodiment of the present application, the data sources of the historical data used can be different according to different application scenarios. Then, the contrast learning model is trained through the first training set constructed above, and a trained contrast learning model is obtained. Finally, the n types of to-be-processed data on the m nodes are taken as input data of the trained contrast learning model, and accordingly, the fusion embedding representation of the multi-source data, that is, the common feature representation of the at least two types of historical data corresponding to each other in the same time window, can be obtained, which can be referred to as a fusion representation code.

[0011] In the above-mentioned embodiments of the present application, a specific implementation form of obtaining the fusion representation code is specifically described, that is, the contrast learning model is first trained using the n types of historical data, and then the trained contrast learning model is used to implement the fusion representation coding of the n types of to-be-processed data, thereby realizing the spatio-temporal alignment and fusion representation learning of multi-source heterogeneous data.

[0012] In a possible implementation of the first aspect, an implementation of constructing the first training set according to the n types of historical data can be: first, the n types of historical data are feature-extracted through the trained neural network(s), so as to obtain the data features (also referred to as modal features) corresponding to each type of historical data, that is, one type of historical data corresponds to one type of data feature, and accordingly, n types of data features can be obtained, and then the first training set is constructed according to the n types of data features obtained. In the embodiment of the present application, the neural network used for feature extraction can be determined by judging whether each type of historical data belongs to time series data. Taking traffic data as an example of historical data, assuming that the traffic data has 3 (i.e., n=3) types, which are image data captured by a camera, traffic flow data collected by a flow sensor, and Internet vehicle speed data, for the image data, a pre-trained convolutional neural network (CNN) can be used to extract image features, and for the traffic flow data and Internet vehicle speed data, a recurrent neural network (RNN), AutoEncoder, etc. can be used to extract time series features. Then, the first training set is constructed based on the different modal features extracted.

[0013] In the above-mentioned embodiments of the present application, it is specifically described that one or more pre-trained neural networks are used to extract features from different types of historical data to obtain data features of each type, and then a first training set is constructed based on the data features.

[0014] In a possible implementation of the first aspect, if the n types of data features come from p time windows, p > 2, an implementation of constructing the first training set according to the n types of data features can be that at least one positive sample pair set is formed by at least two types of data features from the same time window, and the first training set includes the at least one positive sample pair set. In some embodiments of the present application, the forming manner of the at least one positive sample pair set formed by at least two types of data features from the same time window includes but is not limited to: case 1, one positive sample pair set is formed by two types of data features from the same time window. That is, a first type of data feature and a second type of data feature from the same time window form one positive sample pair set, where the first type is any one of the n types, and the second type is any one of the remaining n-1 types except the first type. Case 2, one positive sample pair set is formed by r types of data features from the same time window, n > r > 3. That is, one positive sample pair set is formed by r types of data features from the same time window, n > r > 3, where r is any r types of the n types.

[0015] In the above-mentioned embodiments of the present application, it is specifically described that the first training set includes at least one positive sample pair set (from the same time window), and the purpose of training the model is to make the distance between the positive sample pairs as small as possible. The first training set containing the positive sample pair set can improve the training effect of the model.

[0016] In a possible implementation of the first aspect, in addition to the at least one positive sample pair set, the first training set can also include at least one negative sample pair set, and the negative sample pair set is formed by at least two types of data features from different time windows.

[0017] In the above-mentioned embodiments of the present application, in addition to the positive sample pair set, the first training set also includes a negative sample pair set (from different time windows), and the purpose of training the model is to make the distance between the positive sample pairs as small as possible, and at the same time make the distance between the negative sample pairs as large as possible. The first training set containing the positive sample pair set and the negative sample pair set further improves the training effect of the model.

[0018] In a possible implementation form of the first aspect, the implementation of training the contrastive learning model by the first training set can be that the contrastive learning model is trained by the first training set using a contrastive loss function (e.g., a SimCLR loss function, a MoCo loss function, etc.).

[0019] In the above embodiments of the present application, it is specifically illustrated that the loss function used for training the contrastive learning model is a contrastive loss function, which is feasible.

[0020] In a possible implementation form of the first aspect, the method can further include repeating the step of processing the first dynamic spatial relation graph and the target data using the learned denoising function to obtain the recovered data until a termination condition is reached, for example, a preset number of repeated executions (e.g., 50 times) is reached. Wherein, the imputation values obtained by imputing the data missing in each repeated execution process form the interval estimation of the imputation values. For example, if the repeated execution is 50 times, for each missing value, 50 imputation values will be obtained, which form multiple imputation results of the missing value and form the corresponding interval estimation.

[0021] In the above embodiments of the present application, the step of obtaining the recovered data by repeated execution generates multiple imputation results, based on which the confidence interval of the imputation value corresponding to the missing value can be calculated. This method not only can give the point estimation of the missing value, but also can provide the interval estimation. Compared with the method of only providing the point estimation, the uncertainty of the imputation result can be better evaluated, and a more reliable basis for subsequent decision-making can be provided.

[0022] In a possible implementation form of the first aspect, the denoising function is obtained by training a diffusion model, and the training process of the diffusion model includes: first, constructing a training data set for training the diffusion model, which can be referred to as a second training set. The second training set includes a dynamic spatial relation graph of k nodes and training data, wherein the second dynamic relation graph of the k nodes is obtained based on the fusion representation encoding of j types of historical data on the k nodes, and the training data is the historical data of the target type among the j types of historical data. The second dynamic spatial relation graph is used to represent the change of the constructed spatial relation graph with the change of the j types of historical data in space and time, and k≥2 and j≥n. Then, the diffusion model is trained according to the constructed second training set to obtain the learned denoising function. Specifically, the denoising function can be parameterized using a neural network, such as U-Net, PixelCNN, etc., and the diffusion model is trained by minimizing the reconstruction error.

[0023] In the above-mentioned embodiments of the present application, the traditional diffusion model is improved, and a spatial adaptive adjacency matrix learning module is introduced to adjust the correlation of noise (the spatial correlation can be reflected in the denoising effect of the diffusion model) in combination with the learned spatial relationship (i.e., the second dynamic relationship graph), so that the diffusion model can adaptively learn the spatial correlation between nodes at different positions and in different time periods. Compared with the method of assuming fixed spatial correlation, the spatial and temporal characteristics of complex and variable data application scenarios (such as traffic spatio-temporal characteristics) can be more accurately described, and the interpolation accuracy is further improved by capturing the implicit functional association beyond geographical distance.

[0024] In a possible implementation form of the first aspect, an implementation of the method of processing the first dynamic spatial relationship graph and the target data by using the learned denoising function to obtain the recovered data can be that the target data is added with noise to obtain noisy data, and then the obtained noisy data and the first dynamic spatial relationship graph constructed above are taken as input data of the learned denoising function to realize step-by-step noise removal of the noisy data, so as to obtain the recovered data, i.e., the recovered original data.

[0025] In the above-mentioned embodiments of the present application, the learned denoising function is used to realize the denoising process of the noisy data, so as to restore the noisy data to the interpolated recovered data, which is feasible.

[0026] In a possible implementation form of the first aspect, an implementation of the method of adding noise to the target data to obtain noisy data can be that standard Gaussian noise can be added to the target data step by step until the target data completely becomes noise, i.e., the noisy data is obtained.

[0027] In the above-mentioned embodiments of the present application, a specific implementation of adding noise is specifically described, which is simple and easy to operate.

[0028] In a possible implementation form of the first aspect, an implementation of the method of determining the first dynamic spatial relationship graph of the m nodes according to the fusion representation code can be that the similarity between the m nodes is calculated according to the fusion representation code, and the first dynamic spatial relationship graph is constructed according to the calculated similarity.

[0029] In the above-mentioned embodiments of the present application, the first dynamic spatial relationship graph is specifically constructed based on the similarity between the m nodes, which has specific and wide applicability.

[0030] In a possible implementation form of the first aspect, an implementation form of calculating the similarity between the m nodes according to the fusion representation encoding can be that the similarity between the m nodes is calculated according to the fusion representation encoding by using a K-Nearest Neighbor algorithm, or the similarity between the m nodes is calculated according to the fusion representation encoding by using a method based on a self-attention mechanism, and the specific implementation form of how to calculate the similarity is not limited in the present application.

[0031] In the above implementation form of the present application, several implementation forms of calculating the similarity between the m nodes according to the fusion representation encoding are specifically described, which are optional and flexible.

[0032] In a possible implementation form of the first aspect, the to-be-processed data can include traffic data, environmental detection data, weather forecast data or power load prediction data. According to different application scenarios, the data source of the to-be-processed data is different, and the present application does not limit this.

[0033] In the above implementation form of the present application, the field of the data source of the to-be-processed data is specifically described, which is widely applicable.

[0034] In a possible implementation form of the first aspect, in the case that the to-be-processed data is traffic data, the n types of to-be-processed data at least include any two of the following:

[0035] Traffic flow data, vehicle speed data, image data, video data and laser radar data.

[0036] In the above implementation form of the present application, if the to-be-processed data is traffic data, the specific forms of the n types are specifically described, and the data types are generally easy to obtain, which facilitates the practical application of the method of the present application.

[0037] The second aspect of the present application provides a computer device having the function of implementing the method of the first aspect or any one of the possible implementation forms of the first aspect. The function can be realized by hardware, or realized by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0038] The third aspect of the present application provides a computer device, which can include a memory, a processor and a bus system, wherein the memory is used to store a computer program (also referred to as a program, computer readable instructions), and the processor is used to call the program stored in the memory to execute the method of the first aspect or any one of the possible implementation forms of the first aspect of the present application.

[0039] The fourth aspect of the present application provides a computer readable storage medium, which stores instructions, when the instructions are executed on a computer, the computer can execute the method of the first aspect or any possible implementation manner of the first aspect.

[0040] The fifth aspect of the present application provides a computer program or a computer program product containing instructions, when the computer program or the computer program product is executed on a computer, the computer executes the method of the first aspect or any possible implementation manner of the first aspect.

[0041] The sixth aspect of the present application provides a chip, which includes at least one processor and at least one interface circuit, the interface circuit and the processor are coupled, at least one interface circuit is used to execute the transceiving function and send instructions to at least one processor, at least one processor is used to run the computer program or instructions, which has the function of realizing the method of the first aspect or any possible implementation manner of the first aspect, the function can be realized by hardware, also can be realized by software, and also can be realized by combination of hardware and software, the hardware or software includes one or more modules corresponding to the above function. In addition, the interface circuit is used to communicate with other modules outside the chip.

[0042] In some implementations of the present application, part of the one or more processors can also be realized by a dedicated hardware to realize part of the steps in the above method, for example, the processing related to the neural network model can be realized by a dedicated neural network processor or a graphics processor.

[0043] The method provided by the embodiment of the present application can be realized by one chip, or can be realized by multiple chips cooperatively. BRIEF DESCRIPTION OF DRAWINGS

[0044] Fig. 1 is a structural schematic diagram of an artificial intelligence main body framework provided by an embodiment of the present application;

[0045] Fig. 2 is a system architecture diagram of a task processing system provided by an embodiment of the present application;

[0046] Fig. 3 is a flow schematic diagram of a training method of a contrast learning model provided by an embodiment of the present application;

[0047] Fig. 4 is a flow schematic diagram of a training method of a diffusion model provided by an embodiment of the present application;

[0048] Fig. 5 is a flow schematic diagram of a data processing method provided by an embodiment of the present application;

[0049] Fig. 6 is a schematic diagram of a system architecture provided by an embodiment of the present application;

[0050] FIG. 7 is a schematic diagram of an example of the data processing method provided by the embodiments of the present application in traffic management;

[0051] FIG. 8 is a schematic diagram of a computer device provided by the embodiments of the present application;

[0052] FIG. 9 is a schematic diagram of a structure of a computer device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0053] The embodiments of the present application provide a data processing method, which is used for mapping data of different modalities to the same feature space by aligning and fusing the actually collected multi-source heterogeneous data to be processed in space-time, obtaining fusion representation encoding, and constructing a first dynamic spatial relationship graph between each pair of nodes based on the fusion representation encoding, and finally realizing the imputation of missing values of target data by using the learned denoising function based on the constructed first dynamic spatial relationship graph. In the method, the n types of data used by the present application are actually collected data, which have no any constraint conditions for the distribution of the data to be processed, and can better represent the actual application scenario. In addition, the first dynamic spatial relationship graph is data-driven, so that the spatial correlation between the nodes can be learned based on the fusion representation encoding, and the implicit spatial correlation can reflect the complex dependence relationship of the space-time sequence of the data between the nodes. Furthermore, the dynamic spatial relationship graph can change according to the input data, so as to overcome the limitations of the static adjacency matrix in the traditional method.

[0054] The terms "first", "second", etc. in the specification and claims of the present application and in the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the terms thus used can be interchanged under appropriate circumstances, and this is merely a distinguishing way used in the description of the embodiments of the present application to describe the objects with the same properties. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, so that a process, method, system, product or device containing a series of units does not have to be limited to those units, but can include other units not clearly listed or inherent to the process, method, product or device.

[0055] In order to better understand the scheme of the embodiments of the present application, the related terms and concepts that may be involved in the embodiments of the present application will be introduced first. It should be understood that the explanation of the related terms and concepts may be limited to the specific situation of the embodiments of the present application, but it does not mean that the present application can only be limited to this specific situation, and the specific situation of different embodiments may also be different, and the specific situation is not limited here.

[0056] (1) Neural network

[0057] The neural network can be composed of neural units, and can specifically be a neural network having an input layer, hidden layers, and an output layer. Generally, the first layer is the input layer, the last layer is the output layer, and the layers in between are hidden layers. A neural network having many hidden layers is referred to as a deep neural network (DNN). The work of each layer in the neural network can be described by a mathematical expression From a physical perspective, the work of each layer in the neural network can be understood as completing the transformation of the input space (a set of input vectors) to the output space (i.e., the row space of a matrix to the column space of the matrix) through five operations on the input space, including: 1, dimensionality increase / decrease; 2, magnification / reduction; 3, rotation; 4, translation; and 5, "bending". The operations of 1, 2, and 3 are completed by The operation of 4 is completed by "+b", and the operation of 5 is implemented by "a()". The reason for using the word "space" is that the objects being classified are not single things, but a class of things, and the space refers to the set of all individuals of this class of things. W is a weight matrix of each layer of the neural network, and each value in the matrix represents the weight value of a neuron in the layer. The matrix W determines the spatial transformation of the input space to the output space described above, i.e., W of each layer of the neural network controls how to transform the space. The purpose of training the neural network is to ultimately obtain the weight matrix of all layers of the trained neural network. Therefore, the training process of the neural network is essentially learning the way to control the spatial transformation, and more specifically, learning the weight matrix.

[0058] It should be noted that in the embodiments of the present application, the models used based on machine learning tasks are essentially neural networks, such as contrast learning models, diffusion models, etc.

[0059] (2) Loss Function

[0060] In the process of training a neural network, because the output of the neural network is expected to be as close as possible to the value that is actually intended to be predicted, the weight matrix of each layer of the neural network can be updated according to the difference between the predicted value of the current network and the target value that is actually intended to be predicted (of course, before the first update, there is usually an initialization process, that is, the parameters of each layer of the neural network are pre-configured), for example, if the predicted value of the network is too high, the weight matrix is adjusted to make it predict lower, and the adjustment is continuously made until the neural network can predict the target value that is actually intended to be predicted. Therefore, it is necessary to define "how to compare the difference between the predicted value and the target value", which is the loss function or the objective function, which is an important equation for measuring the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, and then the training of the neural network becomes a process of trying to minimize the loss.

[0061] In the training process of the neural network, the back propagation (BP) algorithm can be used to correct the size of the parameters in the initial neural network model, so that the reconstruction error loss of the neural network model becomes smaller and smaller. Specifically, the input signal is forward propagated until the output produces an error loss, and the parameters in the initial neural network model are updated by back propagating the error loss information, so as to make the error loss converge. The back propagation algorithm is a back propagation movement dominated by error loss, aiming to obtain the optimal parameters of the neural network model, such as the weight matrix.

[0062] The embodiments of the present application will be described below in conjunction with the drawings. Those skilled in the art can know that, with the development of technology and the appearance of new scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0063] Firstly, the overall workflow of the artificial intelligence system is described, please refer to FIG. 1, which shows a structural schematic diagram of an artificial intelligence main framework, and the above-mentioned artificial intelligence theme framework is described below from two dimensions of "intelligent information chain" (horizontal axis) and "IT value chain" (vertical axis). Among them, the "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it can be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, data undergoes a condensation process of "data-information-knowledge-wisdom". The "IT value chain" reflects the value brought by artificial intelligence to the information technology industry from the bottom infrastructure of human intelligence, information (provision and processing technology implementation) to the industrial ecological process of the system.

[0064] (1) Infrastructure

[0065] Infrastructure provides computing power support for artificial intelligence systems, realizes communication with the external world, and realizes support through the underlying platform. Communication with the outside world through sensors; computing power is provided by intelligent chips (CPU, NPU, GPU, ASIC, FPGA, etc. Hardware acceleration chips); the underlying platform includes distributed computing framework and network-related platform support and support, which can include cloud storage and computing, interconnection network, etc. For example, sensors and external communication obtain data, which are provided to intelligent chips in the distributed computing system provided by the underlying platform for calculation.

[0066] (2) Data

[0067] The data of the upper layer of the infrastructure is used to represent the data source in the field of artificial intelligence. The data relates to graphics, images, voice, text, and also relates to the Internet of Things data of traditional devices, including the business data of existing systems and the sensing data of force, displacement, liquid level, temperature, humidity, etc.

[0068] (3) Data processing

[0069] Data processing usually includes data training, machine learning, deep learning, search, reasoning, decision, etc.

[0070] Among them, machine learning and deep learning can model, extract, preprocess, train, etc. Symbolic and formalized intelligent information.

[0071] Reasoning refers to the process of simulating human intelligent reasoning methods in computers or intelligent systems, using formalized information to perform machine thinking and solve problems according to reasoning control strategies, and the typical function is search and matching.

[0072] Decision-making refers to the process of decision-making after intelligent information is reasoned, which usually provides functions such as classification, sorting, prediction, etc.

[0073] (4) General capabilities

[0074] After the data is processed as mentioned above, some general capabilities can be formed based on the results of data processing, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0075] (5) Intelligent products and industry applications

[0076] Intelligent products and industry applications refer to products and applications of artificial intelligence systems in various fields, which are the packaging of the overall solution of artificial intelligence, and realize the application of intelligent information decision-making products. Its application fields mainly include: intelligent terminals, intelligent manufacturing, intelligent transportation, smart home, intelligent medical treatment, intelligent security, autonomous driving, smart city, etc.

[0077] The embodiments of the present application can be applied to the training and application of the contrast learning model and the diffusion model. The trained contrast learning model and diffusion model can be applied in various subfields of the field of artificial intelligence. For example, it can be applied to the field of missing time sequence data (such as long-term missing of traffic flow data time sequence).

[0078] The model framework of the contrast learning model and the diffusion model described in the embodiments of the present application can be deployed on a training device (i.e., a computer device used for training a model). Please refer to FIG. 2, which is a system architecture diagram of a task processing system provided by the embodiments of the present application. In FIG. 2, the task processing system 200 includes an execution device 210, a training device 220, a database 230, a client device 240, a data storage system 250, and a data acquisition device 260. The execution device 210 includes a computing module 211. The data acquisition device 260 is used to obtain large-scale data sets (collectively referred to as training data sets or training sets) required by users, and store the training data sets in the database 230. The training device 220 trains the model (i.e., the contrast learning model 201 and the diffusion model 202 described in FIG. 2) based on the training data sets maintained in the database 230. It should be noted that since the model to be trained in the present application is not only one, the corresponding training data set is also not only one. Generally speaking, one model corresponds to one training data set. For example, in the embodiments of the present application, the training data set used to train the contrast learning model 201 can be referred to as the first training set, and the training data set used to train the diffusion model 202 can be referred to as the second training set. However, it should be noted that in the embodiments of the present application, the construction of the second training set depends on the fusion representation encoding obtained by the trained contrast learning model 201. The trained model can be used on the execution device 210. The execution device 210 can call data, code, etc. in the data storage system 250, or store data, instructions, etc. in the data storage system 250. The data storage system 250 can be placed in the execution device 210, or the data storage system 250 can be an external memory relative to the execution device 210.

[0079] The contrast learning model 201 and the diffusion model 202 trained via the training device 220 can select one with the optimal performance evaluation index as the target model (i.e., the trained contrast learning model 201 and the trained diffusion model 202, in the embodiment of the present application, the trained diffusion model 202 adopts the learned denoising function) based on the validation data set, and the target model can be applied to different systems or devices (i.e., the execution device 210), which can be edge devices or end-side devices, such as mobile phones, tablets, notebook computers, monitoring systems (such as cameras), security systems, and the like. In FIG. 2, the execution device 210 is configured with an I / O interface 212 for data interaction with external devices, and a "user" can input data to the I / O interface 212 through a client device 240. For example, the client device 240 can be a related device for data collection, such as a camera device deployed on a high-speed gantry, a flow sensor, or a terminal device such as a vehicle-mounted device or a mobile phone, a tablet, and the like. These devices can collect the to-be-processed data in real time, for example, the to-be-processed data can be target images captured by a camera device on a gantry, traffic flow data collected by a flow sensor, Internet vehicle speed data collected by a vehicle-mounted device (or a mobile phone, a tablet, etc. deployed on a vehicle), and the like. The to-be-processed data is input to the computing module 211 of the execution device 210 as input data, and the computing module 211 performs the interpolation processing of data missing on the input to-be-processed data. In addition, in some embodiments of the present application, the client device 240 can also be integrated in the execution device 210, for example, when the execution device 210 is a mobile phone, the to-be-processed data can be obtained directly through the mobile phone or received by the mobile phone from other devices (such as another mobile phone), and then the computing module 211 in the mobile phone performs the interpolation processing of data missing on the to-be-processed data. The product form of the execution device 210 and the client device 240 is not limited here.

[0080] It is worth noting that FIG. 2 is only a schematic diagram of a system architecture provided by an embodiment of the present application, and the positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation, for example, in FIG. 2, the data storage system 250 is an external memory relative to the execution device 210, and in other cases, the data storage system 250 can also be placed in the execution device 210; in FIG. 2, the client device 240 is an external device relative to the execution device 210, and in other cases, the client device 240 can also be integrated in the execution device 210.

[0081] It should be further noted that the training of the models described in the above embodiments can be implemented on the cloud side, for example, the training data set can be obtained by the training device 220 on the cloud side (which can be arranged on one or more servers or virtual machines), and the involved models can be trained according to the training data set, for example, the contrast learning model 201 is trained according to the first training set, and the diffusion model 202 is trained according to the second training set. The trained model is sent to the execution device 210 for application, for example, the trained diffusion model 202 is sent to the execution device 210 for the imputation processing of the data missing of the to-be-processed data. Exemplarily, the system architecture corresponding to FIG. 2 is that the contrast learning model 201 and the diffusion model 202 are trained by the training device 220, and the trained contrast learning model 201 and the diffusion model 202 are sent to the execution device 210 for use. The training of each model described in the above embodiments can also be implemented on the terminal side, that is, the training device 220 can be located on the terminal side, for example, the training data set (that is, the first training set and the second training set) can be obtained by a terminal device (such as a mobile phone, a tablet computer, etc.), a monitoring system device, etc., and each model can be trained according to the training data set to obtain a trained model. The trained model can be directly used on the terminal device, or can be sent to other devices for use. The application embodiments do not limit the device (cloud side or terminal side) on which each model is trained or applied.

[0082] In combination with the above description, the specific implementation process of the training phase and the application phase of the data processing method provided by the application embodiments will be described below.

[0083] I. Training phase

[0084] In the application embodiments, the training phase describes the process in which the training device 220 in FIG. 2 performs a training operation on the contrast learning model 201 and the diffusion model 202 by using a training data set, which will be described below.

[0085] A. Training phase of the contrast learning model

[0086] For details, please refer to FIG. 3, which is a flowchart of a training method of a contrast learning model provided by the application embodiments. The training method of the contrast learning model can specifically include the following steps:

[0087] 301. Constructing a first training set according to n types of historical data.

[0088] First, a training set for training the contrast learning model is constructed according to n types of historical data, which can be referred to as a first training set, and n≥2.

[0089] It should be noted that, in the embodiments of the present application, the data source of the historical data can be different according to different application scenarios.

[0090] As an example, if the data processing method provided by the present application is applied to a traffic monitoring system, the historical data can be traffic data, such as traffic flow data, vehicle speed data (such as average vehicle speed, which can be collected by each driving vehicle and transmitted via the Internet, and thus can also be referred to as Internet vehicle speed), image data, video data, and laser radar data. The traffic flow data, vehicle speed data, image data, video data, and laser radar data are different types of historical data. In the embodiments of the present application, n types of historical data are used, and n≥2.

[0091] As another example, if the data processing method provided by the present application is applied to environmental monitoring, the historical data can be atmospheric pollutant concentration data, wind direction angle data, wind speed data, and noise measurement data. The atmospheric pollutant concentration data, wind direction angle data, wind speed data, and noise measurement data are different types of historical data. In the embodiments of the present application, n types of historical data are used, and n≥2. These data can be collected based on specific sensors or devices, and the present application does not repeat the details.

[0092] Similarly, according to different application scenarios, the historical data can also be weather forecast data and power load prediction data, which are not limited by the present application.

[0093] It should be noted that, in some embodiments of the present application, since the historical data is used to construct the first training set, the historical data can be data with missing values or data without missing values, which are not limited by the present application.

[0094] It should also be noted that, in some other embodiments of the present application, the implementation of constructing the first training set from n types of historical data can be as follows: first, the n types of historical data are subjected to feature extraction by the trained neural network, so as to obtain data features (also referred to as modal features) corresponding to each type of historical data, that is, one type of historical data corresponds to one type of data feature, and thus n types of data features can be obtained; and then, the first training set is constructed according to the n types of data features obtained.

[0095] It should be noted that in the embodiments of the present application, the neural network applied to extracting features can be determined by judging whether each type of historical data belongs to time series data. Taking traffic data as an example, assuming that the traffic data has a total of 3 (i.e. n = 3) types, which are image data captured by a camera, traffic flow data collected by a flow sensor, and Internet vehicle speed data, for image data, a pre-trained CNN can be used to extract image features, such as using ResNet, VGG, etc. to extract image features, and for traffic flow data and Internet vehicle speed data, RNN, AutoEncoder, etc. can be used to extract time series features. Then, the first training set is constructed based on the extracted different modal features.

[0096] Specifically, assuming that the n types of data features extracted come from p (p≥2) time windows, that is, each time window contains n types of data features, then one implementation of constructing the first training set according to the n types of data features can be: at least two types of data features from the same time window form at least one positive sample pair set, and the first training set includes the at least one positive sample pair set.

[0097] To facilitate understanding of the above-mentioned correspondence between data features and time windows, an example is given as follows:

[0098] Assuming that the extracted data features have a total of 3 types, which are type 1, type 2, and type 3, and assuming that the data features of each type come from 4 time windows, which are window 1, window 2, window 3, and window 4. Assuming that the data features of type 1 from window 1 are a1 (which can be one or more, similar hereinafter, not described in detail), the data features of type 1 from window 2 are a2, the data features of type 1 from window 3 are a3, and the data features of type 1 from window 4 are a4, then the data features A of type 1 are the set of data features of type 1 from the above-mentioned 4 time windows, that is, data features A = {a1, a2, a3, a4}; Similarly, assuming that the data features of type 2 from window 1 are b1, the data features of type 2 from window 2 are b2, the data features of type 2 from window 3 are b3, and the data features of type 2 from window 4 are b4, then the data features B of type 2 are the set of data features of type 2 from the above-mentioned 4 time windows, that is, data features B = {b1, b2, b3, b4}; Similarly, the set of data features C of type 3 can be further obtained according to similar assumptions (not described in detail here), that is, data features C = {c1, c2, c3, c4}.

[0099] It is noted that, in some embodiments of the present application, the manner of constructing at least one positive sample pair set from at least two types of data features from the same time window includes but is not limited to:

[0100] Case 1: Two types of data features from the same time window are constructed into one positive sample pair set.

[0101] A first type of data feature and a second type of data feature from the same time window are constructed into one positive sample pair set, wherein the first type is any one of the n types, and the second type is any one of the remaining n-1 types other than the first type. In this way, each positive sample pair set can be expressed as the following formula (1): P i-x ={x1i, x2i} (1)

[0102] Wherein, P is one positive sample pair set, i is any one of the p time windows, x1 is a data feature corresponding to any one of the n types, and x2 is a data feature corresponding to any one of the remaining n-1 types other than x1.

[0103] For ease of understanding, the following will still take the above example as an example:

[0104] Suppose the extracted data features have a total of 3 types, which are type 1, type 2, and type 3. It is assumed that the data features of each type are derived from 4 time windows, which are window 1, window 2, window 3, and window 4, and the data features A = {a1, a2, a3, a4}, data features B = {b1, b2, b3, b4}, and data features C = {c1, c2, c3, c4}. Wherein, a1, b1, c1 are all from the same time window 1, a2, b2, c2 are all from the same time window 2, a3, b3, c3 are all from the same time window 3, and a4, b4, c4 are all from the same time window 4.

[0105] Therefore, in this case, it is assumed that the time window to be aligned is window 1, and three positive sample pair sets can be constructed: data feature a1 and data feature b1 form one positive sample pair set, which can be expressed as P 1-ab ={a1, b1}; Similarly, data feature a1 and data feature c1 form one positive sample pair set, which can be expressed as P 1-ac ={a1, c1}; data feature b1 and data feature c1 form one positive sample pair set, which can be expressed as P 1-bc ={b1, c1}. Similarly, if the time window to be aligned is window 2, then P 2-ab ={a2, b2}, P2- ac = {a2, c2}, P 2-bc = {b2, c2}. Other positive sample pair sets are constructed in a similar manner, which is not described here.

[0106] To further understand the process of constructing the positive sample pair set, the following still takes traffic data as an example for description:

[0107] Suppose that the extracted data features have a total of 3 types, which are image features extracted based on image data, traffic features extracted based on traffic flow data, and vehicle speed features extracted based on Internet vehicle speed. Then, in the time window to be aligned, a set of image features (containing one or more) from the same time window and a corresponding set of traffic features (containing one or more) form a positive sample pair set; similarly, a set of image features from the same time window and a corresponding set of vehicle speed features (containing one or more) form a positive sample pair set; a set of traffic features from the same time window and a corresponding set of vehicle speed features form a positive sample pair set. Accordingly, a total of 3 positive sample pair sets are obtained. If the time window to be aligned has p, then 3 positive sample pair sets can be obtained for each time window, and accordingly a total of 3*p positive sample pair sets can be obtained.

[0108] It should be noted that in the application embodiment, the first training set includes at least one positive sample pair set constructed as described above, and the target positive sample pair set can be selected from all the positive sample pair sets constructed as described above to form the first training set according to actual needs, for example, the 3*p positive sample pair sets can be used as the first training set, or part of the positive sample pair sets can be selected from the 3*p positive sample pair sets as the first training set based on a preset selection manner, which is not limited in the present application.

[0109] Case 2: r types of data features from the same time window form a positive sample pair set, n≥r≥3.

[0110] r types of data features from the same time window form a positive sample pair set, n≥r≥3, where r is any r of the n types, and each positive sample pair set can be expressed as the following formula (2): i-x = {x1i, x2i, …, x r i} (2)

[0111] Where P is a positive sample pair set, i is any one of the p time windows, x1, x2, …, x r are data features corresponding to any r types of the n types respectively.

[0112] In this case, the manner of constructing each positive sample pair set is similar to that of the above case 1, please refer to the above case 1 for details, which will not be repeated here.

[0113] It should be noted that in some embodiments of the present application, the first training set can include at least one negative sample pair set in addition to the at least one positive sample pair set described above, and the negative sample pair set is composed of at least two types of data features from different time windows. Each negative sample pair set can be represented as formula (3) as follows: N i-x ={x1i,x2j,……,x r k} (3)

[0114] Wherein, N is a negative sample pair set, i, j, k are respectively any different time windows in the p time windows, x1, x2, …, x r are data features corresponding to any r types of the n types respectively.

[0115] It should also be noted that in the application embodiment, the first training set includes at least one negative sample pair set constructed above in addition to the at least one positive sample pair set constructed above. The target negative sample pair set can be selected from all negative sample pair sets constructed above and added to the first training set according to actual needs, for example, all negative sample pair sets constructed based on formula (3) can be added to the first training set, or part of the negative sample pair sets can be selected from all negative sample pair sets based on a preset selection method and added to the first training set. The present application does not limit this.

[0116] 302. Train the contrast learning model through the first training set to obtain a trained contrast learning model.

[0117] Then, the first training set constructed above is used to train the contrast learning model, so that the distance between the positive sample pairs is as small as possible, and the distance between the negative sample pairs is as large as possible (if the first training set includes negative sample pairs), until the training termination condition is reached, and accordingly a trained contrast learning model is obtained. In some embodiments of the present application, the contrast loss function (such as SimCLR loss function, MoCo loss function, etc.) can be used to train the contrast learning model to obtain the trained contrast learning model. Accordingly, the fusion embedding representation of multi-source data can be obtained, that is, the common feature representation of the at least two types of historical data corresponding to each other in the same time window, which can be called fusion representation encoding.

[0118] B. Training phase of diffusion model.

[0119] Specifically, refer to FIG. 4, which is a flowchart of a training method of a diffusion model provided in an embodiment of the present application. The training method of the diffusion model can specifically include the following steps:

[0120] 401. Construct a second training set, which includes a second dynamic spatial relationship graph of k nodes and training data.

[0121] First, a training data set for training the diffusion model is constructed, which can be referred to as a second training set. The second training set includes a dynamic spatial relationship graph of k nodes and training data. The second dynamic relationship graph of the k nodes is obtained based on the fusion representation encoding of j types of historical data on the k nodes. The training data is historical data of a target type among the j types of historical data. The second dynamic spatial relationship graph is used to represent the change of the constructed spatial relationship graph with the change of the j types of historical data in space and time. k≥2 and j≥n.

[0122] Therefore, the second training set includes two parts of data, which will be described below:

[0123] The first part is the second dynamic spatial relationship graph of the k nodes.

[0124] In the embodiment of the present application, the construction of the second training set depends on the fusion representation encoding obtained by the trained contrastive learning model. At this time, since the contrastive learning model has been trained, the application of the contrastive learning model is in the inference stage. Assuming that the input data of the contrastive learning model in the inference stage is j types of historical data on k nodes, the obtained is the fusion representation encoding of j types of historical data on k nodes (referred to as inference stage fusion representation encoding). The second dynamic relationship graph of the k nodes is obtained based on the inference stage fusion representation encoding, which is the first part of the second training set.

[0125] The second part is the training data.

[0126] In the embodiment of the present application, the training data is historical data of a target type among j types of historical data. Since the training purpose of the diffusion model is to obtain a good denoising function to realize the interpolation of missing values of target time series data, the training data is time series data, and there is data missing in a preset time length (such as traffic flow data in the past year). The training process of the diffusion model is to learn how to reasonably interpolate the missing values.

[0127] 402. Train the diffusion model through the second training set to obtain a learned denoising function.

[0128] Afterwards, the diffusion model is trained according to the constructed second training set, and a learned denoising function is obtained. Specifically, the denoising function can be parameterized using a neural network, such as U-Net, PixelCNN, etc., and the diffusion model is trained by minimizing the reconstruction error.

[0129] The present application improves the traditional diffusion model, introduces a spatial adaptive adjacency matrix learning module, which is used to adjust the correlation of noise (the spatial correlation can be reflected in the denoising effect of the diffusion model) in combination with the learned spatial relationship (i.e., the second dynamic relationship graph), so that the diffusion model can capture the implicit functional association beyond geographical distance.

[0130] II. Inference phase

[0131] In the embodiments of the present application, the inference phase describes the process of using the trained model to process the input data based on the execution device 210 in FIG. 2 above. For details, please refer to FIG. 5, which is a flowchart of a data processing method provided by an embodiment of the present application. Specifically, it can include the following steps:

[0132] 501. Obtain the fusion representation encoding of n types of data to be processed on m nodes, which are synchronized in space-time, m≥2, n≥2.

[0133] First, the fusion representation encoding of n types of data to be processed on m nodes is obtained, which are synchronized in space-time, m≥2, n≥2.

[0134] Specifically, in some embodiments of the present application, the fusion representation encoding of n types of data to be processed on m nodes can be obtained based on the trained contrastive learning model obtained in the training phase described above, i.e., the m nodes n types of data to be processed are taken as the input data of the trained contrastive learning model, and the fusion representation encoding is obtained.

[0135] It should be noted that in some embodiments of the present application, the data source of the data to be processed can also be different according to different application scenarios.

[0136] As an example, if the data processing method provided in the application is applied to a traffic monitoring system, the to-be-processed data can be traffic data, such as traffic flow data (such as collected by a flow sensor deployed on a gantry), vehicle speed data (such as average vehicle speed, which can be collected by each vehicle traveling and transmitted via the Internet, and thus can also be referred to as Internet vehicle speed), image data (such as collected by a camera deployed on the side of a road), video data (such as collected by a camera deployed on the side of a road), laser radar data (such as collected by a millimeter wave radar sensor deployed on the side of a road), and the like. Therefore, the n types of to-be-processed data at least include any two of the following: traffic flow data, vehicle speed data, image data, video data, and laser radar data.

[0137] As another example, if the data processing method provided in the embodiment of the application is applied to environmental monitoring, the to-be-processed data can be atmospheric pollutant concentration data, wind direction angle data, wind speed data, noise measurement data, and the like, and the atmospheric pollutant concentration data, wind direction angle data, wind speed data, and noise measurement data are different types of to-be-processed data. In the embodiment of the application, n types of to-be-processed data are used, and n≥2. These data can all be collected based on specific sensors or devices, which will not be described in detail herein.

[0138] Similarly, according to different application scenarios, the to-be-processed data can also be weather forecast data, power load prediction data, and the like, which are not limited in the application.

[0139] It should be noted that, in the embodiment of the application, taking traffic data as an example, the node (including the m nodes in the inference stage and the k nodes in the training stage) can be a target road section, and in this case, the corresponding to-be-processed data (historical data in the training stage) can be average vehicle speed, Internet vehicle speed, and the like. The node can also be a gantry, and in this case, the corresponding to-be-processed data can be cross-sectional flow data, single-point vehicle speed, and the like. The node can also be a toll station, and in this case, the corresponding to-be-processed data can be toll data, traffic flow data, and the like. The specific form of the node is not limited in the application, and is subject to the actual application scenario.

[0140] 502、determine a first dynamic spatial relationship graph of the m nodes according to the fusion representation code.

[0141] Then, a dynamic spatial relationship graph (which can be referred to as a first dynamic spatial relationship graph) of the m nodes is determined according to the obtained fusion representation code, wherein the first dynamic spatial relationship graph is used to represent that the constructed spatial relationship graph changes with the change of the n types of to-be-processed data in space-time.

[0142] Specifically, the similarity between the m nodes (e.g., between pairs of nodes) can be calculated based on the fusion representation encoding, and the first dynamic spatial relationship graph can be constructed based on the obtained similarity (the adjacency matrix in the prior art is static).

[0143] In some embodiments of this application, the similarity between m nodes can be calculated by using the K-nearest neighbor algorithm based on the fusion representation encoding, or by using a self-attention mechanism based on the fusion representation encoding. This application does not limit the specific method of calculating similarity.

[0144] It should be noted that during the training phase of the diffusion model, the second dynamic relationship graph of k nodes can be obtained in a similar manner. For details, please refer to the construction method of the first dynamic relationship graph of m nodes mentioned above, which will not be elaborated here.

[0145] 503. The learned denoising function is used to process the first dynamic spatial relationship graph and the target data to obtain the restored data. The target data is the data to be processed that belongs to the target type among the n types of data to be processed. The target data is time series data. There are data gaps in the target data within a preset time period. The restored data is the target data after interpolating the data gaps.

[0146] This step constitutes the inference phase of the diffusion model. Using the learned denoising function, the Gaussian noise obtained from sampling (e.g., vehicle speed and traffic flow data collected within a target time period) is restored to an interpolated vehicle speed or traffic flow sequence through multiple denoising operations. During the denoising process, for missing values, the diffusion model generates reasonable interpolated values ​​based on the learned spatiotemporal dependencies.

[0147] Specifically, the learned denoising function is used to process the first dynamic spatial relationship graph and the target data to obtain the restored data. The target data is the data to be processed that belongs to the target type among the n types of data to be processed. The target data is time series data. The target data has data gaps within a preset time period (e.g., traffic flow data of the past year). The restored data is the target data after interpolating the data gaps.

[0148] It should be noted that in some embodiments of the present application, one implementation of the processing of the first dynamic spatial relationship graph and the target data using the learned denoising function to obtain the recovered data can be: adding noise to the target data to obtain noisy data, for example, standard Gaussian noise can be gradually added to the target data until the target data becomes completely noisy, i.e., the noisy data is obtained; then, the obtained noisy data and the first dynamic spatial relationship graph constructed above are used as input data of the learned denoising function to gradually remove the noise of the noisy data, thereby obtaining the recovered data, i.e., the original data.

[0149] 504, repeat step 503 until a termination condition is reached, wherein the imputed values obtained by imputing the data missing in each repeated execution of step 503 constitute interval estimates of the imputed values.

[0150] Repeat step 503 until a termination condition is reached, for example, a predetermined number of repeated executions (e.g., 100 times) is reached. In each repeated execution of step 503, the imputed values obtained by imputing the data missing constitute interval estimates of the imputed values. For example, if repeated 100 times, for each missing value, 100 imputed values will be obtained, which constitute multiple imputed results of the missing value, forming the corresponding interval estimate, according to which the confidence interval of the imputed value corresponding to the missing value can be calculated. This method not only gives point estimates of missing values, but also provides interval estimates to evaluate the uncertainty of imputation.

[0151] In summary, the overall framework of the above-mentioned embodiment method of the present application can be as shown in FIG. 6 (for ease of understanding, FIG. 6 shows traffic data to be processed), which is a schematic diagram of a system framework provided by an embodiment of the present application, and the system framework mainly includes multi-source data fusion based on multi-view contrast learning, spatiotemporal interpolation based on a spatial adaptive diffusion model, and traffic flow interpolation and uncertainty quantification (not shown in FIG. 6). As shown in FIG. 6, for image data (i.e., data source 1) captured by a camera, traffic flow data (i.e., data source 2) collected by a flow sensor deployed on a gantry, Internet average vehicle speed (i.e., data source 3), and other multi-source heterogeneous data, the present application designs a contrast learning model to realize alignment thereof in space and time, thereby obtaining a fusion representation code of multi-modal fusion representation. The obtained fusion representation code is applied to spatiotemporal interpolation of the spatial adaptive diffusion model, which is mainly used to learn spatial relationships and, in the denoising process, adjust the spatial correlation of noise in combination with the learned dynamic spatial relationship graph, so that the diffusion model can capture the implicit functional association beyond geographical distance. Finally, a neural network is used to parameterize the denoising function, and the model is trained by minimizing the reconstruction error. Finally, using the learned noise function, the sampled Gaussian noise is restored to the interpolated time series data sequence (such as a traffic flow sequence) through multiple denoising.

[0152] For ease of understanding the above process, the following describes the method of processing the above data by taking a specific example scenario as an example:

[0153] In this example scenario, the purpose is to interpolate traffic flow data of a certain urban road network. It is assumed that the road network includes 100 road segments, each of which is installed with a flow sensor for collecting traffic flow data every minute. Due to sensor failure and other reasons, there are long-term missing traffic flow data for some road segments. For details, please refer to FIG. 7, which is an example framework diagram of the data processing method in traffic management provided by an embodiment of the present application, and includes four parts, which are described as follows:

[0154] (1) Data preprocessing

[0155] Firstly, the data involved in the present example includes the following types:

[0156] a. Traffic flow data

[0157] including historical traffic flow data of the past year, containing missing values.

[0158] b. Image data

[0159] including image data collected by road monitoring cameras in the past year, and the collected image data is synchronized with the above-mentioned traffic flow data in time and space.

[0160] c. Internet vehicle speed data

[0161] The data includes vehicle speed data uploaded by taxis, online car hailing services, or Global Positioning System (GPS) devices in the past year, synchronized in time with the traffic flow data described above.

[0162] It should be noted that after obtaining the above-mentioned several types of original traffic data, the above-mentioned original traffic data can be directly applied to the subsequent processing process. In the present example, in order to improve the processing efficiency of the subsequent processing, the original traffic data can be pre-processed first, for example, for traffic flow data, missing values can be marked, and abnormal values can be processed and removed; for image data, scaling, normalization and other operations can be performed; for Internet vehicle speed data, missing values can be interpolated, and abnormal values can be processed.

[0163] (2) Multi-view contrastive learning

[0164] The multi-view contrastive learning process is the process of training and reasoning the contrastive learning model, which includes the following stages:

[0165] a. Feature extraction

[0166] For the above-mentioned several types of traffic data, a pre-trained CNN (such as RseNet, VGG, etc.) can be used to extract image features of image data; similarly, a pre-trained RNN or AutoEncoder can be used to extract time series features of traffic flow data and Internet vehicle speed data.

[0167] b. Contrastive learning (training phase)

[0168] At least two types of data features from the same time window are combined to form at least one positive sample pair set, for example, image features and corresponding traffic flow features belonging to the same time window form a positive sample pair set, and vehicle speed features and corresponding traffic flow features belonging to the same time window form another positive sample pair set. According to the similar construction method (see the construction method shown in the above formulas (1) and (2)), at least one positive sample pair set can be constructed.

[0169] Similarly, data features belonging to different time windows are combined to form at least one negative sample pair set (see the construction method shown in the above formula (3)).

[0170] The constructed positive sample pair set and negative sample pair set constitute a first training set for training the contrast learning model. The contrast learning model is trained using a contrast loss function (e.g., a SimCLR loss function, a MoCo loss function, etc.) to obtain a trained contrast learning model.

[0171] It should be noted that since this stage is a training stage, the data used is historical data, not the data obtained in the above data preprocessing process.

[0172] c. Fusion representation (inference stage)

[0173] Based on the trained contrast learning model, the several types of traffic data (i.e., multi-source data) obtained in the above data preprocessing process are processed to obtain the fusion embedding representation of the multi-source data, that is, the fusion representation encoding.

[0174] (3) Spatial adaptive diffusion model training

[0175] The spatial adaptive diffusion model training includes the following stages:

[0176] a. Spatial relationship learning

[0177] Based on the fusion representation encoding of the multi-source data obtained by the trained contrast learning model, the similarity between each node included in the road network is calculated, and a dynamic spatial relationship graph is constructed according to the calculated similarity.

[0178] b. Diffusion process

[0179] In the denoising process, the spatial correlation of the noise is adjusted in combination with the constructed dynamic spatial relationship graph.

[0180] c. Denoising function training

[0181] A neural network is used to parameterize the denoising function, such as U-Net, PixelCNN, etc., and the model is trained by minimizing the reconstruction error.

[0182] (4) Traffic flow data interpolation

[0183] The traffic flow data interpolation includes the following stages:

[0184] a. Noise injection

[0185] Standard Gaussian noise is gradually added to the target data to be interpolated until the target data becomes completely noisy. For example, the data to be interpolated is the traffic flow data of the past year in the data preprocessing process, and the target data is the traffic flow data.

[0186] b. Denoising process

[0187] With the learned denoising function, the noise of the target data that has completely become noise is gradually removed, and the original data is recovered.

[0188] c. Interpolation generation

[0189] In the denoising process, for missing values in the target data, the model will generate reasonable imputation values according to the learned spatio-temporal dependency.

[0190] d. Interval estimation

[0191] Based on the learned conditional distribution, multiple imputation results are generated by multiple sampling (i.e., for the same target data, the above noise injection, denoising process, and interpolation generation process are repeated), and the confidence interval of the imputation value is calculated.

[0192] The embodiments of the present application demonstrate the application of the data processing method provided by the present application in the actual traffic flow imputation scene, and describe the entire data processing process in detail. Compared with the prior art, the method of the present application can more accurately and robustly impute traffic flow spatio-temporal sequences, and provide more reliable data support for traffic management, traffic prediction and other applications.

[0193] Therefore, in summary, compared with the imputation method of the prior art, the present application has the following advantages:

[0194] Firstly, the present application utilizes multi-source data such as traffic flow data, image data and Internet vehicle speed data, which can more comprehensively depict the spatio-temporal characteristics of traffic flow and improve the imputation accuracy compared with methods using only a single data source. Through contrastive learning, the model can learn deeper semantic associations between different modal data, such as the relationship between traffic congestion and camera images, further improving the imputation accuracy.

[0195] In addition, the present application uses a spatially adaptive diffusion model, which can adaptively learn the spatial correlation of different positions and different time periods, and can more accurately depict the complex and variable traffic spatio-temporal characteristics compared with methods that assume fixed spatial correlation, further improving the imputation accuracy.

[0196] Finally, the present application can provide the confidence interval of the imputation value, which can better evaluate the uncertainty of the imputation result compared with methods that only provide point estimates, and provide more reliable basis for traffic management decisions.

[0197] Since the data processing method in the present application can be used in intelligent security, smart city and other fields to perform task processing (such as traffic management, traffic prediction, etc.), the following will take traffic data as an example to introduce multiple application scenarios landing to products.

[0198] (1) Intelligent Traffic System (ITS)

[0199] Traffic flow prediction: Accurately predict the traffic flow in the future period of time, provide data support for traffic signal control, traffic guidance, traffic incident detection, etc. The embodiments of the present application can integrate traffic data from multiple sources, such as loop detector data, floating car data, video detection data, etc., to improve traffic management efficiency.

[0200] Traffic speed prediction: Accurately predict the traffic speed in the future period of time, provide data support for travel route planning, congestion warning, traffic accident prevention, etc. The embodiments of the present application can combine historical speed data, real-time traffic flow, road event information, etc., to more accurately predict traffic speed and improve traffic travel experience.

[0201] Missing value imputation: Due to sensor failure, communication interruption, etc., there are often missing values in traffic data. The embodiments of the present application can use the spatio-temporal correlation between multi-source data to impute missing traffic flow, speed, etc. data, ensure the integrity and accuracy of the data, and improve the reliability of traffic data analysis.

[0202] (2) Highway traffic flow prediction

[0203] Toll station flow prediction: Accurately predict the import and export flow of the highway toll station in the future period of time, provide decision basis for toll station resource scheduling, congestion relief, travel information release, etc. The embodiments of the present application can combine historical flow data, holiday information, weather conditions, etc. to more accurately predict the toll station flow and improve the efficiency of highway operation.

[0204] Road section speed prediction: Accurately predict the average speed of each road section of the highway in the future period of time, provide data support for traffic incident detection, speed guidance, variable speed limit, etc. The embodiments of the present application can integrate various sensor data, road condition information, weather information, etc. to more accurately predict the road section speed and ensure the smoothness of the highway.

[0205] (3) Urban traffic flow prediction

[0206] Regional flow prediction: Accurately predict the traffic flow of each region in the city in the future period of time, provide data support for traffic signal coordination control, traffic congestion governance, public transportation scheduling, etc. The embodiments of the present application can combine taxi trajectory data, shared bicycle data, bus data, etc. to more comprehensively predict regional traffic flow and improve the efficiency of urban traffic operation.

[0207] Intersection traffic prediction: Accurately predict the traffic flow of each intersection in the city in the future period, provide data support for traffic signal optimization, traffic guidance, traffic accident prevention, etc. The embodiments of the present application can integrate various sensor data, traffic flow model, traffic event information, etc., more finely predict the intersection flow, and ensure the order of urban traffic.

[0208] (4) Network traffic prediction

[0209] Link traffic prediction: Accurately predict the traffic load of each link in the network in the future period, provide decision basis for network resource allocation, traffic scheduling, network congestion control, etc. The embodiments of the present application can combine historical traffic data, user behavior patterns, network topology structure, etc., more accurately predict the link traffic, and improve the network resource utilization rate.

[0210] Server load prediction: Accurately predict the load of the server in the future period, provide data support for server resource scheduling, load balancing, service quality guarantee, etc. The embodiments of the present application can integrate various monitoring indicators, user access patterns, server performance parameters, etc., more accurately predict the server load, and ensure the stability and reliability of network service.

[0211] In addition to the field of transportation, the data processing method proposed in the present application can also be applied to other time and space sequence data that need to be processed, and the fields that face the challenges of data missing and multi-source heterogeneous information fusion, such as power load prediction: predict the power demand in the future period, provide data support for power dispatching, power grid planning, energy management. The specific application field of the present application is not limited.

[0212] Based on the above embodiments, in order to better implement the above scheme of the embodiments of the present application, the related equipment for implementing the above scheme is also provided below. Referring to FIG. 8, FIG. 8 is a schematic diagram of a computer device provided by an embodiment of the present application. The computer device 800 can specifically include: an acquisition module 801, a determination module 802, and a data recovery module 803. The acquisition module 801 is configured to acquire a fusion representation code of n types of to-be-processed data on m nodes, the n types of to-be-processed data are synchronous in space-time, m≥2, n≥2; the determination module 802 is configured to determine a first dynamic spatial relationship graph of the m nodes according to the fusion representation code, the first dynamic spatial relationship graph is used to represent that the constructed spatial relationship graph changes with the change of the n types of to-be-processed data in space-time; and the data recovery module 803 is configured to process the first dynamic spatial relationship graph and target data by using a learned denoising function to obtain recovered data, wherein the target data is to-be-processed data of a target type in the n types of to-be-processed data, the target data is time series data, the target data has a data missing condition within a preset time length, and the recovered data is the target data after the data missing condition is interpolated.

[0213] In a possible design, the acquisition module 801 is specifically configured to: construct a first training set according to n types of historical data; train a contrastive learning model by using the first training set to obtain a trained contrastive learning model; and input the n types of to-be-processed data on the m nodes into the trained contrastive learning model to obtain the fusion representation code.

[0214] In a possible design, the acquisition module 801 is specifically further configured to: perform feature extraction on the n types of historical data by using a trained neural network to obtain n types of data features, one type of historical data corresponds to one type of data feature; and construct the first training set according to the n types of data features.

[0215] In a possible design, the n types of data features come from p time windows, p≥2, and the acquisition module 801 is specifically further configured to: construct at least one positive sample pair set by using at least two types of data features from a same time window, and the first training set includes the at least one positive sample pair set.

[0216] In a possible design, the first training set further includes: at least one negative sample pair set, the negative sample pair set is constructed by using at least two types of data features from different time windows.

[0217] In a possible design, the acquisition module 801 is specifically further configured to: train a contrastive learning model by using a contrastive loss function through the first training set.

[0218] In a possible design, the computer device 800 further includes a loop triggering module 804, configured to trigger the data recovery module 803 to repeatedly perform the step of processing the first dynamic spatial relationship graph and the target data by using the learned denoising function to obtain the recovered data until a termination condition is reached, where the imputation values obtained by imputing the data missing in each repeated execution process form the interval estimation of the imputation values.

[0219] In a possible design, the denoising function is obtained by training a diffusion model, and a training process of the diffusion model includes: constructing a second training set, where the second training set includes a second dynamic spatial relationship graph of k nodes and training data, the second dynamic spatial relationship graph of the k nodes is obtained based on fusion representation encoding of j types of historical data on the k nodes, the training data is historical data of a target type in the j types of historical data, the second dynamic spatial relationship graph is used to represent changes in the constructed spatial relationship graph with changes in the j types of historical data in space-time, k≥2, j≥n; training the diffusion model by using the second training set to obtain the denoising function.

[0220] In a possible design, the data recovery module 803 is specifically configured to: add noise to the target data to obtain noisy data; and use the noisy data and the first dynamic spatial relationship graph as input data of the learned denoising function to implement denoising of the noisy data to obtain recovered data.

[0221] In a possible design, the data recovery module 803 is specifically configured to: add Gaussian noise to the target data to obtain noisy data.

[0222] In a possible design, the determination module 802 is specifically configured to: calculate the similarity between the m nodes according to the fusion representation encoding; and construct the first dynamic spatial relationship graph according to the similarity.

[0223] In a possible design, the determination module 802 is specifically configured to: calculate the similarity between the m nodes according to the fusion representation encoding by using a K-nearest neighbor algorithm or a self-attention mechanism based method.

[0224] In a possible design, the to-be-processed data includes traffic data, environmental detection data, meteorological forecast data, or power load prediction data.

[0225] In a possible design, in a case where the to-be-processed data is traffic data, the n types of to-be-processed data at least include any two of the following: traffic flow data, vehicle speed data, image data, video data, and lidar data.

[0226] It should be noted that the information interaction and execution process between the modules / units in the computer device 800 are based on the same concept as the method embodiments described above, and the specific content can be referred to the description of the method embodiments described above, which will not be repeated here.

[0227] Next, another computer device provided by the embodiments of the present application is introduced. Referring to FIG. 9, FIG. 9 is a structural schematic diagram of a computer device provided by the embodiments of the present application. The computer device 900 can be deployed with the modules described in the corresponding embodiments of FIG. 8, and used to implement the functions of the computer device 800 in the corresponding embodiments of FIG. 8. Specifically, the computer device 900 is implemented by one or more servers, and the computer device 900 can be quite different due to different configurations or performances, and can include one or more central processing units (CPU) 922 and memories 932, one or more storage media 930 (such as one or more mass storage devices) for storing application programs 942 or data 944. The memories 932 and the storage media 930 can be temporary storage or persistent storage. The programs stored in the storage media 930 can include one or more modules (not shown in the figure), and each module can include a series of instruction operations on the computer device 900. Further, the central processing unit 922 can be configured to communicate with the storage medium 930 and execute a series of instruction operations in the storage medium 930 on the computer device 900.

[0228] The computer device 900 can also include one or more power supplies 926, one or more wired or wireless network interfaces 950, one or more input and output interfaces 958, and / or one or more operating systems 941, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0229] In the embodiments of the present application, the central processor 922 is configured to execute the steps performed by the computer device in the corresponding embodiments of FIGS. 3-5. For example, the central processor 922 can be configured to: first, obtain the fusion representation codes of n types of to-be-processed data on m nodes, where the n types of to-be-processed data are synchronized in space-time, m≥2, and n≥2. Then, according to the obtained fusion representation codes, determine a dynamic spatial relationship graph (which can be referred to as a first dynamic spatial relationship graph) of the m nodes, where the first dynamic spatial relationship graph is used to represent that the constructed spatial relationship graph changes with the changes of the n types of to-be-processed data in space-time. The first dynamic spatial relationship graph and target data are processed using the learned denoising function to obtain recovered data, where the target data is to-be-processed data of a target type among the n types of to-be-processed data, the target data is time series data, the target data has a data missing condition within a preset time length (for example, traffic flow data in the past year), and the recovered data is the target data after the data missing is interpolated. Finally, the step 503 is repeatedly executed until a termination condition is reached, for example, a preset number of repeated executions (for example, 100 times) is reached. In each repeated execution 503, the interpolation values obtained by interpolating the data missing form an interval estimation of the interpolation values, for example, if the repeated execution is 100 times, for each missing value, 100 interpolation values will be obtained, which form multiple interpolation results of the missing value, and form a corresponding interval estimation, according to which the confidence interval of the interpolation value corresponding to the missing value can be calculated. The method not only can give point estimation of the missing value, but also can provide interval estimation and evaluate the uncertainty of interpolation.

[0230] It should be noted that the specific manner in which the central processor 922 executes the above steps is based on the same concept as the method embodiments corresponding to FIGS. 3-5 of the present application, and the resulting technical effects are the same as those of the above embodiments of the present application. For specific content, refer to the description of the method embodiments described above.

[0231] In addition, it should be noted that the device embodiments described above are only schematic, and the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the device embodiment provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which can be implemented as one or more communication buses or signal lines.

[0232] Those skilled in the art can clearly understand the application by the description of the above embodiments. The application can be realized by means of software and the necessary universal hardware, and of course can also be realized by means of dedicated hardware including special integrated circuit, special CPU, special memory, special components and the like. Generally, the functions completed by computer program can be easily realized by corresponding hardware, and the specific hardware structure for realizing the same function can be various, such as analog circuit, digital circuit or special circuit and the like. However, for the application, the software program implementation is a better embodiment. Based on such understanding, the technical solutions of the application can be embodied in the form of software product, which is stored in a readable storage medium, such as a floppy disk, a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk and the like, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a training device or a network device and the like) to execute the methods described in the embodiments of the application.

[0233] In the above embodiments, all or part of the embodiments can be realized by software, hardware, firmware or any combination thereof. When realized by software, all or part of the embodiments can be realized in the form of a computer program product.

[0234] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions can be transmitted from one website, computer, training device or data center to another website, computer, training device or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave and the like) mode. The computer readable storage medium can be any available medium that can be stored by the computer or a data storage device such as a training device, a data center and the like integrated with one or more available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)) and the like.

Claims

1. A method of processing data, characterized by, The method comprises: obtaining a fusion representation code of n types of to-be-processed data on m nodes, the n types of to-be-processed data being synchronous in space-time, m≥2, n≥2; determining a first dynamic spatial relationship graph of the m nodes according to the fusion representation code, the first dynamic spatial relationship graph being used to represent that the constructed spatial relationship graph changes with the change of the n types of to-be-processed data in space-time; processing the first dynamic spatial relationship graph and target data by using a learned denoising function to obtain recovered data, wherein the target data is to-be-processed data of a target type in the n types of to-be-processed data, the target data is time series data, the target data has data missing within a preset time length, and the recovered data is the target data after the data missing is interpolated.

2. The method of claim 1, wherein, The method comprises: constructing a first training set according to n types of historical data; training a contrastive learning model through the first training set to obtain a trained contrastive learning model; inputting the n types of to-be-processed data on the m nodes into the trained contrastive learning model to obtain the fusion representation code.

3. The method of claim 2, wherein, The method comprises: extracting features of the n types of historical data through a trained neural network to obtain n types of data features, one type of historical data corresponding to one type of data feature; constructing a first training set according to the n types of data features.

4. The method of claim 3, wherein, The n types of data features come from p time windows, p≥2, and the method comprises: constructing at least one positive sample pair set from at least two types of data features from the same time window, and the first training set comprises the at least one positive sample pair set.

5. The method of claim 4, wherein, The first training set further comprises: at least one negative sample pair set, the negative sample pair set being constructed from at least two types of data features from different time windows.

6. The method according to any one of claims 2-5, characterized in that, The method comprises: training the contrastive learning model by using a contrastive loss function through the first training set.

7. The method according to any one of claims 1 to 6, characterized in that, The method further comprises: repeating the step of processing the first dynamic spatial relationship graph and the target data by using the learned denoising function to obtain the recovered data until a termination condition is reached, wherein interpolation values obtained by interpolating the data missing in each repeated execution process constitute an interval estimation of the interpolation values.

8. The method according to any one of claims 1-7, characterized in that, The denoising function is obtained by training a diffusion model, and the training process of the diffusion model comprises: constructing a second training set, the second training set comprising a second dynamic spatial relationship graph of k nodes and training data, the second dynamic spatial relationship graph of the k nodes being obtained based on a fusion representation code of j types of historical data on the k nodes, the training data being historical data of a target type in the j types of historical data, the second dynamic spatial relationship graph being used to represent changes of the constructed spatial relationship graph with changes of the j types of historical data in space-time, k≥2, j≥n; training the diffusion model through the second training set to obtain the denoising function.

9. The method according to any one of claims 1-8, characterized in that, the processing of the first dynamic spatial relationship graph and target data by using the learned denoising function to obtain recovered data comprises: adding noise to the target data to obtain noisy data; the noisy data and the first dynamic spatial relationship graph are used as input data of the learned denoising function to realize denoising of the noisy data and obtain recovered data.

10. The method of claim 9, wherein, the adding of noise to the target data to obtain noisy data comprises: adding Gaussian noise to the target data to obtain noisy data.

11. The method according to any one of claims 1-10, characterized in that, the determination of the first dynamic spatial relationship graph of the m nodes according to the fusion representation code comprises: calculating the similarity between the m nodes according to the fusion representation code; constructing the first dynamic spatial relationship graph according to the similarity.

12. The method of claim 11, wherein, the calculation of the similarity between the m nodes according to the fusion representation code comprises: adopting a K-nearest neighbor algorithm or a self-attention mechanism method to calculate the similarity between the m nodes according to the fusion representation code.

13. The method according to any one of claims 1-12, characterized in that, the to-be-processed data comprises: traffic data, environmental detection data, weather forecast data or power load prediction data.

14. The method of claim 13, wherein, in the case where the to-be-processed data is traffic data, the n types of to-be-processed data at least include any two of the following: traffic flow data, vehicle speed data, image data, video data and laser radar data.

15. A computer device, comprising: comprise: an acquisition module configured to acquire a fusion representation code of n types of to-be-processed data on m nodes, the n types of to-be-processed data being synchronous in space-time, m≥2, n≥2; a determination module configured to determine a first dynamic spatial relationship graph of the m nodes according to the fusion representation code, the first dynamic spatial relationship graph being used to represent changes of the constructed spatial relationship graph with changes of the n types of to-be-processed data in space-time; a data recovery module configured to process the first dynamic spatial relationship graph and target data by using a learned denoising function to obtain recovered data, wherein the target data is to-be-processed data of a target type in the n types of to-be-processed data, the target data is time-series data, the target data has data missing in a preset time length, and the recovered data is the target data after the data missing is interpolated.

16. The apparatus of claim 15, wherein, the acquisition module is specifically configured to: construct a first training set according to n types of historical data; train a contrast learning model through the first training set to obtain a trained contrast learning model; The m nodes are used as input data of the trained contrastive learning model, and n types of to-be-processed data are obtained.

17. The apparatus of claim 16, wherein, The acquisition module is further configured to: The n types of historical data are extracted by the trained neural network to obtain n types of data features, and one type of historical data corresponds to one type of data feature. The first training set is constructed according to the n types of data features.

18. The apparatus of claim 17, wherein, The n types of data features come from p time windows, and p is greater than or equal to 2. The acquisition module is further configured to:

19. The apparatus of claim 18, wherein, The first training set includes at least one positive sample pair set composed of data features of at least two types from the same time window. The first training set further includes:

20. The apparatus of any one of claims 16-19, wherein, At least one negative sample pair set composed of data features of at least two types from different time windows. The acquisition module is further configured to:

21. The apparatus of any one of claims 15-20, wherein, The contrastive learning model is trained by using a contrastive loss function based on the first training set. The device further includes:

22. The apparatus of any one of claims 15-21, wherein, The cyclic triggering module triggers the data recovery module to repeatedly perform the step of processing the first dynamic spatial relationship graph and the target data by using the learned denoising function to obtain recovered data until a termination condition is reached, wherein the interpolation values obtained by interpolating the data loss in each repeated execution process form an interval estimation of the interpolation values. The denoising function is obtained by training a diffusion model, and the training process of the diffusion model includes: A second training set is constructed, the second training set includes a second dynamic spatial relationship graph of k nodes and training data, the second dynamic spatial relationship graph of the k nodes is obtained based on the fusion representation encoding of j types of historical data on the k nodes, the training data is historical data of a target type in the j types of historical data, and the second dynamic spatial relationship graph is used to represent the change of the constructed spatial relationship graph with the change of the j types of historical data in space and time, k is greater than or equal to 2, and j is greater than or equal to n.

23. The apparatus of any one of claims 15-22, wherein, The diffusion model is trained based on the second training set to obtain the denoising function. The data recovery module is specifically configured to: The target data is added with noise to obtain noise-added data.

24. The apparatus of claim 23, wherein, The noise-added data and the first dynamic spatial relationship graph are used as input data of the learned denoising function to realize denoising of the noise-added data and obtain recovered data. The data recovery module is further configured to:

25. The apparatus of any one of claims 15-24, wherein, Gaussian noise is added to the target data to obtain noise-added data. The determination module is specifically configured to: Calculate the similarity between the m nodes according to the fusion representation encoding; 26. The apparatus of claim 25, wherein, Construct the first dynamic spatial relationship graph according to the similarity. The determination module is further configured to:

27. The apparatus of any one of claims 15-26, wherein, Calculate the similarity between the m nodes according to the fusion representation encoding by using a K-nearest neighbor algorithm or a self-attention mechanism-based method. The to-be-processed data includes: Traffic data, environmental detection data, weather forecast data, or power load prediction data.

28. The apparatus of claim 27, wherein, In the case that the to-be-processed data is traffic data, the n types of to-be-processed data at least include any two of the following: Traffic flow data, vehicle speed data, image data, video data, and laser radar data. 29.A computer device, comprising a processor and a memory, wherein the processor is coupled to the memory, and characterized in that: the memory is configured to store a program; the processor is configured to execute the program in the memory, so that the computer device performs the method according to any one of claims 1-14.

30. A computer storage medium, comprising, The computer readable instructions are stored, and the computer readable instructions implement the method according to any one of claims 1-14 when executed by a processor.

31. A computer program product, characterised in that, The computer program product comprises computer readable instructions, and the computer readable instructions implement the method according to any one of claims 1-14 when executed by a processor.

32. A chip comprising a processor and a data interface, characterized in that The processor reads the instructions stored on the memory through the data interface, and executes the method according to any one of claims 1-14.

Images