Data association relationship determination method, system and device, medium and program product

Abstract

The invention discloses a data association relationship determination method, system and device, a medium and a program product applied to an artificial intelligence technology, and the method comprises the steps: taking to-be-associated data of a data association task and all source data in associated data information as node features, and inputting the node features into an association relationship recognition model; performing graph sampling on the incidence relation identification model, aggregating feature information of each sampling node and adjacent neighbor nodes based on a target polynomial, and determining an incidence relation identification result between the sampling nodes corresponding to the current graph sampling according to the current aggregation feature; and determining target data with an association relationship in the association data information for the to-be-associated data according to each association relationship identification result. The problem that task precision and task execution efficiency cannot be considered at the same time in the prior art can be solved, and on the basis of improving the task execution efficiency determined by the data association relationship, it is ensured that a high-precision data association result is output.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, system, electronic device, non-volatile storage medium and computer program product for determining a data association relationship. Background Art

[0002] With the rapid development of artificial intelligence technology, the data involved in the execution of machine learning tasks includes not only ordered Euclidean data, but also complex non-Euclidean data, such as graph data. This type of data contains dependencies between data, which leads to the need to determine the data association relationship.

[0003] With the rapid growth of graph data, related technologies cannot guarantee both the accuracy of determining association relationships and the efficiency of task execution when determining data association relationships for ultra-large graph data, and cannot meet user needs.

[0004] In view of this, it is a technical problem that technicians in this field need to solve while improving the execution efficiency of the data association relationship determination task and ensuring the accuracy of the data association relationship results that are finally output.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] The present invention provides a data association relationship determination method, system, electronic device, non-volatile storage medium and computer program product, which can ensure the accuracy of data association relationship determination results on the basis of improving the execution efficiency of data association relationship determination tasks.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: In one aspect, the present invention provides a method for determining a data association relationship, comprising: A data association task including at least data to be associated and associated data information is received, and each source data of the data to be associated and the associated data information is used as a node feature and input into a pre-trained association relationship recognition model; wherein the association relationship recognition model is based on a graph neural network model, and a target polynomial is used to represent a frequency domain convolution kernel; graph sampling is performed on the association relationship recognition model, and feature information of each sampling node and its adjacent neighbor nodes is aggregated based on the target polynomial, and an association relationship recognition result between each sampling node corresponding to the current graph sampling is determined according to the current aggregated feature; and according to each association relationship recognition result, target data having an association relationship is determined in the associated data information for the data to be associated.

[0008] In a first exemplary embodiment, feature information of each sampling node and its adjacent neighbor nodes is aggregated based on the target polynomial, including: for the current sampling node, edge features between neighbor nodes of the current layer and the adjacent layer, input features of the current layer, and weight parameters of the current layer are used as inputs of the target polynomial, and calculation results of the target polynomial are used as output features of the current layer; wherein the input features of the current layer are output features of the previous layer; according to the output features of the last layer, the aggregate features of the current sampling node are determined, and the current aggregate features are obtained according to the aggregate features of each sampling node sampled in the current graph.

[0009] In a second exemplary embodiment, the target polynomial is a Chebyshev polynomial, and the edge features between the neighbor nodes of the current layer and the adjacent layer, the input features of the current layer, and the weight parameters of the current layer are used as inputs of the target polynomial, including: generating an adjacency list according to the edge features between the neighbor nodes of the adjacent layers, and performing a Laplace transform on the adjacency list to obtain a Laplace matrix; and inputting the Laplace matrix into the Chebyshev polynomial to approximate the adjacency matrix calculated by wavelet.

[0010] In a third exemplary embodiment, edge features between neighbor nodes of a current layer and an adjacent layer, input features of the current layer, and weight parameters of the current layer are used as inputs of the target polynomial, including: multiplying the Laplacian matrix corresponding to the neighbor nodes of the current layer and the adjacent layer by the input features of the current layer to obtain node features of the current layer; and taking the product of the node features and the weight parameters of the current layer as feature information of the current layer, and inputting the result into the target polynomial.

[0011] In a fourth exemplary embodiment, edge features between neighbor nodes of a current layer and an adjacent layer, input features of the current layer, and weight parameters of the current layer are used as inputs of the target polynomial, including: pre-constructing an adjacency matrix table, an input feature matrix table, and an output feature matrix table; the adjacency matrix table includes a first region, a second region, a third region, and a fourth region, the first region and the third region, the second region, and the fourth region are respectively aligned in a vertical direction, the first region and the second region are located in the same horizontal direction, and the third region and the fourth region are located in the same direction; the input feature matrix table includes a fifth region and a sixth region aligned in the vertical direction, and the output feature matrix table includes a seventh region and an eighth region aligned in the vertical direction; the sum of the matrix products of the adjacency matrix table and the input feature table is stored in the corresponding region of the output feature matrix table; the feature information of neighbor nodes with connecting edges is stored in the second region, and the input features of the current layer are stored in the sixth region; and the information of the seventh region of the output feature matrix table is read as the input of the target polynomial.

[0012] In a fifth exemplary embodiment, feature information of each sampling node and its adjacent neighbor nodes is aggregated based on the target polynomial, including: generating sampling subgraph data according to each sampling node and its multi-order neighbor nodes; from outside to inside, transferring node features of each layer of the sampling subgraph data to node features of the previous layer in sequence based on the target polynomial until they are transferred to each sampling node, thereby obtaining aggregate features of each sampling node corresponding to the current graph sampling.

[0013] In a sixth exemplary embodiment, determining the association relationship identification result between the sampling nodes corresponding to the current graph sampling based on the current aggregation features includes: inputting the current aggregation features corresponding to the current graph sampling into the fully connected layer, and obtaining the association relationship identification result between the sampling points corresponding to the current graph sampling based on the output of the fully connected layer.

[0014] In the seventh exemplary implementation, it also includes: when a computing scale adjustment instruction is received, a target sampling number and / or a target sampling layer number is obtained by parsing the computing scale adjustment instruction; based on the target sampling layer number and the target sampling number, a corresponding number of nodes are sampled on the target layer of the association relationship identification model to obtain sampling nodes for current graph sampling.

[0015] In an eighth exemplary embodiment, the training process of the association relationship identification model includes: obtaining an association task graph data training set; each graph sample data of the association task graph data training set has a label that marks the association relationship; using each graph sample data of the association task graph data training set as a node, performing graph sampling on the association task graph data training set to obtain sampling data; constructing multiple training sub-processes, each training sub-process reads a corresponding number of sampling nodes and their adjacent neighbor nodes from the sampling data according to a preset batch size, aggregates feature information of each sampling node and its adjacent neighbor nodes based on a target polynomial, and determines the association relationship identification result between each sampling node corresponding to the current graph sampling according to the current aggregated feature; based on the association relationship identification result of each training sub-process, obtains the predicted label of each graph sample data of the association task graph data training set.

[0016] The present invention also provides an electronic device, comprising a processor, wherein the processor is used to implement the steps of the data association relationship determination method as described in any of the preceding items when executing a computer program stored in a memory.

[0017] The present invention also provides a non-volatile storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method for determining a data association relationship as described in any of the preceding items are implemented.

[0018] The present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the method for determining a data association relationship as described in any of the preceding items.

[0019] Finally, the present invention also provides a data association relationship determination system, including a first computing device and a second computing device; the first computing device and the second computing device are connected through a target bus, and the second computing device has parallel computing capabilities; wherein the first computing device is configured to receive a data association task that includes at least data to be associated and associated data information, and send the data to be associated and the associated data information to the second computing device through a target bus interface, and the second computing device is configured to perform graph sampling on the association relationship identification model, aggregate the feature information of each sampling node and its adjacent neighbor nodes based on the target polynomial, determine the association relationship identification result between each sampling node corresponding to the current graph sampling according to the current aggregated feature, and send the association relationship identification result to the first computing device through the target bus interface; the association relationship identification model is based on a graph neural network model, and uses the target polynomial to represent a frequency domain convolution kernel; the first computing device is also configured to determine target data with an association relationship in the associated data information for the data to be associated according to each association relationship identification result.

[0020] In a first exemplary embodiment, the second computing device includes a controller, a data storage device and an output result buffer; the controller, the data storage device and the output result buffer communicate through an on-chip network, and the second computing device communicates with the target bus interface through the on-chip network; wherein the controller is configured to perform graph sampling on the association relationship recognition model, aggregate feature information of each sampling node and its adjacent neighboring nodes based on the target polynomial, and determine the association relationship recognition result between each sampling node corresponding to the current graph sampling according to the current aggregated feature; the data storage device is configured to store feature data corresponding to the sampling node and its neighboring nodes sent by the first computing device, and weight parameters of each layer of the association relationship recognition model; the output result buffer is configured to store the association relationship recognition result between each sampling node obtained by each graph sampling.

[0021] In a second exemplary embodiment, the controller of the second computing device includes an instruction executor, a processor array, a nonlinear processor, an intermediate result buffer and a weight data buffer; the instruction executor is respectively connected to the data memory, the processor array and the nonlinear processor, the processor array is respectively connected to the weight data buffer, the intermediate result buffer and the nonlinear processor, and the intermediate result buffer is connected to the output result buffer; the weight data buffer is configured to read the weight parameters of each layer of the association relationship recognition model from the data memory; the instruction executor, upon receiving the write start instruction, reads the adjacency list from the data memory, and sends an adjacency matrix generation instruction carrying the adjacency list to the processor array, and the processor array performs lapping on the adjacency list. The instruction executor calculates the transformation and stores the adjacency matrix in the intermediate result buffer; the instruction executor reads the input features of each layer from the data storage device, and reads the weight parameters of the corresponding layer from the weight data buffer, and sends a node feature information generation instruction to the processor array at the same time, the processor array calculates the product of the input features of the current layer and the weight parameters, and stores it as the node feature information of the current layer in the intermediate result buffer; the instruction executor sends an aggregate feature generation instruction to the processor array, the processor array inputs the node feature information and the adjacency matrix into the target polynomial for calculation, outputs the calculation result of the target polynomial to the nonlinear processor for nonlinear processing, obtains the aggregate feature of the current layer, and stores the aggregate feature of the current layer in the intermediate result buffer.

[0022] The advantage of the technical solution provided by the present invention is that when executing a data association task, a small batch of nodes are collected from large-scale graph data for processing through a graph sampling method, and the neighbor node features of the sampled nodes are involved in the calculation. This can not only reduce the memory requirements of large-scale graph data by reducing the number of nodes and edges to be processed, but also maintain the characteristics of local convolution operations to ensure the accuracy of the association task results. Furthermore, the association relationship recognition model uses a target polynomial instead of a convolution kernel in the spectral domain, and approximates the spectrum filter through polynomial approximation, thereby reducing the computational overhead during the execution of the association task, improving the execution efficiency of the data association task, and reducing the computational cost and memory consumption during the execution of the association task, thereby ensuring the high-precision data association relationship results of the final output on the basis of improving the execution efficiency of the data association relationship determination task.

[0023] In addition, the present invention also provides a corresponding implementation system, electronic device, non-volatile storage medium and computer program product for the method of determining data association relationships, which further makes the method more practical. The system, electronic device, non-volatile storage medium and computer program product have corresponding advantages.

[0024] The technical features mentioned above, the technical features to be mentioned below, and the technical features shown separately in the drawings can be arbitrarily combined with each other, as long as the combined technical features are not contradictory. All feasible feature combinations are technical contents clearly recorded in this article. Any of the multiple sub-features contained in the same sentence can be applied independently, and does not have to be applied together with other sub-features. It should be understood that the above general description and the detailed description below are only exemplary and cannot limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 A schematic diagram of a flow chart of a method for determining a data association relationship provided by the present invention; Figure 2 A schematic diagram of an exemplary model structure of the association relationship identification model provided by the present invention; Figure 3 A schematic diagram of a graph convolution operation in an exemplary application scenario provided by the present invention; Figure 4A schematic diagram of a two-dimensional convolution operation in an exemplary application scenario provided by the present invention; Figure 5 A schematic diagram of graph sampling in an exemplary application scenario provided by the present invention; Figure 6 A schematic diagram of feature aggregation in an exemplary application scenario provided by the present invention; Figure 7 A schematic diagram of feature aggregation in another exemplary application scenario provided by the present invention; Figure 8 A schematic diagram of message propagation in an exemplary application scenario provided by the present invention; Fig. 9 A schematic diagram of the training process of the association relationship recognition model provided by the present invention in an exemplary application scenario; Fig.10 A structural framework diagram of an exemplary implementation of the data association relationship determination device provided by the present invention; Fig.11 A structural diagram of an exemplary embodiment of an electronic device provided by the present invention; Fig.12 A structural diagram of an exemplary implementation of a data association relationship determination system provided by the present invention; Fig.13 A schematic diagram of an exemplary structural framework of a second computing device provided by the present invention; Fig.14 A schematic diagram of an exemplary data processing flow of the controller provided by the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Among them, the terms "first", "second", "third", "fourth", etc. in the specification and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations of the two are intended to cover non-exclusive inclusions. The term "exemplary" means "used as an example, embodiment or illustrative". Any embodiment described here as "exemplary" is not necessarily interpreted as being superior or better than other embodiments.

[0028] In the field of artificial intelligence technology, machine learning tasks such as image classification, speech recognition, and natural language processing process Euclidean data with a certain size, dimension, and ordered arrangement. There is also a type of machine learning task that processes data represented by complex non-Euclidean data, such as graph data. This type of data not only contains the data itself, but also includes the dependencies between data, such as social networks, protein molecular structures, customer data in e-commerce platforms, etc. For related fields of this type of data and applications, such as social media, e-commerce, knowledge graphs, etc., related technologies usually use graph neural networks to perform corresponding tasks.

[0029] As the scale of graph data continues to increase, the scale of graph neural network also continues to increase, and its computing power and storage requirements for hardware are getting higher and higher, and the software implementation efficiency of its algorithm is very low. For ultra-large-scale graph data, although the related technology uses hardware acceleration methods to solve its demand for computing resources, it cannot guarantee the prediction accuracy. It can be seen that the related technology still has the problem of not being able to take into account the computing power requirements, memory requirements, and prediction accuracy loss when using graph neural network to process data association tasks. In view of this, in the process of data association relationship identification, the present invention controls the number of nodes and edges of the graph currently involved in the calculation through graph sampling, reduces the computing cost and memory consumption, and adopts the target term calculation method to approximate the wavelet calculation of the frequency domain of the graph neural network to ensure the task execution accuracy, which not only accelerates the processing efficiency of the association relationship identification model for graph data, but also introduces neighbor node features to ensure the accuracy of the association relationship identification model for graph data. On the basis of improving the execution efficiency of the data association relationship determination task, it can ensure the accuracy of the data association relationship result output in the end. After introducing the technical scheme of the present invention, various non-limiting embodiments of the present invention are described in detail below. In order to better illustrate the present invention, many specific details are given in the specific embodiments below. Those skilled in the art will appreciate that the present invention can be implemented without these specific details. In other examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present invention.

[0030] First see Figure 1 , Figure 1 A flowchart of a method for determining a data association relationship provided in this embodiment may include the following contents: S101: receiving a data association task including at least data to be associated and associated data information, and inputting source data of the data to be associated and the associated data information as node features into a pre-trained association relationship recognition model.

[0031] In this step, the data association task is a user task that needs to be executed. When the user issues the task, at least the data to be associated and the associated data information need to be issued at the same time. The data to be associated is the execution target of the data association task, that is, the association relationship is identified for the target. The associated data information is the execution scope of the data association task, that is, the data with an association relationship with the data to be associated is identified within which scope. The associated data information is usually a series of data, such as a data set or a database. For the convenience of description, each data in it is defined as source data. The data to be associated and the associated data information can be text data, image data, audio and video data. The data association task can be the association relationship between text data, such as the citation relationship between papers, or the association relationship between text data and audio and video data, such as emotion recognition tasks and subtitle matching tasks. The emotion recognition task is to determine the emotion recognition result from the emotion description text database based on audio data or video data. It can also be the association relationship between text data and image data, such as gene annotation tasks based on gene expression data and text-gene matching tasks. Of course, the association relationship between audio and video data and image data can also be identified, such as similar video retrieval tasks and similar image retrieval tasks in duplicate checking tasks.

[0032] In this embodiment, the association relationship identification model is built based on the graph neural network model, that is, its essence is a graph neural network model. The association relationship identification model can represent a bunch of entities and the relationship between each entity. The entity is a node of the graph, and the relationship between the entity and the entity constitutes the edge of the graph. If there is an edge between the two, it means that the two entities have a connection relationship. The association relationship identification model can be expressed as G={V, E}, G is the association relationship identification model, V represents the node set, and E represents the edge set. Taking a social network as an example, a user can be a node, and the friend relationship between users can be an edge. Correspondingly, each source data of the data to be associated and the associated data information is a node feature of the association relationship identification model. The association relationship identification model identifies which source data in the associated data information have edges with the data to be associated by performing graph data processing on each source data of the data to be associated and the associated data information, thereby determining the target data that has an association relationship with the data to be associated. The target data is the source data in the associated data information that has an association relationship with the data to be associated. Taking emotional tasks as a class, the association recognition model can process unstructured information in emotional data, such as emotional tendency and emotional intensity. This information exists in the form of nodes and edges in the graph structure. By mapping the emotional data to the graph neural network structure, the powerful ability of the graph neural network structure of the association recognition model is used to capture the complex relationships and patterns between emotions. Furthermore, each node of the association recognition model also has a vertex degree, that is, the number of edges associated with the node. In a directed graph, the degree of a node is equal to the sum of the in-degree and out-degree of the node. The in-degree of a node is the number of directed edges with the node as the end point, and the out-degree of a node is the number of directed edges with the node as the starting point. For an undirected graph, the degree of a node is equal to the in-degree of the node and also equal to the out-degree of the node.

[0033] Exemplarily, the association relationship recognition model may include at least an input layer, a graph convolution module and an output layer, wherein the graph convolution module is a combination of multiple graph convolution layers and activation function layers, that is, each graph convolution layer is connected to an activation function layer, and the activation function layer of the current layer is connected to the graph convolution layer of the next layer. The activation function layer may be, for example, Relu (Linear rectification function), such as Figure 2 As shown, the graph convolution module may include a first graph convolution layer, a first activation function layer, a second graph convolution layer, and a second activation function layer, and the output layer may be a fully connected layer. The graph structure data, the data to be associated, and the node feature data corresponding to each source data are input to the first graph convolution layer through the input layer, as shown in FIG. Figure 3As shown in the figure, the first graph convolution layer uses the nearest neighbor function to sample the features of the nodes around the current node, calculates the average value, and processes the intermediate result through the first activation function layer. The output of the first activation function layer is then input into the second graph convolution layer until the correlation relationship between the data to be associated and the source data is output. The graph convolution layer of the graph convolution module is different from the two-dimensional convolution layer. During the processing, the neighbor nodes of the current node are uncertain and unordered, belonging to non-Euclidean data, while the two-dimensional convolution layer uses the convolution kernel to sample the features of the surrounding nodes and calculate the weighted average value. The number of its neighboring nodes is certain and ordered, belonging to Euclidean data, such as Figure 4 As shown. The graph convolution module realizes the aggregation of node features and also needs to perform graph sampling operations. Accordingly, it can also include the network model structure used for the corresponding graph sampling. Taking DGL sampling as an example, the graph convolution module can also include the Blocks subgraph structure of DGL, which is a unidirectional subgraph. Of course, the association relationship recognition model can use any graph neural network model structure, and the present invention does not make any limitation to this. Accordingly, any matching training method can be used to train the association relationship recognition model accordingly.

[0034] Among them, the frequency domain processing process of the graph neural network is based on the graph Fourier transform to realize the conversion of the graph signal from the spatial domain to the frequency domain, and then perform the graph convolution operation in the frequency domain. The graph Fourier transform uses the eigenvectors and eigenvalues ​​of the graph Laplacian matrix as the basis function and frequency to convert the graph signal from the spatial domain to the frequency domain. When calculating in the frequency domain, the spatial domain signal and the convolution kernel are converted to the frequency domain for multiplication, and then the result is converted back to the spatial domain through the inverse Fourier transform. For example, GCN (Graph Convolutional Network) converts the input signal and the convolution kernel parameters to the frequency domain through the graph Fourier transform, multiplies them, and then obtains the convolution result through the inverse transform. Although the association recognition model can adopt this frequency domain calculation method, as the scale of the graph data to be processed increases, that is, the number of source data contained in the association information increases, the frequency domain calculation of the graph structure of the association recognition model becomes more and more difficult. In order to meet the processing of large-scale data, the scale of the association recognition model will increase accordingly. As the number of layers of the association recognition model increases, the computational cost increases exponentially. Saving the information of the entire graph and the representation of each node in each layer requires a large amount of memory space, which leads to a large demand for computing resources and memory resources. If the method does not use the entire graph information and the representation of each node in each layer, the execution accuracy of the data association task will be lost. The traditional spectral domain graph convolution is a graph convolution method defined in the frequency domain. Although the graph convolution can be intuitively understood and has good local properties, the computational complexity of the basic spectral domain graph convolution is usually high due to the need for Fourier transform, inverse transform and matrix decomposition, which are computationally expensive frequency domain operations. Especially when processing large graph data. In addition, the parameters in the spectral domain method, such as the convolution kernel size and sampling method, have a great impact on the performance, but adjusting these parameters usually requires certain experience or attempts, which are difficult to understand intuitively. As the amount of graph structure data is too large, it is difficult to directly operate the parameters of the convolution kernel, and feature decomposition is required. Although the graph wavelet basis can be calculated using graph Laplace eigendecomposition, the adjacency matrix of the entire graph structure is generally an N×N matrix, where N is the order of the adjacency matrix of the graph structure. When the network uses wavelet calculation methods to calculate the adjacency matrix of the entire graph structure, it is time-consuming and occupies too much memory and video memory resources. For very large graph data, the adjacency matrix cannot be calculated. This shows that this method also has a high computational cost and cannot handle disconnected graphs.Therefore, in order to take into account the efficiency, computational cost and task accuracy of the association relationship identification model in performing data association tasks, the association relationship identification model of the present invention adopts a target polynomial to represent the frequency domain convolution kernel. The target polynomial can use, for example, Bernstein polynomials, Chebyshev polynomials, and least squares method, which can represent the convolution kernel in the spectral domain in the form of a polynomial. By using the target polynomial as an approximation function, the complex calculation of the original function can be converted into a simple calculation of the polynomial, thereby speeding up the calculation speed and achieving efficient convolution operations, that is, approximating the spectrum filter by polynomial approximation, and approximating the convolution kernel in the spectral domain by the target polynomial to optimize the traditional spectral domain convolution operation, avoiding the complex feature decomposition process, reducing the computational complexity and amount of calculation, thereby reducing the computational overhead and improving the efficiency of task execution. In addition, the polynomial approximation method not only reduces the computational complexity, but also retains the ability of local aggregation, making the association relationship identification model more effective in processing graph data, and improving the accuracy of the association relationship finally output by the association relationship identification model. In addition, using the target polynomial instead of the convolution kernel in the spectral domain can not only replace the time-consuming and resource-consuming wavelet calculation, but also facilitate its application on FPGA (Field Programmable Gate Array) with limited resources.

[0035] S102: Perform graph sampling on the association relationship recognition model, aggregate feature information of each sampling node and its adjacent neighbor nodes based on the target polynomial, and determine the association relationship recognition result between the sampling nodes corresponding to the current graph sampling according to the current aggregated features.

[0036] Based on the wavelet calculation in the frequency domain of the graph neural network approximated by the target polynomial calculation method in this embodiment, in order to further reduce the demand for computing resources and storage resources of the association relationship recognition model during data processing, this step also performs graph sampling on the association relationship recognition model, and significantly reduces the memory demand and computing demand for large-scale graphs by processing small batches of nodes one by one. In order to further improve the execution accuracy of data association tasks and maintain the characteristics of local convolution operations, multiple neighbor node features are also obtained at the same time to participate in the graph convolution operation, making full use of the local information of each node. Any graph sampling method can be used for graph sampling. The graph can use DGL (Deep Graph Library, graph sampler) to sample the association relationship recognition model. Of course, other sampling methods can also be used, such as local message compensation. This method can make up for the problem of missing edge node neighbors caused by subgraph sampling, ensure that the performance of the model trained on the subgraph is equivalent to the full graph, and solve the problem of precision loss. Through graph sampling, the association relationship recognition model can significantly reduce the number of nodes and edges processed, thereby reducing computing costs and memory consumption, and effectively dealing with computing power and memory problems. In addition, although graphs are inherently sparse structures, graph sampling can convert graph data into a format that is more suitable for computing devices such as FPGAs and graphics processors, improve computing efficiency, and thus alleviate the problems caused by hardware limitations to a certain extent. By reducing computational complexity and memory usage, data association tasks can be completed in a reasonable time, maintaining high performance even in resource-constrained environments.

[0037] Among them, when sampling the association relationship recognition model, for example, L layers can be sampled, and dozens of nodes can be sampled in each layer. In order to embed the target node, the target node is the sampling node participating in the current calculation, and a low-dimensional vector representation of the graph is obtained. The convolution kernel in the spectral domain is expressed using the target polynomial, and the node features of each layer are continuously aggregated to the target node to obtain the aggregated features of the target node. The aggregation operation is performed on each sampling node obtained by sampling the graph once in turn, and the aggregated features of all sampling nodes obtained by sampling the graph once are used as the current aggregated features, that is, the aggregated features obtained by sampling the graph once are defined as the current aggregated features. According to the current aggregated feature information, the labels of the sampling nodes obtained by the current graph sampling can be predicted, that is, whether there is an association relationship between the sampling nodes. Repeat this process until all nodes of the association relationship recognition model are sampled, and the labels of all nodes of the association relationship recognition model are obtained.

[0038] S103: According to the identification results of each association relationship, target data having an association relationship is determined in the associated data information for the data to be associated.

[0039] After the labels of each node of the association relationship identification model are obtained in the previous step, the data with an association relationship is determined for the data to be associated according to whether there is an association relationship between the nodes. Exemplarily, the fully connected layer can be used to perform graph task identification on the features, and accordingly, the current aggregated features obtained for each graph sampling can be input into the fully connected layer, that is, the current aggregated features corresponding to the current graph sampling are input into the fully connected layer, and the association relationship identification result between the sampling points corresponding to the current graph sampling is obtained according to the output of the fully connected layer.

[0040] In the technical solution provided in this embodiment, when executing the data association task, a small batch of nodes are collected from large-scale graph data for processing through a graph sampling method, and the neighbor node features of the sampled nodes are involved in the calculation. This can not only reduce the memory requirements of large-scale graph data by reducing the number of nodes and edges to be processed, but also maintain the characteristics of local convolution operations to ensure the accuracy of the association task results. Furthermore, the association relationship recognition model uses a target polynomial instead of a convolution kernel in the spectral domain, and approximates the spectral filter through polynomial approximation, thereby reducing the computational overhead during the execution of the association task, improving the execution efficiency of the data association task, and reducing the computational cost and memory consumption during the execution of the association task, thereby improving the execution efficiency of the data association relationship determination task, and ensuring the high-precision data association relationship results of the final output.

[0041] Considering that the network result of graph sampling implemented by the graph convolution module of the association relationship recognition model is a unidirectional subgraph, and the graph data belongs to a non-Euclidean data type, the matrix is ​​sparse, and the node position is random, if the node aggregation method of message propagation is used, it will lead to irregular storage access and difficulty in reusing data. Based on the above embodiment, the present invention also provides an exemplary aggregation method, which may include the following content: For the current sampling node, the edge features between the neighbor nodes of the current layer and the adjacent layer, the input features of the current layer and the weight parameters of the current layer are used as the input of the target polynomial, and the calculation result of the target polynomial is used as the output feature of the current layer; wherein, the input feature of the current layer is the output feature of the previous layer; according to the output feature of the last layer, the aggregation feature of the current sampling node is determined, and the current aggregation feature is obtained according to the aggregation features of each sampling node sampled in the current graph.

[0042] Among them, the edge features between the neighbor nodes of the current layer and the adjacent layer are used as the feature data for generating the adjacency matrix. The adjacency matrix, the input features of the current layer and the weight parameters of the current layer are input into the target polynomial, and the target polynomial is calculated to obtain the output features of the current layer. The output features of the current layer are input into the next layer as the next input features. At the same time, the edge features between the neighbor nodes of the next layer and its adjacent layer and the weight parameters of the next layer are input and continuously passed until the last layer is reached. The output features of the last layer are the aggregate features of the current sampling node.

[0043] From the above, it can be seen that this embodiment significantly reduces the memory requirement for large-scale graphs by processing small batches of nodes and their neighbors one by one, while maintaining the characteristics of local convolution operations, which is more in line with the needs of small batch processing of large-scale graph data in practical applications. At the same time, when implementing graph convolution, it fully utilizes the local information of each node, which can not only improve the execution accuracy of data association tasks, but also improve the execution accuracy of data association tasks.

[0044] The above embodiment illustrates the aggregation process from the perspective of the input and output of the target polynomial and the data flow of the network model structure of the association relationship recognition model. This embodiment also describes the aggregation process from another perspective, which may include the following content: According to each sampling node and its multi-order neighbor nodes, the sampling subgraph data is generated. From the outside to the inside, the node features of each layer of the sampling subgraph data are transferred to the node features of the previous layer in sequence based on the target polynomial until they are transferred to each sampling node, thereby obtaining the aggregated features of each sampling node corresponding to the current graph sampling.

[0045] In this embodiment, the association relationship recognition model is sampled at L layers to obtain multiple sampling nodes, and the L-order neighbor nodes of each sampling node are obtained to form the sampled subgraph data. For the convenience of description, it is defined as the sampling subgraph data, and the features after the aggregation of these multiple sampling nodes are obtained from the outermost layer to the inner layer. Figure 5 As shown in the figure, taking L=2 and the target polynomial as Chebyshev polynomial as an example, the first-order neighbor nodes and second-order neighbor nodes of each sampling node are obtained, and the edges between the second-order neighbor nodes and the first-order neighbor nodes are used to build an adjacency table, which is Laplace transformed and put into the Chebyshev polynomial to approximate the adjacency matrix of wavelet calculation. The edge information between the first-order neighbor nodes and the sampling nodes is used to perform the above operations, and the outermost node features are passed to the sampling nodes layer by layer to obtain the aggregated feature information of the sampling nodes. Figure 6 As shown in the figure, the aggregation of label 0 neighbors and the aggregation of label 1 neighbors represent the output of aggregation features of different orders, which are aggregated from the outside to the inside, aggregating the second-order neighbor features to the first-order neighbors, and then aggregating the first-order neighbors to the current sampling node.

[0046] As can be seen from the above, this embodiment designs an L-layer network structure, corresponding to the L-layer neighbors of the sampling node, and uses matrix product to aggregate from the outside to the inside, which is conducive to running the association relationship recognition model on the low-frequency multi-computing unit.

[0047] The above embodiment does not impose any restrictions on the type of the target polynomial. For example, the target polynomial can be a Chebyshev polynomial, and the frequency domain convolution kernel is represented by a Chebyshev polynomial. The convolution kernel has only K+1 learnable parameters. Generally speaking, K is much smaller than N. K and N are both constants. N is the order of the traditional graph structure adjacency matrix, and the complexity of the parameters is greatly reduced. After the Chebyshev polynomial is used to replace the spectral domain convolution kernel, there is no need to perform eigendecomposition operation on the Laplace matrix, and the most time-consuming step is omitted. The convolution kernel has strict spatial locality, and K is the "receptive field radius" of the convolution kernel, that is, the K-order neighbor node of the central vertex is used as the neighborhood node. For example, the above embodiment does not impose any restrictions on the generation of the adjacency matrix. This embodiment can generate an adjacency list based on the edge features between the neighbor nodes of the adjacent layers, and perform Laplace transformation on the adjacency list to obtain a Laplace matrix, that is, the Laplace matrix is ​​used as the adjacency matrix, and the Laplace matrix is ​​input into the Chebyshev polynomial to approximate the adjacency matrix of the wavelet calculation.

[0048] For example, in order to further improve the task execution efficiency, the Laplace calculation relation can be stored in advance, and the adjacency list is Laplace transformed using the Laplace transformation calculation relation to obtain a Laplace matrix, wherein the Laplace calculation relation is as shown in the following relation (5). The process of determining the Laplace calculation relation can be: The Laplace operator is a second-order differential operator for functions in n-dimensional Euclidean space. It is a scalar. The Laplace operator of a two-dimensional function f(x, y) can be expressed as the following equation (1). The first-order derivative of a discrete function f(x, y) can be expressed as the following equation (2) and the second-order derivative can be expressed as the following equation (3) based on the interval between two adjacent independent variables being 1. Correspondingly, the Laplace operator of a one-dimensional discrete function f(x) can be expressed as the following equation (4), and the Laplace operator of a two-dimensional discrete function f(x, y) can be expressed as the following equation (5): ; (1) ; (2) ; (3) ; (4) ; (5) In the formula, x and y represent the coordinate values ​​in two-dimensional data. Represents the Laplace operator, and the Laplace operator solution is implemented on the adjacency list.

[0049] For example, in order to further improve the task execution efficiency, the Chebyshev calculation relation may be stored in advance, the Laplace matrix is ​​input into the Chebyshev calculation relation and the calculation is performed. The generation process of the Chebyshev calculation relation may include the following contents: First, the Chebyshev polynomial can be expressed as the following relation (6): ; (6) In the formula, k represents the order of Chebyshev polynomial, x represents the input feature vector, is a Chebyshev polynomial of order k. For the graph wavelet convolution operation, this embodiment uses Chebyshev polynomials to approximate the graph Laplacian filter g(A), given a graph Laplacian matrix L and the maximum eigenvalue , graph convolution can be approximated as the following relation (7): ; (7) in, , are the coefficients of the Chebyshev polynomial, K is the maximum order of the Chebyshev polynomial that can be set, the value of K is greater than or equal to k, I represents the diagonal matrix of the adjacency matrix, It is the formula calculation result of matrix L. Taking the input signal as X, is the calculation result of X. The operation of graph convolution can be expressed as the following relation (8): ; (8) Based on equation (8), each term of equation (6) The Chebyshev calculation relation can be obtained by recursive calculation. The Chebyshev calculation relation can be expressed as the following relation (9): . (9)

[0050] As can be seen from the above, this embodiment can use discrete Laplace approximation calculations for complex calculations such as derivation in Laplace calculations, simplify the calculation process and amount of calculations, not only reduce the task execution time of the association relationship recognition model, but also facilitate application in limited resource application scenarios. By using Chebyshev polynomials instead of convolution kernels in the spectral domain and time-consuming and resource-consuming wavelet calculations, dynamic, new, and unknown node types in large-scale graphs are predicted, and graphs with large numbers of nodes and rich node features are optimized to reduce memory overhead and computing overhead, which is also conducive to application in limited resource scenarios.

[0051] Exemplarily, the Laplacian matrix corresponding to the neighbor nodes of the current layer and the adjacent layer can be multiplied with the input features of the current layer to obtain the node features of the current layer; the product of the node features and the weight parameters of the current layer is used as the feature information of the current layer and input into the target polynomial.

[0052] For example, take DGL sampling of batch (batch size) nodes in the association recognition network and take the L-order neighbor nodes of the sampled nodes as an example. Figure 7 As shown in L=2. After graph sampling in S102, an L-layer network can be obtained, and L Blocks (block information) data blocks can be constructed, which at least include input node features, output node features, and unidirectional edge information. Each layer of the network constructs an adjacency table through the corresponding Block information, and the adjacency matrix L is obtained by Laplace calculation. The feature X of the input node is multiplied by the weight data of each layer to obtain the feature , L and The output features of each layer are calculated by feeding them into the Chebyshev polynomial approximate graph convolution formula, which are the output node features after the input node features are aggregated. The features of each layer are aggregated in turn to obtain the aggregated features of batch sampling nodes. Classification is performed through the fully connected layer to obtain the classification results of batch nodes.

[0053] From the above, it can be seen that this embodiment optimizes the network structure and the operator as a matrix product, which is conducive to the efficient application of the association relationship recognition model on a low-clock multi-core computing device, such as an FPGA, and is therefore suitable for use in scenarios with limited resources.

[0054] It can be understood that the graph convolution network updates the feature vector by multiplying the node feature vector by the adjacency matrix, so that the feature vector can contain the information of the neighboring nodes. This process is essentially part of message propagation, that is, the mechanism of information transmission and update in the graph neural network. Message propagation allows nodes to update the state or representation of nodes by exchanging information, which is usually based on the adjacency matrix and node features, and can be implemented by summation, weighted summation, etc. Matrix product and message propagation work together in the graph convolution network to achieve effective processing and analysis of graph data. Although this method helps to capture the structural information of graph data and the complex relationship between nodes, it is necessary to process the information exchange of multiple single nodes when implementing it for computing devices such as FPGAs. Frequent processing of small amounts of data is not conducive to the low-clock multi-computing core method. In order to meet its normal operation in actual hardware devices, this embodiment uses a sampled adjacency table to calculate the neighbor matrix for a one-way message propagation method, which may include the following: Pre-construct an adjacency matrix table, an input feature matrix table, and an output feature matrix table; store the feature information of neighbor nodes with connecting edges in the second area, and store the input features of the current layer in the sixth area; read the information in the seventh area of ​​the output feature matrix table as the input of the target polynomial.

[0055] Among them, Figure 8 As shown, the adjacency matrix table includes the first area A1, the second area A2, the third area A3 and the fourth area A4. The first area A1 and the third area A3, the second area A2 and the fourth area A4 are aligned in the vertical direction, respectively. The first area A1 and the second area A2 are located in the same horizontal direction, and the third area A3 and the fourth area A4 are located in the same direction. The association relationship recognition model is sampled, and the corresponding features of dst (target node) and src (source node) are obtained and placed in the A2 area of ​​the adjacency matrix table. In this way, only the A2 area has a value, and the A1, A3, and A4 areas are all 0. The input feature matrix table includes a fifth region X1 and a sixth region X2 aligned in the vertical direction, and the output feature matrix table includes a seventh region O1 and an eighth region O2 aligned in the vertical direction; each region is taken as an element of the matrix, and the data of the adjacency matrix table and the input feature table are subjected to matrix multiplication, and the sum of the matrix products of the adjacency matrix table and the input feature table is stored in the corresponding region of the output feature matrix table; the adjacency matrix Adj after Laplace transformation is multiplied by the node feature X, whose X1 region is 0 and X2 region has a value, and the output feature matrix table of the sum of the matrix products has a value in the seventh region O1 and a value in the O2 region is 0, which satisfies the message propagation of the neighboring nodes, aggregates the features of the source node to the features of the target node, and completes the message propagation of the neighboring nodes.

[0056] From the above, it can be seen that this embodiment establishes a unidirectional adjacency matrix without using the traditional message propagation method, which not only facilitates the application of matrix operations of graph convolution on low-clock multi-core computing devices, but also reduces the number of processed nodes and edges, thereby reducing computing costs and memory consumption.

[0057] Furthermore, in order to improve the flexibility of the association relationship identification model and make it applicable to an environment with a variety of computing resources and storage resources, the present invention also supports adjusting the data processing amount of the association relationship identification model, which may include the following contents:

[0058] When a computing scale adjustment instruction is received, the target sampling number and / or target sampling layer number are obtained by parsing the computing scale adjustment instruction; based on the target sampling layer number and the target sampling number, the target layer of the association relationship recognition model is sampled with a corresponding number of nodes to obtain the sampling nodes of the current graph sampling.

[0059] This embodiment can flexibly set the maximum number of sampling neighbors at each order according to the actual application scenario, and limit the number of nodes participating in the calculation each time. The adjacency matrix and node feature data established in this way have a fixed size, which can make the amount of intermediate data of the entire network within a reasonable range, ensuring the normal operation of the association relationship recognition model and better practicality.

[0060] The above embodiment does not limit the training process of the association relationship identification model. In order to make those skilled in the art more clearly understand the implementation of the present invention, the present invention also provides an exemplary training implementation method of the association relationship identification model, which may include the following contents:

[0061] Acquire the associated task graph data training set; use each graph sample data of the associated task graph data training set as a node, perform graph sampling on the associated task graph data training set, and obtain sampling data; construct multiple training subprocesses, each training subprocess reads a corresponding number of sampling nodes and their adjacent neighbor nodes from the sampling data according to a preset batch size, aggregates the feature information of each sampling node and its adjacent neighbor nodes based on the target polynomial, and determines the association relationship recognition result between each sampling node corresponding to the current graph sampling according to the current aggregated feature; obtain the predicted label of each graph sample data of the associated task graph data training set according to the association relationship recognition result of each training subprocess.

[0062] In this embodiment, each graph sample data of the association task graph data training set has a label that marks the association relationship. Taking the data association task as the paper citation relationship as an example, the association task graph data training set can be ogbn-papers100M (dataset name) containing more than 100 million nodes and 1 billion edges. The ogbn-papers100M dataset is a paper citation network, a directed graph, including 111 million papers, representing the citation relationship between computer science papers on arxiv (academic paper preprint repository). The nodes of ogbn-papers100M represent papers, and the edges represent the citation relationship of papers, which can truly reflect the citation relationship between papers in the academic network. Each paper is obtained by the average 128-dimensional feature vector obtained on the word2vec (a model for generating word vectors) model through the embedding amount of words in the title and abstract. The entire dataset is divided into 172 categories according to the field of computer science.

[0063] like Fig. 9As shown in the figure, taking DGL for graph sampling as an example, the entire training set of the associated task graph data is read and split into a training set, a validation set, and a test set according to a certain ratio, such as 7:2:1. Then, the batch (batch size), optimizer, number of iterations, gradient and other training parameters of the associated relationship recognition model are set. The main process loads the adjacency table adj information of the entire graph and the node ID of the training set of the associated task graph data to build a sampler for training. This process can be implemented using the method provided by the DGL framework, such as randomly sampling b points from the training set of the associated task graph data, finding the first- and second-order neighbor information through the graph structure, and building the adjacency table adj based on the edge information between nodes. The main process distributes data to M subprocesses, where M represents the number of processes. This value can be the same as the number of computing units of the computing device running the training task, such as the number of image processor cards participating in the training. Each subprocess obtains the training batch node information and its L-order neighbor node information from the sampler to form the Blocks information related to the L-layer network. Each Block contains the source node and target node features and edge structure. Use the edges between L-order neighbor nodes and L-1-order neighbor nodes to construct an adjacency list adj, and perform Laplace transform on the adjacency list, substitute the obtained Laplace matrix into the Chebyshev polynomial to approximate the adjacency matrix calculated by the wavelet, aggregate the feature information from the neighborhood, and pass the node features of the outer layer to the L-1-order neighbor node. Repeat this process until the feature information of batch nodes is aggregated. Use the aggregated feature information of batch nodes as the input of the fully connected layer to predict the labels of the target batch nodes. In the process of data training the associated task recognition model, the network model parameters of the associated task recognition model are continuously updated. When the model training stop condition is reached, such as the number of iterations reaches the preset total number of iterations, the currently obtained associated task recognition model is used as the associated task recognition model in the above-mentioned embodiment S101 for performing data association tasks.

[0064] It should be noted that there is no strict order of execution between the steps in the present invention. As long as they comply with the logical order, these steps can be executed simultaneously or in a certain preset order. Figure 1 and Fig. 9 This is just a schematic and does not mean that this is the only execution order.

[0065] The present invention also provides a corresponding device for the data association relationship determination method, which further makes the method more practical. Among them, the device can be described from the perspective of functional modules and hardware. The data association relationship determination device provided by the present invention is introduced below. The device is used to implement the data association relationship determination method provided by the present invention. In this embodiment, the data association relationship determination device may include or be divided into one or more program modules, and the one or more program modules are stored in a storage medium and executed by one or more processors to complete the data association relationship determination method disclosed in Example 1. The program module referred to in this embodiment refers to a series of computer program instruction segments that can perform specific functions, which is more suitable for describing the execution process of the data association relationship determination device in the storage medium than the program itself. The following description will specifically introduce the functions of each program module of this embodiment. The data association relationship determination device described below and the data association relationship determination method described above can be referenced to each other.

[0066] From the perspective of functional modules, see Fig.10 , Fig.10 The structure diagram of the data association relationship determination device provided in this embodiment under a specific implementation manner, the device may include: The task receiving module 101 is used to receive a data association task including at least the data to be associated and associated data information.

[0067] The task execution module 102 is used to input the source data of the data to be associated and the associated data information as node features into a pre-trained association relationship recognition model; wherein the association relationship recognition model is based on a graph neural network model, and uses a target polynomial to represent a frequency domain convolution kernel; the association relationship recognition model is subjected to graph sampling, and the feature information of each sampling node and its adjacent neighbor nodes is aggregated based on the target polynomial, and the association relationship recognition result between each sampling node corresponding to the current graph sampling is determined based on the current aggregated feature; based on each association relationship recognition result, target data having an association relationship is determined in the associated data information for the data to be associated.

[0068] Exemplarily, in some implementations of the present embodiment, the task execution module 102 may also be used to: for the current sampling node, use the edge features between the neighbor nodes of the current layer and the adjacent layer, the input features of the current layer, and the weight parameters of the current layer as the input of the target polynomial, and use the calculation result of the target polynomial as the output feature of the current layer; wherein the input feature of the current layer is the output feature of the previous layer; according to the output feature of the last layer, determine the aggregation feature of the current sampling node, and obtain the current aggregation feature according to the aggregation features of each sampling node sampled in the current graph.

[0069] As an exemplary implementation of the above embodiment, the above task execution module 102 can be further used to: generate an adjacency list based on the edge features between neighbor nodes in adjacent layers, and perform Laplace transformation on the adjacency list to obtain a Laplace matrix; input the Laplace matrix into the Chebyshev polynomial to approximate the adjacency matrix calculated by wavelet.

[0070] As an exemplary implementation of the above embodiment, the above task execution module 102 can be further used to: multiply the Laplacian matrix corresponding to the neighbor nodes of the current layer and the adjacent layer with the input features of the current layer to obtain the node features of the current layer; and use the product of the node features and the weight parameters of the current layer as the feature information of the current layer, and input it into the target polynomial.

[0071] As another exemplary implementation of the above embodiment, the above task execution module 102 can also be further used to: pre-construct an adjacency matrix table, an input feature matrix table and an output feature matrix table; the adjacency matrix table includes a first area, a second area, a third area and a fourth area, the first area and the third area, the second area and the fourth area are respectively aligned in the vertical direction, the first area and the second area are located in the same horizontal direction, and the third area and the fourth area are located in the same direction; the input feature matrix table includes a fifth area and a sixth area aligned in the vertical direction, and the output feature matrix table includes a seventh area and an eighth area aligned in the vertical direction; the sum of the matrix products of the adjacency matrix table and the input feature table is stored in the corresponding area of ​​the output feature matrix table; the feature information of the neighboring nodes with connecting edges is stored in the second area, and the input features of the current layer are stored in the sixth area; the information of the seventh area of ​​the output feature matrix table is read as the input of the target polynomial.

[0072] Exemplarily, in some other implementations of the present embodiment, the task execution module 102 may also be used to: generate sampling subgraph data according to each sampling node and its multi-order neighbor nodes; and sequentially transfer node features of each layer of the sampling subgraph data to node features of the previous layer based on the target polynomial in an outside-to-inward direction until they are transferred to each sampling node, thereby obtaining aggregate features of each sampling node corresponding to the current graph sampling.

[0073] Exemplarily, in some other implementations of this embodiment, the above-mentioned task execution module 102 can also be used to: input the current aggregated features corresponding to the current graph sampling into the fully connected layer, and obtain the association relationship identification result between the sampling points corresponding to the current graph sampling according to the output of the fully connected layer.

[0074] Exemplarily, in some other implementations of the present embodiment, the above-mentioned device also includes a parameter adjustment module, which is used to obtain the target sampling number and / or the target number of sampling layers by parsing the calculation scale adjustment instruction when receiving the calculation scale adjustment instruction; based on the target number of sampling layers and the target sampling number, the target layer of the association relationship identification model is sampled with a corresponding number of nodes to obtain the sampling nodes of the current graph sampling.

[0075] Exemplarily, in some other implementations of the present embodiment, the above-mentioned device also includes a model training module, which can be used to: obtain an associated task graph data training set; each graph sample data of the associated task graph data training set has a label that marks the association relationship; each graph sample data of the associated task graph data training set is used as a node, and graph sampling is performed on the associated task graph data training set to obtain sampling data; construct multiple training sub-processes, each training sub-process reads a corresponding number of sampling nodes and their adjacent neighbor nodes from the sampling data according to a preset batch size, aggregates the feature information of each sampling node and its adjacent neighbor nodes based on the target polynomial, and determines the association relationship identification result between each sampling node corresponding to the current graph sampling according to the current aggregated feature; obtains the predicted label of each graph sample data of the associated task graph data training set according to the association relationship identification result of each training sub-process.

[0076] The data association relationship determination device mentioned above is described from the perspective of functional modules. Furthermore, the present invention also provides an electronic device, which is described from the perspective of hardware. Fig.11 The following is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention in one implementation manner. Fig.11 As shown, the electronic device includes a memory 110 for storing computer programs; a processor 111 for implementing the steps of the data association relationship determination method mentioned in any of the above embodiments when executing the computer program.

[0077] Among them, the processor 111 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the processor 111 may also be a controller, a microcontroller, a microprocessor or other data processing chip. The processor 111 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 111 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 111 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 111 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0078] The memory 110 may include one or more computer non-volatile storage media, which may be non-transitory. The memory 110 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the memory 110 may be an internal storage unit of an electronic device, such as a hard disk of a server. In other embodiments, the memory 110 may also be an external storage device of an electronic device, such as a plug-in hard disk equipped on a server, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Further, the memory 110 may also include both an internal storage unit of an electronic device and an external storage device. The memory 110 may not only be used to store application software and various types of data installed in the electronic device, such as: the code of the program in the process of executing the data association relationship determination method, but also be used to temporarily store data that has been output or is to be output. In this embodiment, the memory 110 is at least used to store the following computer program 1101, wherein, after the computer program is loaded and executed by the processor 111, the relevant steps of the data association relationship determination method disclosed in any of the aforementioned embodiments can be implemented. In addition, the resources stored in the memory 110 may also include an operating system 1102 and data 1103, etc., and the storage method may be temporary storage or permanent storage. The operating system 1102 may include Windows, Unix, Linux, etc. The data 1103 may include, but is not limited to, data corresponding to the result of determining the data association relationship, etc.

[0079] In some embodiments, the electronic device may further include a display screen 112, an input / output interface 113, a communication interface 114 or a network interface, a power supply 115 and a communication bus 116. Among them, the display screen 112 and the input / output interface 113, such as a keyboard, belong to the user interface, and exemplary user interfaces may also include standard wired interfaces, wireless interfaces, and the like. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, and the like. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface. The communication interface 114 may exemplarily include a wired interface and / or a wireless interface, such as a WI-FI interface, a Bluetooth interface, and the like, which are generally used to establish a communication connection between the electronic device and other electronic devices. The communication bus 116 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, and the like. The bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Fig.11 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0080] Those skilled in the art will understand that Fig.11 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, for example, it may also include a sensor 117 to implement various functions.

[0081] It can be understood that if the data association relationship determination method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile storage medium. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes but is not limited to: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), electrically erasable programmable ROM, register, hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), magnetic memory, removable disk, CD-ROM, magnetic disk or optical disk, etc. Various media that can store program codes. Based on this, the present invention also provides a non-volatile storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the data association relationship determination method recorded in any of the above embodiments are performed.

[0082] It is understandable that if the data association relationship determination method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, the computer software product does not need to be stored in a physical storage medium. For example, it can be directly transmitted to a computer or other device with information processing capabilities through a wired network or a wireless network to perform all or part of the steps of the methods of each embodiment of the present invention. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product. Based on this, the present invention also provides a computer program product that stores a computer program. When the computer program is executed by a processor, it performs the steps of the data association relationship determination method recorded in any of the above embodiments.

[0083] It is understandable that, due to the characteristics of the association relationship recognition model in processing non-Euclidean data, whether it is training the association relationship recognition model or running the association relationship recognition model to perform data association tasks, the computing power and storage capacity of the hardware environment in which it runs are highly required. In addition, in order to meet the computing resource and storage resource requirements of the association relationship recognition model, while balancing the computing throughput and memory bandwidth to ensure the effective use of resources, avoid waste, and ultimately achieve the best performance, the present invention also provides a hardware environment for training the association relationship recognition model and performing data association tasks through the association relationship recognition model, please refer to Fig.12 The present invention also provides a data association relationship determination system, which may include:

[0084] The data association relationship determination system may include a first computing device 121 and a second computing device 122, wherein the first computing device 121 and the second computing device 122 are connected via a target bus such as PCIE (peripheral component interconnectexpress, high-speed serial computer expansion bus), such as the second computing device 122 is connected to the PCIe Gen5×16 of the first computing device 121. The second computing device 122 has parallel computing capabilities, and the first computing device 121 may be, for example, a device using a CPU as a processor, such as a server, and the second computing device 122 may be, for example, a computing device with limited resources such as an FPGA, a GPU, and a low-clock multi-computing core method. Among them, the first computing device 121 is configured to receive a data association task that at least includes the data to be associated and the associated data information, and send the data to be associated and the associated data information to the second computing device 122 through the target bus interface; the second computing device 122 is configured to perform graph sampling on the association relationship identification model, aggregate the feature information of each sampling node and its adjacent neighbor nodes based on the target polynomial, determine the association relationship identification result between each sampling node corresponding to the current graph sampling according to the current aggregated feature, and send the association relationship identification result to the first computing device 121 through the target bus interface; the first computing device 121 is also configured to determine the target data with an association relationship in the associated data information for the data to be associated according to each association relationship identification result.

[0085] As can be seen from the above, through lightweight graph sampling processing, the number of nodes for each training or inference is reduced, thereby effectively reducing the amount of calculation and memory usage, meeting the requirements of computational complexity and memory overhead for large-scale graph data. At the same time, the parallel processing capability of the second computing device is used to accelerate the sampling and feature calculation process of neighbor nodes, realize efficient graph data processing, and maintain a high execution accuracy of data association tasks.

[0086] In order to further improve the efficiency and accuracy of the second computing device running the association relationship identification model, based on the above embodiment, the present invention also provides an exemplary structure of the second computing device, which may include the following contents:

[0087] The second computing device 122 may include a controller, a data storage device, and an output result buffer; the controller, the data storage device, and the output result buffer communicate via an on-chip network, and the second computing device 122 communicates with the target bus interface via a NOC (Network-on-Chip, on-chip network). In order to further improve the data communication efficiency within the second computing device and between the second computing device and the first computing device, as Fig.13As shown, all components inside the second computing device 122 are interconnected through a NoC with high bandwidth. The NoC can also be connected to Ethernet (Ethernet 400GE). On this basis, the RoCE-Lite (protocol name) protocol can also be carried through Ethernet, so that remote storage access can be performed through high-speed Ethernet to support graph computing of massive nodes.

[0088] The controller is configured to perform graph sampling on the association relationship recognition model, aggregate the feature information of each sampling node and its adjacent neighbor nodes based on the target polynomial, and determine the association relationship recognition result between each sampling node corresponding to the current graph sampling according to the current aggregated feature; the data storage is configured to store the feature data corresponding to the sampling node and its neighbor nodes sent by the first computing device and the weight parameters of each layer of the association relationship recognition model; the data storage can use DDR (Double Data Rate, double rate storage) to store the high-speed access data required in the processing of the association relationship recognition model. In order to improve the data reading efficiency, the read and write operations of DDR are controlled by DDR Ctrl (Double Data Rate Controller, memory controller). The output result buffer is configured to store the association relationship recognition results between each sampling node obtained by each graph sampling.

[0089] Furthermore, in order to further improve the execution efficiency of data association tasks and the training efficiency of association relationship recognition models, based on the above embodiments, the present invention also provides an exemplary result of the controller, such as Fig.14 As shown, it may include the following:

[0090] The controller of the second computing device 122 may include an instruction executor, a processor array, a nonlinear processor, an intermediate result buffer, and a weight data buffer. The instruction executor is connected to the data memory, the processor array, and the nonlinear processor, respectively, the processor array is connected to the weight data buffer, the intermediate result buffer, and the nonlinear processor, respectively, and the intermediate result buffer is connected to the output result buffer.

[0091] Among them, the controller is used to implement the establishment of adjacency matrix, calculation of Laplace, Chebyshev polynomials and fully connected layer calculation. Combined with the characteristics of low-clock multi-computing core devices such as FPGA, PE (Processing Element) array can be used to perform these operations. The PE array is composed of many PEs (Processing Element, processor), and generally each PE includes a multiplication and addition unit, a small number of registers and a small amount of control logic. That is, the PE array calculates the Laplace, Chebyshev polynomials, matrix product and fully connected layer operations of the adjacency matrix for graph convolution. The weight data buffer is configured to read the weight parameters of each layer of the association recognition model from the data storage device, and the intermediate result buffer stores the results generated by the intermediate processing process for the next round of iteration.

[0092] Among them, the instruction executor is used to execute a pre-specified operation at a specified time. In this embodiment, the instruction executor, upon receiving the write start instruction, reads the adjacency list from the data storage device, and sends an adjacency matrix generation instruction carrying the adjacency list to the processor array. The processor array performs Laplace transform calculation on the adjacency list and stores the adjacency matrix in the intermediate result buffer; the instruction executor reads the input features of each layer from the data storage device, and reads the weight parameters of the corresponding layer from the weight data buffer, and sends a node feature information generation instruction to the processor array at the same time. The processor array calculates the product of the input features and weight parameters of the current layer, and stores it as the node feature information of the current layer in the intermediate result buffer; the instruction executor sends an aggregate feature generation instruction to the processor array, and the processor array inputs the node feature information and the adjacency matrix into the target polynomial for calculation, outputs the calculation result of the target polynomial to the nonlinear processor for nonlinear processing, obtains the aggregate feature of the current layer, and stores the aggregate feature of the current layer in the intermediate result buffer. It can be seen that the above instruction executor controls the flow of instruction execution, controls the calculation process of Laplace, matrix product, and Chebyshev polynomials, controls the activation function layer to perform nonlinear calculations on the calculation results, and finally stores the intermediate results in the intermediate result buffer. It is also necessary to control the intermediate features of the nodes, that is, to calculate the matrix product of the graph convolution results and the weights of the weight data buffer, and output the final result to the output result buffer through the fully connected layer.

[0093] In order to make the entire implementation process more clear to those skilled in the art, the present invention also takes a CPU and an FPGA to form a data association relationship determination system, uses the data association relationship determination system to execute the association relationship recognition model training process, and uses DGL for graph sampling as an example to explain the data processing flow of the entire data association relationship determination system, which may include the following contents: The CPU-side program executes, reads the entire training set of the associated task graph data, and splits it into a training set, a validation set, and a test set according to a certain ratio, and then sets the training parameters of the associated relationship recognition model. The main process loads the adjacency table adj information of the entire graph and the node ID of the training set of the associated task graph data to build a sampler for training. Data is allocated to the corresponding number of child processes according to the number of FPGA cards. Each child process obtains the training batch node information and its L-order neighbor node information from the sampler to form the Blocks information related to the L-layer network. Each Block contains the source node and target node features and edge structure. The batch node and its neighbor node feature data F and weight data W are written into the FPGA's DDR through the PCIE interface or Ethernet. The unidirectional edges between the L-order neighbor and the L-1-order neighbor nodes are used to construct a unidirectional adjacency table adj, and the adjacency table adj is written into the FPGA's DDR through PCIE or Ethernet. The write start command enables the instruction executor to run, calculate the Laplacian matrix in the PE array, and obtain the adjacency matrix data and put it into the intermediate result buffer. The instruction executor controls the instruction to take the L-order node feature data X0 from the DDR, take the L-th layer weight data W0 from the weight data buffer, and perform the matrix product of the node feature data and the weight data on the PE array to obtain the intermediate data X1 and put it into the intermediate result buffer. The adjacency matrix and X1 data are taken out from the intermediate result buffer, and the instruction executor controls the instruction to input the adjacency matrix and X1 into the Chebyshev polynomial and calculate, aggregate the feature information from the neighborhood, and then perform the activation function operation to obtain the intermediate data X2, put it into the intermediate result buffer, and pass the node features of the outermost layer L to the L-1 order neighbor node. Repeat the above process to calculate the feature information of the L-1 order to L-2 order nodes until the feature information of the final batch of nodes is aggregated and put into the output result buffer. The CPU side takes out the aggregated feature information of the batch nodes from the output result buffer as the input of the fully connected layer to obtain the association relationship between the sampling points of the current graph sampling. Repeat the above process until the association relationship between the training samples of the training set of the associated task graph data is obtained.

[0094] From the above, it can be seen that this embodiment optimizes the network structure of the association relationship identification model to make the association relationship identification model better fit the resources of the FPGA. The parallel computing capability of the FPGA can be used to process the Laplace, Chebyshev polynomial, and matrix product calculation processes of the association relationship identification model. The PE array is used to process the matrix product, which can better handle large calculation amounts, reduce resource loss, and accelerate the execution efficiency and result accuracy of data association tasks.

[0095] The above is a detailed introduction to a method, system, electronic device, non-volatile storage medium and computer program product for determining a data association relationship provided by the present invention. The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can refer to each other. Whether the units and algorithm steps of each example described in each disclosed embodiment are executed in electronic hardware or computer software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods for each specific application to implement the described functions, and such implementation should not be considered to exceed the scope of the present invention. Without departing from the principles of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A method for determining a data association relationship, characterized in that: include: Receiving a data association task including at least data to be associated and associated data information, and inputting each source data of the data to be associated and the associated data information as node features into a pre-trained association relationship recognition model; wherein the association relationship recognition model is based on a graph neural network model, and uses a target polynomial to represent a frequency domain convolution kernel; Performing graph sampling on the association relationship recognition model, aggregating feature information of each sampling node and its adjacent neighbor nodes based on the target polynomial, and determining an association relationship recognition result between each sampling node corresponding to the current graph sampling according to the current aggregated feature; According to the results of identifying each association relationship, target data having an association relationship is determined in the associated data information for the data to be associated.

2. The method for determining a data association relationship according to claim 1, characterized in that: Aggregating feature information of each sampling node and its adjacent neighbor nodes based on the target polynomial includes: For the current sampling node, the edge features between the neighbor nodes of the current layer and the adjacent layer, the input features of the current layer and the weight parameters of the current layer are used as the input of the target polynomial, and the calculation result of the target polynomial is used as the output feature of the current layer; wherein the input feature of the current layer is the output feature of the previous layer; According to the output features of the last layer, the aggregation features of the current sampling node are determined, and the current aggregation features are obtained according to the aggregation features of each sampling node sampled in the current graph.

3. The method for determining data association relationship according to claim 2, characterized in that: The target polynomial is a Chebyshev polynomial, and the edge features between the neighbor nodes of the current layer and the adjacent layer, the input features of the current layer, and the weight parameters of the current layer are used as inputs of the target polynomial, including: Generate an adjacency list according to edge features between neighbor nodes of adjacent layers, and perform Laplace transformation on the adjacency list to obtain a Laplace matrix; The Laplace matrix is ​​input into the Chebyshev polynomials to approximate the adjacency matrix for wavelet computation.

4. The method for determining data association relationship according to claim 3, characterized in that: The edge features between the neighbor nodes of the current layer and the adjacent layer, the input features of the current layer and the weight parameters of the current layer are used as inputs of the target polynomial, including: Multiply the Laplacian matrix corresponding to the neighbor nodes of the current layer and the adjacent layer by the input features of the current layer to obtain the node features of the current layer; The product of the node feature and the weight parameter of the current layer is used as the feature information of the current layer and input into the target polynomial.

5. The method for determining data association relationship according to claim 2, characterized in that: The edge features between the neighbor nodes of the current layer and the adjacent layer, the input features of the current layer and the weight parameters of the current layer are used as inputs of the target polynomial, including: Pre-constructing an adjacency matrix table, an input feature matrix table, and an output feature matrix table; the adjacency matrix table includes a first region, a second region, a third region, and a fourth region, the first region and the third region, the second region and the fourth region are respectively aligned in the vertical direction, the first region and the second region are located in the same horizontal direction, and the third region and the fourth region are located in the same direction; the input feature matrix table includes a fifth region and a sixth region aligned in the vertical direction, and the output feature matrix table includes a seventh region and an eighth region aligned in the vertical direction; the sum of the matrix products of the adjacency matrix table and the input feature table is stored in the corresponding region of the output feature matrix table; storing feature information of neighbor nodes with connecting edges in the second area, and storing input features of the current layer in the sixth area; The information in the seventh area of ​​the output feature matrix table is read as the input of the target polynomial.

6. The method for determining data association relationship according to claim 1, characterized in that: Aggregating feature information of each sampling node and its adjacent neighbor nodes based on the target polynomial includes: Generate sampling subgraph data according to each sampling node and the multi-order neighbor nodes of each sampling node; From outside to inside, the node features of each layer of the sampled subgraph data are sequentially transferred to the node features of the previous layer based on the target polynomial until they are transferred to each sampling node, thereby obtaining the aggregated features of each sampling node corresponding to the current graph sampling.

7. The method for determining data association relationship according to claim 1, characterized in that: The association relationship recognition result between the sampling nodes corresponding to the current graph sampling is determined according to the current aggregation feature, including: The current aggregated features corresponding to the current graph sampling are input into the fully connected layer, and the association relationship recognition result between the sampling points corresponding to the current graph sampling is obtained according to the output of the fully connected layer.

8. The method for determining data association relationship according to claim 1, characterized in that: Also includes: When receiving a calculation scale adjustment instruction, obtaining a target sampling number and / or a target sampling layer number by parsing the calculation scale adjustment instruction; Based on the target number of sampling layers and the target sampling number, a corresponding number of nodes are sampled on the target layer of the association relationship recognition model to obtain sampling nodes for current graph sampling.

9. The method for determining a data association relationship according to any one of claims 1 to 8, characterized in that: The training process of the association relationship recognition model includes: Obtaining a training set of associated task graph data; each graph sample data of the training set of associated task graph data has a label that marks the associated relationship; Taking each graph sample data of the associated task graph data training set as a node, performing graph sampling on the associated task graph data training set to obtain sampling data; Constructing multiple training subprocesses, each training subprocess reads a corresponding number of sampling nodes and their adjacent neighbor nodes from the sampling data according to a preset batch size, aggregates feature information of each sampling node and its adjacent neighbor nodes based on a target polynomial, and determines an association relationship recognition result between each sampling node corresponding to the current graph sampling according to the current aggregated feature; According to the association relationship identification results of each training sub-process, the predicted label of each graph sample data of the associated task graph data training set is obtained.

10. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the processor is used to implement the steps of the method for determining a data association relationship as claimed in any one of claims 1 to 9 when executing a computer program stored in the memory.

11. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for determining a data association relationship according to any one of claims 1 to 9 are implemented.

12. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the method for determining a data association relationship described in any one of claims 1 to 9 are implemented.

13. A data association relationship determination system, characterized in that: The device comprises a first computing device and a second computing device; the first computing device and the second computing device are connected via a target bus, and the second computing device has parallel computing capability; Among them, the first computing device is configured to receive a data association task that at least includes data to be associated and associated data information, and send the data to be associated and the associated data information to the second computing device through a target bus interface; the second computing device is configured to perform graph sampling on the association relationship recognition model, aggregate feature information of each sampling node and its adjacent neighbor nodes based on a target polynomial, determine the association relationship recognition result between each sampling node corresponding to the current graph sampling according to the current aggregated feature, and send the association relationship recognition result to the first computing device through the target bus interface; the association relationship recognition model is based on a graph neural network model, and uses the target polynomial to represent a frequency domain convolution kernel; the first computing device is also configured to determine target data with an association relationship in the associated data information for the data to be associated according to each association relationship recognition result.

14. The data association relationship determination system according to claim 13, characterized in that: The second computing device includes a controller, a data memory, and an output result buffer; the controller, the data memory, and the output result buffer communicate via an on-chip network, and the second computing device communicates with a target bus interface via the on-chip network; The controller is configured to perform graph sampling on the association relationship recognition model, aggregate feature information of each sampling node and its adjacent neighbor nodes based on the target polynomial, and determine the association relationship recognition result between the sampling nodes corresponding to the current graph sampling according to the current aggregated features; The data storage device is configured to store feature data corresponding to the sampling node and its neighboring nodes and weight parameters of each layer of the association relationship recognition model sent by the first computing device; The output result buffer is configured to store the association relationship identification results between the sampling nodes obtained by each graph sampling.

15. The data association relationship determination system according to claim 13, characterized in that: The controller of the second computing device includes an instruction executor, a processor array, a nonlinear processor, an intermediate result buffer and a weight data buffer; the instruction executor is connected to the data memory, the processor array and the nonlinear processor respectively, the processor array is connected to the weight data buffer, the intermediate result buffer and the nonlinear processor respectively, and the intermediate result buffer is connected to the output result buffer; The weight data buffer is configured to read the weight parameters of each layer of the association relationship recognition model from the data storage; The instruction executor, upon receiving the write start instruction, reads the adjacency list from the data memory, and sends an adjacency matrix generation instruction carrying the adjacency list to the processor array, the processor array performs Laplace transform calculation on the adjacency list, and stores the adjacency matrix in the intermediate result buffer; The instruction executor reads the input features of each layer from the data storage device, and reads the weight parameters of the corresponding layer from the weight data buffer, and sends a node feature information generation instruction to the processor array at the same time. The processor array calculates the product of the input features and weight parameters of the current layer, and stores it in the intermediate result buffer as the node feature information of the current layer; the instruction executor sends an aggregate feature generation instruction to the processor array, and the processor array inputs the node feature information and the adjacency matrix into the target polynomial for calculation, outputs the calculation result of the target polynomial to the nonlinear processor for nonlinear processing, obtains the aggregate feature of the current layer, and stores the aggregate feature of the current layer in the intermediate result buffer.

Images