Feature extraction model training method, data classification method, device and equipment

Abstract

The present disclosure provides a method for training a feature extraction model, a data classification method, a device and equipment, and relates to the field of data processing technology. The method for training a feature extraction model includes: obtaining training sample data, the training sample data is constructed by graph structure sample data and real labels; inputting the graph structure sample data into a pre-constructed initial feature extraction model to train the initial feature extraction model in a feature extraction process to obtain a target feature extraction model; the feature extraction process includes: using a graph neural network to perform graph convolution processing on the graph structure sample data to obtain a feature representation corresponding to the nodes in the graph structure sample data; inputting the feature representation into a variational coding network to convert the feature representation into a latent variable distribution; inputting the latent variable distribution into a feature enhancement network based on a self-attention mechanism to obtain enhanced features. The technical solution in the present disclosure can improve the expressiveness of the model and the feature extraction effect on graph structure data.

Description

Background Art

[0002] Graph structured data is a type of data represented by a data structure called a graph. A graph consists of nodes and edges. Nodes can represent basic units in data, such as users in social networks, road intersections in traffic networks, etc., while edges represent the relationship between nodes.

[0003] Graph structure data has been widely used in social network analysis, traffic network planning, protein structure analysis and other fields. In related technologies, neural network models are usually used to process graph structure data, such as multi-layer perceptron and graph convolutional network. However, due to the sparsity and long-range dependencies of graph structure data, network models in related technologies often face challenges such as difficulty in effectively capturing the global information of graph structure data, long-range dependencies and limited model expression capabilities, resulting in poor feature extraction of graph structure data.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute relevant technology known to ordinary technicians in the field. Summary of the invention

[0005] The purpose of the embodiments of the present disclosure is to provide a feature extraction model training method, a data classification method, a feature extraction model training device, a data classification device, an electronic device and a computer-readable storage medium, thereby improving the expression ability of the model and the feature extraction effect on graph structure data at least to a certain extent.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to a first aspect of an embodiment of the present disclosure, a method for training a feature extraction model is provided, the method comprising: obtaining training sample data, wherein the training sample data is constructed by graph structure sample data and true labels; the graph structure sample data is input into a pre-constructed initial feature extraction model to perform a feature extraction process training on the initial feature extraction model to obtain a target feature extraction model; wherein the feature extraction process comprises: performing graph convolution processing on the graph structure sample data using a graph neural network to obtain feature representations corresponding to nodes in the graph structure sample data; inputting the feature representation into a variational coding network to convert the feature representation into a latent variable distribution; and inputting the latent variable distribution into a feature enhancement network based on a self-attention mechanism to obtain enhanced features.

[0008] In some example embodiments of the present disclosure, based on the aforementioned scheme, the graph structure sample data is subjected to graph convolution processing by using a graph neural network to obtain feature representations corresponding to nodes in the graph structure sample data, including: multiple graph convolution layers in the graph neural network, according to:

[0009] ;

[0010] Performing graph convolution processing on the graph structure sample data, extracting feature representations corresponding to nodes in the graph structure sample data layer by layer; wherein, represents the characteristic representation, Representation Node The set of neighboring nodes of represents the normalization constant, Indicates The weight matrix of the layer, represents a nonlinear activation function, Represents the number of graph convolutional layers.

[0011] In some example embodiments of the present disclosure, based on the aforementioned scheme, the step of inputting the feature representation into a variational coding network to convert the feature representation into a latent variable distribution includes: inputting the feature representation into a variational coding network, and utilizing a multilayer perceptron in the variational coding network to obtain a mean and variance of latent variables corresponding to the feature representation; obtaining an initial latent variable distribution based on the mean and variance of the latent variables; sampling from the initial latent variable distribution, and reparameterizing the initial latent variable distribution to obtain an optimized latent variable distribution.

[0012] In some example embodiments of the present disclosure, based on the aforementioned scheme, the latent variable distribution is input into a feature enhancement network based on a self-attention mechanism to obtain enhanced features, including: inputting the latent variable distribution into the feature enhancement network, the feature enhancement network using a multi-head self-attention mechanism to obtain multiple parallel attention representations; concatenating and linearly changing the parallel attention representations to obtain optimized attention representations; using a feedforward neural network, according to:

[0013] ;

[0014] The optimized attention representation is enhanced to obtain a reinforced attention representation; wherein, represents the enhanced attention representation, represents the optimized attention representation, , represents the weight matrix of the fully connected layer in the feedforward neural network, , represents the bias vector, represents a nonlinear activation function; the enhanced attention representation is normalized and residually connected to obtain the enhanced feature.

[0015] In some example embodiments of the present disclosure, based on the aforementioned solution, the feature enhancement network uses a multi-head self-attention mechanism to obtain multiple parallel attention representations, including: the feature enhancement network is based on:

[0016] ;

[0017] Perform self-attention calculation on the latent variable distribution to obtain the parallel attention representation; wherein Attention ( Q , K , V ) represents the parallel attention representation, represents the dimension of the key vector, Q represents the query matrix, K represents the key matrix, T represents transpose, V represents the value matrix; Q , K , V according to Calculated, where , and Represent the trainable weight matrices, Z represents a set of latent variables.

[0018] In some example embodiments of the present disclosure, based on the aforementioned scheme, the feature extraction process training of the initial feature extraction model to obtain a target feature extraction model includes: inputting the graph structure sample data into the initial feature extraction model to obtain the enhanced features; mapping the enhanced features using a decoder constructed based on multi-layer perception to obtain classification results of the graph structure sample data; and iteratively training and adjusting parameters of the initial feature extraction model based on the classification results and the true labels corresponding to the graph structure sample data to obtain the target feature extraction model.

[0019] In some example embodiments of the present disclosure, based on the aforementioned scheme, the initial feature extraction model is iteratively trained and parameterized based on the classification results and the true labels corresponding to the graph structure sample data, including: calculating a loss function based on the classification results and the true labels corresponding to the graph structure sample data; determining the gradient of the parameters of the initial feature extraction model using a back propagation algorithm based on the loss function; updating the parameters using an optimization algorithm based on the gradient of the parameters, and iteratively training the initial feature extraction model until an iteration termination condition is met.

[0020] According to a second aspect of an embodiment of the present disclosure, a data classification method is provided, the method comprising: obtaining data to be classified, wherein the data to be classified is graph structure data; inputting the data to be classified into a target feature extraction model to obtain a classification result of the data to be classified; wherein the target feature extraction model is trained according to the training method of the feature extraction model as described in the above embodiment.

[0021] According to a third aspect of an embodiment of the present disclosure, a training device for a feature extraction model is provided, the device comprising: a sample construction module, used to obtain training sample data, wherein the training sample data is constructed by graph structure sample data and true labels; a model training module, used to input the graph structure sample data into a pre-constructed initial feature extraction model to perform a feature extraction process training on the initial feature extraction model to obtain a target feature extraction model; wherein the feature extraction process comprises: performing graph convolution processing on the graph structure sample data using a graph neural network to obtain feature representations corresponding to nodes in the graph structure sample data; inputting the feature representation into a variational coding network to convert the feature representation into a latent variable distribution; and inputting the latent variable distribution into a feature enhancement network based on a self-attention mechanism to obtain enhanced features.

[0022] According to a fourth aspect of an embodiment of the present disclosure, a data classification device is provided, which includes: a data acquisition module, used to acquire data to be classified, wherein the data to be classified is graph structure data; a data classification module, used to input the data to be classified into a target feature extraction model to obtain a classification result of the data to be classified; wherein the target feature extraction model is trained according to the training method of the feature extraction model as described in the above embodiment.

[0023] According to a fifth aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising: a processor; and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the training method of the feature extraction model or the data classification method described above is implemented.

[0024] According to a sixth aspect of an embodiment of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the training method or data classification method according to the above-mentioned feature extraction model is implemented.

[0025] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0026] In the training method of the feature extraction model in the example embodiment of the present disclosure, on the one hand, the graph neural network processes the graph structure sample data through the graph convolution operation to extract the feature representation of the node, which can not only aggregate the information of the node and its neighborhood, thereby learning the local relationship and structural characteristics between the nodes, but also reduce the original high-dimensional node features to a lower-dimensional potential feature space, thereby reducing the model complexity and improving the training efficiency; in addition, the feature representation corresponding to the node is converted into a latent variable distribution, and the variational inference method is used to capture the potential structure of the data, thereby enhancing the expressive power of the model; on the other hand, the enhanced features are obtained by using the feature enhancement network based on the self-attention mechanism, which can capture the complex long-range dependencies between nodes, thereby better understanding the global structure of the data; on the other hand, the target feature extraction model includes a graph neural network, a variational coding network and a feature enhancement network, so that the model integrates local structural information, potential structure and long-range dependencies, thereby being able to more comprehensively understand the data and effectively extract the deep features of the graph structure data.

[0027] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0029] Figure 1 A flowchart of a method for training a feature extraction model according to some embodiments of the present disclosure is schematically shown.

[0030] Figure 2 The schematic diagram of the architecture of the feature extraction model according to some embodiments of the present disclosure is schematically shown.

[0031] Figure 3 The flowchart of the data classification method according to some embodiments of the present disclosure is schematically shown.

[0032] Figure 4 A block diagram of a training device for a feature extraction model according to some embodiments of the present disclosure is schematically shown.

[0033] Figure 5 A block diagram of a data classification device according to some embodiments of the present disclosure is schematically shown.

[0034] Figure 6A schematic diagram of the structure of a computer system of an electronic device according to some embodiments of the present disclosure is schematically shown.

[0035] Figure 7 A schematic diagram of a computer-readable storage medium according to some embodiments of the present disclosure is schematically shown.

[0036] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0037] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Instead, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0038] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. The singular forms "a", "the" and "the" used in this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0039] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0040] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more comprehensive and complete and will fully convey the concept of the example embodiments to those skilled in the art.

[0041] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.

[0042] Furthermore, the drawings are only schematic illustrations and are not necessarily drawn to scale. The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0043] Graph structure data often appears in practical applications, covering a variety of data from social networks, protein structures in bioinformatics, transportation networks in urban planning, and even workflow diagrams in project management. These data contain not only rich node attributes, but also complex relationships between nodes, which brings huge challenges to data analysis.

[0044] Related technologies usually use neural network models to process graph structured data, such as multi-layer perceptrons and graph convolutional networks. However, due to the sparsity and long-range dependencies of graph structured data, network models in related technologies often face challenges such as difficulty in effectively capturing the global information of graph structured data, long-range dependencies, and limited model expression capabilities, resulting in poor feature extraction effects on graph structured data.

[0045] In order to solve all or part of the technical problems in the above-mentioned related technologies, in an exemplary embodiment of the present disclosure, a training method for a feature extraction model is firstly proposed. Figure 1 The following schematically shows a flow chart of a method for training a feature extraction model according to some embodiments of the present disclosure. Figure 1 As shown, the training method of the feature extraction model may include the following steps:

[0046] Step S110, obtaining training sample data, where the training sample data is constructed by graph structure sample data and true labels;

[0047] Step S120, inputting the graph structure sample data into a pre-built initial feature extraction model to perform feature extraction process training on the initial feature extraction model to obtain a target feature extraction model;

[0048] Among them, the feature extraction process includes: using a graph neural network to perform graph convolution processing on the graph structure sample data to obtain the feature representation corresponding to the nodes in the graph structure sample data; inputting the feature representation into a variational coding network to convert the feature representation into a latent variable distribution; inputting the latent variable distribution into a feature enhancement network based on a self-attention mechanism to obtain enhanced features.

[0049] The above feature extraction model combines the advantages of graph neural networks, variational coding networks, and feature enhancement networks. It can effectively extract and model the deep features of complex structured data, significantly improving the model's ability to process complex structured data and reasoning speed, and is especially suitable for processing complex structured data such as graph structured data. Through the end-to-end training framework, the various parts of the model are optimized collaboratively to ensure the improvement of the overall performance and efficiency of the model.

[0050] Next, the training method of the feature extraction model in the above example embodiment will be further described.

[0051] In step S110, training sample data is obtained, where the training sample data is constructed from graph structure sample data and true labels.

[0052] The graph structured sample data may represent sample data represented in the form of a graph, and the graph structured data includes nodes and edges, wherein nodes represent basic units of data and edges represent relationships between nodes. In the disclosed embodiment, the graph structured data may be other graph structured data such as traffic network information data, natural language processing data, user information data, and chemical material data.

[0053] In step S120, the graph structure sample data is input into a pre-built initial feature extraction model to perform feature extraction process training on the initial feature extraction model to obtain a target feature extraction model.

[0054] Among them, the target feature extraction model can be constructed by connecting the graph neural network, the variational encoding network and the feature enhancement network in sequence.

[0055] The graph structure sample data is processed by graph convolution using a graph neural network to obtain the feature representation corresponding to the nodes in the graph structure sample data. Among them, the graph neural network can represent a deep learning model specifically used to process graph structure data, which can capture the information of nodes and their neighborhoods in the graph, and extract the feature representation of nodes layer by layer through multi-layer graph convolution operations. Graph convolution processing can represent an operation in the graph neural network, which updates the node features by aggregating the information of the node and its neighborhood, so that the node features contain not only its own information, but also the information of its neighboring nodes.

[0056] Specifically, the graph neural network first passes The original data is converted into graph structure sample data with graph structure, where:G Represents graph structure sample data, V Represents a collection of nodes. E Represents the edge set. Each node is assigned an initial feature representation. For each node , its initialization feature can be expressed as , these features can include the attribute information of the node. Then, for each node, calculate the weighted average of the feature representations of its neighboring nodes, and assign weights to the neighboring nodes. The weights are usually related to the connection strength between the nodes; combine the node's own feature representation with the weighted average feature representation of the neighboring nodes, such as through summation or concatenation operations; apply a nonlinear activation function, such as ReLU, to perform a nonlinear transformation on the combined feature representation; through multi-layer graph convolution operations, each layer uses a different weight matrix and activation function to extract the node's feature representation layer by layer, and gradually integrate more neighborhood information. Finally, each node will obtain a feature representation that contains its own information and the information of its neighboring nodes.

[0057] In some embodiments, a graph convolution process is performed on the graph structure sample data using a graph neural network to obtain feature representations corresponding to nodes in the graph structure sample data, specifically including the following steps: multiple graph convolution layers in the graph neural network perform graph convolution process on the graph structure sample data according to the following formula (1), and extract feature representations corresponding to nodes in the graph structure sample data layer by layer:

[0058] (1)

[0059] in, Indicates feature representation, Representation Node The set of neighboring nodes of represents the normalization constant, Indicates The weight matrix of the layer, represents a nonlinear activation function, Indicates the number of graph convolutional layers.

[0060] Specifically, for the node , first aggregate its own features with the features of its neighboring nodes. This process can be done through the weight matrix and the normalization constant Implementation, where the weight matrix Learn the relationship between node features, while the normalization constant The influence of node connectivity is taken into account to avoid nodes with high connectivity occupying the majority of the feature representation. Then, the aggregated neighborhood information is passed through a nonlinear activation function Processing, introducing nonlinear relationships, so as to capture the complex relationship between node features, after nonlinear transformation, the node In the +1 Feature representation of graph neural network layer .

[0061] The feature representation is input into the variational encoding network to convert the feature representation into a latent variable distribution. Among them, the variational encoding network, also known as the variational autoencoder (VAE), is a neural network model, which can be composed of two parts: an encoder and a decoder. Among them, the encoder can be used to compress the input data into a set of statistical parameters, usually the mean and variance of the latent variable, which define a probability distribution, that is, the latent variable distribution. The latent variable distribution can be assumed to be a Gaussian distribution, which can be fully described by the mean and variance. The latent variable distribution reflects the potential structure and uncertainty of the data. In this embodiment, the expressiveness and generalization ability of the model are enhanced by converting the feature representation corresponding to the node into a latent variable distribution. The introduction of the latent variable distribution also enables the model to capture the complex structure of the data and learn the potential representation of the data, thereby improving the performance and robustness of the model.

[0062] In some embodiments, the feature representation is input into a variational coding network to convert the feature representation into a latent variable distribution, which specifically includes the following steps: inputting the feature representation into the variational coding network, using a multilayer perceptron in the variational coding network to obtain the mean and variance of the latent variables corresponding to the feature representation; obtaining an initial latent variable distribution based on the mean and variance of the latent variables; sampling from the initial latent variable distribution, reparameterizing the initial latent variable distribution, and obtaining an optimized latent variable distribution.

[0063] Among them, a multilayer perceptron can represent a feedforward neural network, which consists of multiple fully connected layers, each of which contains multiple neurons, and the neurons are connected through a weight matrix. Reparameterization can represent random variables as linear combinations of deterministic variables and random variables.

[0064] Specifically, for the node feature representation obtained from the graph neural network , where each Representation Node The feature vector of can be calculated using two independent multilayer perceptrons to calculate the mean of its latent variables and variance , the calculation process is shown in the following formula (2):

[0065] (2)

[0066] in, represents a multilayer perceptron that computes the mean of latent variables, represents a multilayer perceptron that computes the variance of latent variables, Representation Node The feature vector of .

[0067] Then, based on the calculated mean and variance, the initial latent variable distribution is generated: ,in, Indicates the characteristics of a given node In the case of The conditional probability distribution of .

[0068] Next, the initial latent variable distribution is reparameterized and samples are taken from the initial latent variable distribution to ensure that the model is differentiable so that back propagation can be performed. The reparameterization process can be expressed as follows (3):

[0069] (3)

[0070] in, can represent noise sampled from a standard normal distribution.

[0071] Finally, we get the hidden variable set , where each The latent variable distribution corresponding to a node will be used as the input of the subsequent network.

[0072] The latent variable distribution is input into the feature enhancement network based on the self-attention mechanism to obtain enhanced features. The self-attention mechanism can represent a mechanism that allows the model to pay attention to the relationship between different parts of the input sequence. The feature enhancement network can represent a neural network used to further transform and enhance the latent variable representation of each node, which can be a Transformer network. The feature enhancement network based on the self-attention mechanism deeply processes the latent variable distribution through self-attention heads and feedforward neural networks, captures long-range dependency information, enhances feature representation, and finally obtains richer and highly correlated enhanced features, providing better input features for subsequent networks.

[0073] In some embodiments, the latent variable distribution is input into a feature enhancement network based on a self-attention mechanism to obtain enhanced features, which specifically includes the following steps: inputting the latent variable distribution into the feature enhancement network, and the feature enhancement network uses a multi-head self-attention mechanism to obtain multiple parallel attention representations; concatenating and linearly changing the parallel attention representations to obtain an optimized attention representation; and using a feedforward neural network to enhance the optimized attention representation according to the following formula (4) to obtain an enhanced attention representation:

[0074] (4)

[0075] in, Indicates strengthening of attention. represents the optimized attention representation, , represents the weight matrix of the fully connected layer in the feedforward neural network, , represents the bias vector, represents a nonlinear activation function; the enhanced attention representation is normalized and residually connected to obtain enhanced features.

[0076] In the above embodiment, the multi-head self-attention mechanism can be used to capture the dependencies between elements at different positions in the input data sequence, which is an important part of the Transformer architecture. Among them, the multi-head self-attention mechanism can be implemented by the following steps: First, the input latent variable distribution is projected into multiple query, key and value matrices, which can be generated by multiplying with their corresponding weight matrices, and the weight matrix is ​​obtained through training and learning. Then, for each attention head, the attention weight is calculated using the query and key matrices, which can be completed by a dot product operation, that is, the dot product of the query matrix and the transposed key matrix, and then divided by a scaling factor. The obtained score matrix is ​​converted into a probability distribution through a softmax function, that is, the attention weight. Then, the attention weight is multiplied by the value matrix to obtain a weighted sum, that is, the context vector of each position, which is a weighted average of all positions in the input data sequence, and the weight is determined by the attention mechanism. Finally, the context vectors generated by all attention heads are concatenated to form a multidimensional vector, and then linearly transformed through a fully connected layer to obtain the final attention representation.

[0077] A feedforward neural network (FFN) can represent a deep learning network used to perform nonlinear transformations on features, which can enhance the expressiveness of the model. A feedforward neural network can be composed of a series of fully connected layers, each of which contains a set of weight matrices and bias vectors. In the Transformer architecture, the feedforward neural network is usually located after the multi-head self-attention layer to further enhance and adjust the feature representation generated by the attention mechanism.

[0078] In addition, normalization can ensure stable data distribution, prevent gradient vanishing or exploding, and accelerate model convergence. Residual connections can alleviate the gradient vanishing problem in deep networks, accelerate the training process, and improve the generalization ability of the model.

[0079] In some embodiments, the feature enhancement network uses a multi-head self-attention mechanism to obtain multiple parallel attention representations, which specifically includes the following steps: the feature enhancement network performs self-attention calculation on the latent variable distribution according to the following formula (5) to obtain parallel attention representations;

[0080] (5)

[0081] Among them, Attention Q , K , V ) represents the parallel attention representation, represents the dimension of the key vector, Q represents the query matrix, K represents the key matrix, T represents transpose, V represents the value matrix; Q , K , V according to Calculated, where , and Represent the trainable weight matrices, Z represents a set of latent variables.

[0082] After multiple self-attention heads are calculated in parallel, the parallel attention representations are concatenated and linearly transformed through the following formula (6) to obtain the optimized attention representation:

[0083] (6)

[0084] in, , head represents the attention head, Z represents the set of latent variables, h represents the number of attention heads, Represents the linear transformation matrix of the output, which is the optimized attention representation.

[0085] Then, a feedforward neural network is constructed, and the optimized attention representation corresponding to each node is further transformed and enhanced through two fully connected layers and a nonlinear activation function to obtain an enhanced attention representation.

[0086] Next, the enhanced attention representation is normalized and residually connected through the following formula (7) to ensure stable training and gradient flow, and the enhanced features are obtained:

[0087] (7)

[0088] in, represents the feature representation after normalization. represents the feature representation after residual connection, Z represents a set of hidden variables, MultiHead represents the multi-head self-attention mechanism module in the Transformer layer, and FFNN represents the feedforward neural network module in the Transformer layer. Presentation layer standardization.

[0089] The final output enhanced features can be expressed as , each Is a node Enhanced feature representation.

[0090] In some embodiments, the graph neural network, the variational coding network and the feature enhancement network are connected in sequence to obtain an initial feature extraction model.

[0091] Specifically, the graph neural network, variational coding network and feature enhancement network are integrated in sequence to form an end-to-end model framework. The input data of the model framework first passes through the graph neural network to extract features, then generates latent variable distribution through the variational coding network, and then is sent to the feature enhancement network to obtain enhanced features.

[0092] The initial feature extraction model is trained using the graph structure sample data to obtain the target feature extraction model. Specifically, the initial feature extraction model can be iteratively trained and parameter adjusted using a loss function and an optimization algorithm to obtain the final target feature extraction model.

[0093] In some embodiments, the initial feature extraction model is trained for a feature extraction process to obtain a target feature extraction model, which specifically includes the following steps: inputting graph structure sample data into the initial feature extraction model to obtain enhanced features; mapping the enhanced features using a decoder built based on multi-layer perception to obtain classification results of the graph structure sample data; based on the classification results and the true labels corresponding to the graph structure sample data, iteratively training and adjusting parameters of the initial feature extraction model to obtain a target feature extraction model.

[0094] Specifically, a decoder based on multi-layer perception is connected after the feature enhancement network. The decoder can be composed of L fully connected layers, and each layer is connected to a nonlinear activation function to extract and transform features layer by layer. The input feature of the decoder is , No. The weight matrix of the layer is , the bias is The calculation process of each hidden layer in the decoder is shown in the following formula (8):

[0095] (8)

[0096] in, Equal to the input feature , Indicates The linear transformation result of the layer, Indicates The activation output of the layer, represents a non-linear activation function.

[0097] Next, the last layer of the decoder, that is, the output layer, no longer applies a nonlinear activation function, but directly performs a linear transformation on the activation output to obtain the final classification result of the graph structure sample data. , Representation Node The calculation process is shown in the following formula (9):

[0098] (9)

[0099] in, Represents the output classification result, represents the weight matrix of the output layer of the decoder, Indicates The input features of the layer, represents the bias vector of the output layer, Indicates the number of graph convolutional layers.

[0100] In some embodiments, based on the classification results and the true labels corresponding to the graph structure sample data, the initial feature extraction model is iteratively trained and the parameters are adjusted, specifically including the following steps: based on the classification results and the true labels corresponding to the graph structure sample data, the loss function is calculated; based on the loss function, the gradient of the parameters of the initial feature extraction model is determined using a back propagation algorithm; based on the gradient of the parameters, the parameters are updated using an optimization algorithm, and the initial feature extraction model is iteratively trained until the iteration termination condition is met.

[0101] Specifically, the graph structure sample data and its corresponding true labels are obtained, and these data are divided into training set, validation set and test set to evaluate the model performance and generalization ability. Then, the training set is input into the initial feature extraction model, and the classification result is obtained after being processed by the graph neural network, variational coding network, feature enhancement network and decoder. Secondly, based on the classification result and the true label of the graph structure sample data, the loss function is calculated. The loss function includes reconstruction error and KL divergence, and its formula is shown in the following formula (10):

[0102] (10)

[0103] in, represents the posterior distribution expectations, represents the prior distribution, represents a hidden variable, z represents observed data, KL represents Kullback-Leibler divergence, which is used to measure the difference between the variational distribution and the prior distribution. Represents the loss function.

[0104] Next, based on the calculated loss function, the back propagation algorithm is used to determine the gradient of the parameters of the initial feature extraction model. The chain rule is used to calculate the gradient layer by layer, from the output layer to the input layer. Finally, based on the calculated gradient, the optimization algorithm is used to update the model parameters. The specific formula is shown in the following formula (11):

[0105] (11)

[0106] in, represents the model parameters, represents the learning rate, represents the loss function, Represents the updated model parameters.

[0107] At the same time, the model performance is evaluated on the validation set, and the hyperparameters and model structure are adjusted to prevent overfitting. Then, cross-validation and other methods are used to ensure that the model has good generalization ability. The process of forward propagation, loss calculation, back propagation and parameter update is repeated until the performance of the model on the validation set is no longer significantly improved or the preset number of training times is reached. Finally, the performance of the final model is evaluated on the test set to verify the actual effect and generalization ability of the model until the iteration termination condition is met, and the target feature extraction model is obtained.

[0108] The training method of the feature extraction model in the above example embodiment is that, on the one hand, the graph neural network processes the graph structure sample data through the graph convolution operation to extract the feature representation of the node, which can not only aggregate the information of the node and its neighborhood, thereby learning the local relationship and structural characteristics between the nodes, but also reduce the original high-dimensional node features to a lower-dimensional potential feature space, thereby reducing the complexity of the model and improving the training efficiency; in addition, the feature representation corresponding to the node is converted into a latent variable distribution, and the variational inference method is used to capture the potential structure of the data, thereby enhancing the expressive power of the model; on the other hand, the enhanced features are obtained by using the feature enhancement network based on the self-attention mechanism, which can capture the complex long-range dependencies between nodes, thereby better understanding the global structure of the data; on the other hand, the network composed of the graph neural network, the variational encoding network and the feature enhancement network connected in sequence is trained to obtain the target feature extraction model, which integrates local structural information, potential structure and long-range dependencies, thereby being able to more comprehensively understand the data and effectively extract the deep features of the graph structure data.

[0109] The present disclosure also provides a schematic diagram of the architecture of a feature extraction model, see Figure 2 As shown in the figure, the feature extraction model mainly consists of four parts: graph neural network layer, variational encoder layer, Transformer layer and decoder layer. The input data is graph structure data, each graph consists of nodes and edges, and each node has its corresponding features.

[0110] The graph neural network layer propagates local information to the entire graph through multi-layer graph convolution operations, generating feature representations for each node. These features take into account the information of the node itself and the relationship between the node and other nodes. The main function of the graph neural network is to extract features from graph structure data.

[0111] The variational encoder is a multi-layer perceptron that receives data features from the graph neural network layer and generates a latent variable distribution, which can be assumed to be a Gaussian distribution and is obtained by calculating the mean and variance of the features. Through reparameterization, feature latent variables can be sampled from this distribution. The main function of the variational encoder is to convert feature representation into probability distribution, introducing uncertainty modeling.

[0112] The Transformer layer includes a multi-head attention mechanism module, a feedforward neural network module, and a normalization and residual connection module. Among them, the multi-head attention mechanism allows the model to focus on different aspects of the input at the same time. Each attention head calculates the attention score independently, and then splices them together to form an optimized attention representation. This mechanism can capture the complex structure and long-range dependencies of the input data. The feedforward neural network further enhances the feature representation to obtain an enhanced attention representation. Through nonlinear transformation and multi-layer design, the expressive power of the model is improved. The role of the feedforward neural network is to enhance the feature representation and make it more suitable for downstream tasks. Normalization and residual connection ensure the training stability of the model and the smooth flow of gradients, which helps the convergence speed and performance of the model. Normalization can prevent gradients from exploding or disappearing, while residual connections help the model learn complex mappings more easily.

[0113] The decoder layer includes a fully connected layer and an output layer, whose main function is to convert the enhanced features into the required output format, such as classification labels, regression values, or other forms of predictions.

[0114] In the feature extraction model of the above embodiment, the graph neural network layer is first applied to the graph structure data, and the features of the nodes are extracted layer by layer through multi-layer graph convolution operations to capture local and global structural information. Subsequently, these feature representations are input into the variational encoder, and the latent variable distribution is generated using the variational inference method to effectively capture the potential structure of the data and enhance the expressiveness of the model. The latent variable distribution generated by the variational encoder is then input into the Transformer layer, and the complex relationship between the nodes is further modeled through the self-attention mechanism to capture long-range dependency information. The Transformer layer effectively models non-local information through a multi-head self-attention mechanism and a feedforward neural network, thereby improving the expressiveness and predictive performance of the model. Finally, the output of the Transformer layer is input into the decoder, and the deep features are mapped to the prediction results of the specific task through a multi-layer perceptron. The entire model is integrated in an end-to-end training framework, optimized by the back-propagation algorithm, and the loss function is minimized to ensure the efficiency and performance of the model.

[0115] Furthermore, in an embodiment of the present disclosure, a data classification method is also provided, which includes the following steps: obtaining data to be classified, wherein the data to be classified is graph structure data; inputting the data to be classified into a target feature extraction model to obtain a classification result of the data to be classified; wherein the target feature extraction model is trained according to the training method of the feature extraction model in the above embodiment.

[0116] The data to be classified can be any form of graph structure data, such as user relationships in social networks, protein-protein interactions, literature citation networks, traffic network information, etc. Specifically, for user relationship data, the friendship relationships between users constitute a graph, each user is a node, and the connections between them are edges. By analyzing this graph structure data, users' interest preferences, social circles, etc. can be predicted; protein-protein interaction data can also be represented by a graph. By classifying these graph structure data, we can better understand the complex interactions in organisms and identify potential disease markers; for literature citation network data, the citation relationship between academic papers constitutes a literature citation network, each paper is a node, and the citation relationship is an edge. By classifying the literature citation network, we can identify research hotspots or paper types in a certain field; for traffic network information data, the road network, public transportation routes, etc. in the city can also be represented by a graph, each intersection or station is a node, and the road or line is an edge. By classifying these graph structure data, we can optimize traffic flow management and predict congestion.

[0117] Figure 3The flowchart of the data classification method according to some embodiments of the present disclosure is schematically shown. Specifically, the data to be classified is the Cora dataset, which contains 2708 scientific publications, which are divided into 7 categories, and the reference relationship between each publication constitutes 5429 edges. Each node has a 1433-dimensional feature, which represents the occurrence of words in the vocabulary. The specific data classification process includes the following steps:

[0118] First, we prepare graph data and extract graph data from the Cora dataset. Nodes represent scientific publications, and edges represent citation relationships between publications. Node features are represented by 1433-dimensional word occurrences. The dataset is divided into training, validation, and test sets for model training, validation, and testing.

[0119] Secondly, the graph convolution is performed using the graph neural network layer to obtain the feature representation corresponding to the nodes in the input data. Specifically, the initial feature of each node in the input graph structure data is a 1433-dimensional feature vector. Two layers of graph convolution operations are set to extract node feature representations, and RELU is selected as the activation function. The first graph convolution layer reduces the node feature from 1433 dimensions to 128 dimensions based on the following formula (12):

[0120] (12)

[0121] in, Representation Node The corresponding feature representation of the first graph convolutional layer output, Representation Node The set of neighboring nodes of and Respectively represent nodes and nodes Normalization constant, represents the weight matrix of the first graph convolutional layer, represents a nonlinear activation function, Representation Node j The initial eigenvector of .

[0122] The second graph convolutional layer further reduces the 128-dimensional node features to 64 dimensions based on the following formula (13) to obtain the feature representation corresponding to the node:

[0123] (13)

[0124] in, Representation Node The corresponding feature representation of the second graph convolutional layer output, Representation Node The set of neighboring nodes of and Respectively represent nodes and nodes Normalization constant, represents the weight matrix of the second graph convolutional layer, represents a nonlinear activation function, Representation Node j Feature vector at the output of the first graph convolutional layer.

[0125] Then, the feature representation output by the graph neural network layer is converted into a latent variable distribution using the variational encoder layer. Specifically, the 64-dimensional feature representation output by the graph neural network layer is input into the variational encoder layer. Two layers of multilayer perceptrons are set in the input variational encoder layer. The first multilayer perceptron converts the 64-dimensional feature representation into 32 dimensions, and the second multilayer perceptron converts the 32-dimensional feature representation into 16 dimensions. Then, the mean and variance of the latent variable are calculated based on the following formula (14):

[0126] (14)

[0127] in, represents the activation function. In this embodiment, the RELU function can be used as the activation function. represents the mean of the latent variable, represents the variance of the latent variable, W represents the weight matrix, b represents the bias vector, Representation Node The corresponding feature representation of the second graph convolutional layer output. Then, according to the mean and variance of the latent variable, the initial latent variable distribution is obtained, sampling is performed from the initial latent variable distribution, and the initial latent variable distribution is reparameterized to obtain the optimized latent variable distribution.

[0128] Next, the Transformer layer is used to enhance the features of the latent variable distribution output by the variational encoder to obtain enhanced features. The 16-dimensional latent variable distribution is input into the Transformer layer, and then the feature representation is enhanced through the multi-head self-attention mechanism. Four attention heads are used, including the first attention head, the second attention head, the third attention head, and the fourth attention head. The dimension of each attention head is 4. The feature enhancement process can be expressed as follows (15):

[0129] (15)

[0130] Among them, Attention Q , K , V ) represents the parallel attention representation, represents the dimension of the key vector, Q represents the query matrix,K represents the key matrix, T represents transpose, V represents the value matrix, Q , K , V The matrix dimension is 4×16. Next, the output of multi-head self-attention is concatenated and linearly transformed. The concatenated dimension is 16 dimensions, and the dimension is still 16 dimensions after linear transformation. Then the feature representation is further transformed using a feedforward neural network. The feedforward neural network is set up with two layers, namely the first feedforward neural network and the second feedforward neural network. The first layer converts the 16-dimensional features into 32 dimensions, and the second layer converts the 32-dimensional features into 16 dimensions. Finally, application layer normalization and residual connection are used, and the enhanced feature representation of each node is 16 dimensions.

[0131] Finally, the decoder layer converts the enhanced features into specific classification results. Specifically, the 16-dimensional enhanced features are input into the decoder layer, and the enhanced feature representation is converted into the prediction results of the specific task through the multilayer perceptron. The decoder layer includes two layers of decoders, namely the first multilayer perceptual decoder and the second multilayer perceptual decoder. The first multilayer perceptual decoder converts the 16-dimensional features into 32 dimensions, and the second multilayer perceptual decoder converts the 32-dimensional features into 7 dimensions, which are used for the 7 categories of the corresponding data set, including the first category, the second category, the third category, the fourth category, the fifth category, the sixth category and the seventh category. Then, the loss function is calculated based on the prediction results output by the model and the actual publication classification labels. The cross entropy loss function is used to measure the difference between the prediction results and the actual labels, and the KL divergence is calculated to measure the difference between the latent variable distribution and the prior distribution. The gradient of the model parameters is calculated using the backpropagation algorithm. The gradient of each layer parameter is calculated backward from the output layer layer by layer through the chain rule. The model parameters are updated using the Adam optimization algorithm, which adjusts the update step size of each parameter through an adaptive learning rate, which can improve the training efficiency of the model. Evaluate the model performance on the validation set, and calculate the loss value and other performance indicators of the validation set, such as accuracy, precision, or recall. According to the validation results, adjust the model's hyperparameters and model structure to prevent overfitting and improve the model's generalization ability. Evaluate the performance of the final model on the test set. The test set is used to verify the performance of the model on unknown data and evaluate its actual effect and generalization ability. Record the performance indicators of the test set to ensure the consistency and reliability of the model on different data sets.

[0132] It should be noted that, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0133] Next, in the embodiment of the present disclosure, a training device for a feature extraction model is also provided. Figure 4 As shown in , the training device 400 of the feature extraction model can be composed of a sample construction module 401 and a model training module 402. Among them: the sample construction module 401 can be used to obtain training sample data, and the training sample data is constructed by graph structure sample data and true labels; the model training module 402 can be used to input the graph structure sample data into a pre-constructed initial feature extraction model to train the initial feature extraction model for feature extraction process to obtain a target feature extraction model; wherein the feature extraction process includes: using a graph neural network to perform graph convolution processing on the graph structure sample data to obtain feature representations corresponding to nodes in the graph structure sample data; inputting the feature representation into a variational coding network to convert the feature representation into a latent variable distribution; inputting the latent variable distribution into a feature enhancement network based on a self-attention mechanism to obtain enhanced features.

[0134] In the embodiment of the present disclosure, a data classification device is also provided, referring to Figure 5 As shown in , the data classification device 500 can be composed of a data acquisition module 501 and a data classification module 502. Among them: the data acquisition module 501 can be used to obtain the data to be classified, and the data to be classified is graph structure data; the data classification module 502 can be used to input the data to be classified into a target feature extraction model to obtain the classification result of the data to be classified; wherein the target feature extraction model is trained according to the training method of the above feature extraction model.

[0135] It should be noted that the specific details of each part of the training device of the above-mentioned feature extraction model have been described in detail in the implementation method of the training method of the feature extraction model. The undisclosed details can be found in the implementation method of the method part, and will not be repeated here.

[0136] In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above-mentioned feature extraction model training method is also provided.

[0137] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware embodiments, complete software embodiments (including firmware, microcode, etc.), or embodiments combining hardware and software aspects, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0138] Refer to the following Figure 6 hereinafter describes an electronic device 600 according to such an embodiment of the present disclosure. Figure 6 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0139] like Figure 6 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include but are not limited to: the at least one processing unit 610, the at least one storage unit 620, a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610), and a display unit 640.

[0140] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 executes the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.

[0141] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 621 and / or a cache memory unit 622 , and may further include a read-only memory unit (ROM) 623 .

[0142] The storage unit 620 may also include a program / utility 624 having a set (at least one) of program modules 625, such program modules 625 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0143] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0144] The electronic device 600 may also communicate with one or more external devices 670 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication may be performed through an input / output (I / O) interface 650. In addition, the electronic device 600 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter 660. As shown, the network adapter 660 communicates with other modules of the electronic device 600 through a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0145] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiment of the present disclosure.

[0146] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of the present specification.

[0147] refer to Figure 7 As shown, a program product 700 for implementing the training method of the feature extraction model described above according to an embodiment of the present disclosure is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus, or a device.

[0148] The program product may adopt any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0149] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0150] The program code contained on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, electromagnetic waves, etc., or any suitable combination of the foregoing.

[0151] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0152] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0153] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiment of the present disclosure.

[0154] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0155] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for training a feature extraction model, characterized in that: include: Acquire training sample data, wherein the training sample data is constructed by graph structure sample data and true labels, wherein the graph structure sample data is traffic network information data, intersections represent nodes of the traffic network information data, and roads represent edges of the traffic network information data; Inputting the graph structure sample data into a pre-built initial feature extraction model to obtain enhanced features; Mapping the enhanced features using a decoder built based on multi-layer perception to obtain a classification result of the graph structure sample data; Based on the classification result and the real label corresponding to the graph structure sample data, iteratively train and adjust parameters of the initial feature extraction model to obtain a target feature extraction model; Wherein, the feature extraction process includes: Using multiple graph convolutional layers in the graph neural network, according to: ； Performing graph convolution processing on the graph structure sample data, extracting feature representations corresponding to nodes in the graph structure sample data layer by layer; wherein, represents the characteristic representation, Representation Node The set of neighboring nodes of represents the normalization constant, Indicates The weight matrix of the layer, represents a nonlinear activation function, Represents the number of layers of the graph convolutional layer; Inputting the feature representation into a variational coding network, and using a multilayer perceptron in the variational coding network to obtain the mean and variance of latent variables corresponding to the feature representation; According to the mean and variance of the latent variable, an initial latent variable distribution is obtained; Sampling from the initial latent variable distribution, performing reparameterization processing on the initial latent variable distribution, and obtaining an optimized latent variable distribution; The latent variable distribution is input into the feature enhancement network, and the feature enhancement network is based on: ； Perform self-attention calculation on the latent variable distribution to obtain parallel attention representation; where Attention ( Q , K , V ) represents the parallel attention representation, represents the dimension of the key vector, Q represents the query matrix, T represents transpose, K represents the key matrix, V represents the value matrix; Q , K , V according to Calculated, among which, , and Represent the trainable weight matrices, Z represents a set of latent variables; Concatenating and linearly transforming the parallel attention representations to obtain an optimized attention representation; Using a feedforward neural network, according to: ； The optimized attention representation is enhanced to obtain a reinforced attention representation; wherein, represents the enhanced attention representation, represents the optimized attention representation, , represents the weight matrix of the fully connected layer in the feedforward neural network, , represents the bias vector, represents a nonlinear activation function; The enhanced attention representation is normalized and residually connected to obtain the enhanced feature.

2. The method for training a feature extraction model according to claim 1, characterized in that: The iterative training and parameter adjustment of the initial feature extraction model based on the classification result and the real label corresponding to the graph structure sample data includes: Calculating a loss function based on the classification result and the true label corresponding to the graph structure sample data; Determining the gradient of the parameters of the initial feature extraction model using a back propagation algorithm according to the loss function; According to the gradient of the parameter, the parameter is updated by using an optimization algorithm, and the initial feature extraction model is iteratively trained until an iteration termination condition is met.

3. A data classification method, characterized in that: include: Acquire data to be classified, wherein the data to be classified is graph structure data, the graph structure data is traffic network information data, intersections represent nodes of the traffic network information data, and roads represent edges of the traffic network information data; Inputting the data to be classified into a target feature extraction model to obtain a classification result of the data to be classified; Wherein, the target feature extraction model is trained according to the feature extraction model training method as described in any one of claims 1-2.

4. A training device for a feature extraction model, characterized in that: include: A sample construction module is used to obtain training sample data, wherein the training sample data is constructed by graph structure sample data and true labels, wherein the graph structure sample data is traffic network information data, intersections represent nodes of the traffic network information data, and roads represent edges of the traffic network information data; A model training module, used for inputting the graph structure sample data into a pre-built initial feature extraction model to obtain enhanced features; Mapping the enhanced features using a decoder built based on multi-layer perception to obtain a classification result of the graph structure sample data; Based on the classification result and the real label corresponding to the graph structure sample data, iteratively train and adjust parameters of the initial feature extraction model to obtain a target feature extraction model; Wherein, the feature extraction process includes: Using multiple graph convolutional layers in the graph neural network, according to: ； Performing graph convolution processing on the graph structure sample data, extracting feature representations corresponding to nodes in the graph structure sample data layer by layer; wherein, represents the characteristic representation, Representation Node The set of neighboring nodes of represents the normalization constant, Indicates The weight matrix of the layer, represents a nonlinear activation function, Represents the number of layers of the graph convolutional layer; Inputting the feature representation into a variational coding network, and using a multilayer perceptron in the variational coding network to obtain the mean and variance of latent variables corresponding to the feature representation; According to the mean and variance of the latent variable, an initial latent variable distribution is obtained; Sampling from the initial latent variable distribution, performing reparameterization processing on the initial latent variable distribution, and obtaining an optimized latent variable distribution; The latent variable distribution is input into the feature enhancement network, and the feature enhancement network is based on: ； Perform self-attention calculation on the latent variable distribution to obtain parallel attention representation; where Attention ( Q , K , V ) represents the parallel attention representation, represents the dimension of the key vector, Q represents the query matrix, T represents transpose, K represents the key matrix, V represents the value matrix; Q , K , V according to Calculated, among which, , and Represent the trainable weight matrices, Z represents a set of latent variables; Concatenating and linearly transforming the parallel attention representations to obtain an optimized attention representation; Using a feedforward neural network, according to: ； The optimized attention representation is enhanced to obtain a reinforced attention representation; wherein, represents the enhanced attention representation, represents the optimized attention representation, , represents the weight matrix of the fully connected layer in the feedforward neural network, , represents the bias vector, represents a nonlinear activation function; The enhanced attention representation is normalized and residually connected to obtain the enhanced feature.

5. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the training method of the feature extraction model described in any one of claims 1-2 by executing the executable instructions.

Images