Method and apparatus for training neural network model, and electronic device

Abstract

The application discloses a neural network model training method and device and electronic equipment, and belongs to the technical field of Internet. The method obtains a first feature vector corresponding to each of a plurality of first nodes and a weight matrix corresponding to a plurality of association relationships between the plurality of first nodes, integrates the plurality of weight matrices based on a first quantity of the plurality of association relationships to obtain a plurality of first weight parameters, iteratively optimizes the plurality of first weight parameters based on the first feature vector corresponding to each of the plurality of first nodes until the plurality of first weight parameters meet a convergence condition to obtain a plurality of second weight parameters, and performs model training based on the plurality of second weight parameters and the first feature vector corresponding to each of the plurality of first nodes to obtain a target neural network model. The method integrates one weight matrix into one weight parameter, which can greatly reduce the parameter quantity, alleviate the problem of over-parameterization, and improve the model training efficiency.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to a method, apparatus and electronic device for training a neural network model. Background Technology

[0002] In recent years, graph neural networks (GNNs) have been increasingly widely used in various fields such as social networks and biomedicine. GNNs use neural networks to learn graph-structured data to perform tasks such as classification and prediction. Graph-structured data includes nodes and edges. Nodes are entities with features, such as products, images, and videos, while edges represent the relationships between two connected nodes. Because there are often multiple relationships between nodes, multi-relationship graph neural networks have become a focus of current research.

[0003] In related technologies, when training a multi-relationship graph neural network, for multiple nodes in the graph structure data, the feature vectors corresponding to each node and the weight matrices corresponding to the various relationships between the nodes are determined. Each relationship corresponds to a weight matrix, which is used to represent the importance of that relationship. The model is trained based on the feature vectors corresponding to the multiple nodes and the multiple weight matrices to obtain the neural network model.

[0004] However, the weight matrix is ​​generally an n×m matrix, meaning that a weight matrix contains n×m weight parameters, which are used to represent the similarity between two nodes that have a relationship. Both n and m are integers greater than 0. Since one weight matrix corresponds to one relationship, as the number of relationships increases, the number of weight matrices and the number of parameters will also increase, resulting in excessively long model training time and low model training efficiency. Summary of the Invention

[0005] This application provides a method, apparatus, and electronic device for training a neural network model, which can improve model training efficiency. The technical solution is as follows:

[0006] On the one hand, a method for training a neural network model is provided, which includes:

[0007] Obtain the first feature vectors corresponding to multiple first nodes and the weight matrices corresponding to various association relationships between the multiple first nodes, with one weight matrix corresponding to one association relationship;

[0008] Based on the first number of the various associations, the multiple weight matrices are integrated to obtain multiple first weight parameters. Each weight matrix corresponds to one first weight parameter, and the first weight parameter is used to represent the importance of the association.

[0009] Based on the first feature vectors corresponding to the plurality of first nodes, the plurality of first weight parameters are iteratively optimized until the plurality of first weight parameters meet the convergence condition, thereby obtaining a plurality of second weight parameters.

[0010] Based on the multiple second weight parameters and the first feature vectors corresponding to the multiple first nodes, the model is trained to obtain the target neural network model.

[0011] On the one hand, a training device for a neural network model is provided, the device comprising:

[0012] The first acquisition module is used to acquire the first feature vectors corresponding to multiple first nodes and the weight matrices corresponding to various association relationships between the multiple first nodes, with one association relationship corresponding to one weight matrix.

[0013] An integration module is used to integrate multiple weight matrices based on a first number of the various associations to obtain multiple first weight parameters. Each weight matrix corresponds to one first weight parameter, and the first weight parameter is used to represent the importance of the association.

[0014] The optimization module is used to iteratively optimize the multiple first weight parameters based on the first feature vectors corresponding to the multiple first nodes, until the multiple first weight parameters meet the convergence condition, and obtain multiple second weight parameters.

[0015] The training module is used to train the model based on the plurality of second weight parameters and the first feature vectors corresponding to the plurality of first nodes, so as to obtain the target neural network model.

[0016] On one hand, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one line of program code, which is loaded and executed by the processor to implement a training method for a neural network model as described in any of the possible implementations above.

[0017] On the one hand, a computer-readable storage medium is provided, wherein at least one piece of program code is stored in the computer-readable storage medium, the at least one piece of program code being loaded and executed by a processor to implement a method for training a neural network model as described in any of the possible implementations above.

[0018] On the one hand, a computer program product is provided, wherein at least one piece of program code is stored in the computer program product, the at least one piece of program code being loaded and executed by a processor to implement a training method for a neural network model as described in any of the above possible implementations.

[0019] The beneficial effects of the technical solutions provided in this application include at least the following:

[0020] This application provides a method for training a neural network model. When training the neural network model, this method integrates multiple weight matrices based on the number of various correlations to obtain multiple first weight parameters. Since one correlation corresponds to one weight matrix, and one weight matrix corresponds to one first weight parameter, each correlation corresponds to one first weight parameter. Therefore, this method integrates a weight matrix into a single weight parameter, which significantly reduces the number of parameters, alleviates the problem of overparameterization, and thus improves model training efficiency. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the implementation environment of a neural network model training method provided in an embodiment of this application;

[0023] Figure 2 This is a flowchart illustrating a method for training a neural network model according to an embodiment of this application;

[0024] Figure 3 This is a flowchart illustrating a method for training a neural network model according to an embodiment of this application;

[0025] Figure 4 This is a schematic diagram of a training neural network model provided in an embodiment of this application;

[0026] Figure 5 This is a flowchart illustrating how a target node is classified using a target neural network model, as provided in an embodiment of this application.

[0027] Figure 6 This is a schematic diagram of the structure of a training device for a neural network model provided in an embodiment of this application;

[0028] Figure 7 This is a structural block diagram of a terminal provided in an embodiment of this application;

[0029] Figure 8 This is a structural block diagram of a server provided in an embodiment of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0031] The terms "first," "second," "third," and "fourth," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0032] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the correlations, feature vectors, weight parameters, etc. involved in this application were obtained with full authorization.

[0033] Figure 1 This is a schematic diagram illustrating the implementation environment of a neural network model training method provided in an embodiment of this application. See also... Figure 1 The implementation environment includes an electronic device, which can be provided as a terminal 101, a server 102, or both a terminal 101 and a server 102. If the electronic device is provided as a terminal 101 and a server 102, the terminal 101 and the server 102 are connected via a wireless or wired network. In this embodiment, the electronic device is not specifically limited.

[0034] If the electronic device is provided as terminal 101, then terminal 101 performs model training to obtain the target neural network model. This target neural network model can be deployed in terminal 101, so that terminal 101 can complete the task through the target neural network model.

[0035] If the electronic device is provided as server 102, then server 102 performs model training to obtain the target neural network model. This target neural network model can be deployed to terminal 101. Accordingly, the implementation environment also includes terminal 101, which can be connected to server 102 via a wireless or wired network. Terminal 101 can complete tasks using the target neural network model.

[0036] If the electronic device provides a terminal 101 and a server 102, the server 102 performs model training to obtain a target neural network model, and then the target neural network model is deployed to the terminal 101, so that the terminal 101 can complete the task through the target neural network model.

[0037] In this embodiment, the target neural network model can perform classification tasks, for example, the terminal 101 can classify goods or images using the target neural network model. The target neural network model can perform recommendation tasks, for example, the terminal 101 can recommend goods, videos, or images to users using the target neural network model. The target neural network model can also perform prediction tasks, for example, the terminal 101 can predict molecular structures using the target neural network model. The target neural network model can also perform search tasks, for example, the terminal 101 can search for content related to search terms using the target neural network model. The target neural network model can also perform other types of tasks, which will not be listed here.

[0038] Terminal 101 is at least one of a mobile phone, tablet computer, and PC (Personal Computer) device. Server 102 can be at least one of a single server, a server cluster consisting of multiple servers, a cloud server, a cloud computing platform, and a virtualization center.

[0039] Figure 2 This is a flowchart of a neural network model training method provided in an embodiment of this application. See also... Figure 2 The method includes:

[0040] Step 201: Obtain the first feature vectors corresponding to the multiple first nodes and the weight matrices corresponding to the various association relationships between the multiple first nodes. Each association relationship corresponds to a weight matrix.

[0041] Step 202: Based on the first number of multiple associations, integrate the multiple weight matrices to obtain multiple first weight parameters. One weight matrix corresponds to one first weight parameter, and the first weight parameter is used to represent the importance of the association.

[0042] Step 203: Based on the first feature vectors corresponding to the multiple first nodes, iteratively optimize the multiple first weight parameters until the multiple first weight parameters meet the convergence condition, and obtain multiple second weight parameters.

[0043] Step 204: Based on multiple second weight parameters and the first feature vectors corresponding to multiple first nodes, perform model training to obtain the target neural network model.

[0044] In one possible implementation, based on the first feature vectors corresponding to multiple first nodes, multiple first weight parameters are iteratively optimized until the multiple first weight parameters satisfy the convergence condition, resulting in multiple second weight parameters, including:

[0045] Obtain the first relation data, which represents the relationship between two independent variables, namely the first feature vector and the first weight parameter;

[0046] Based on the first relation data, the second relation data is obtained by keeping the first feature vectors corresponding to multiple first nodes unchanged, with the first weight parameter as the independent variable and the first weight parameter as the independent variable.

[0047] Based on the second relation data, the current layer neural network iteratively optimizes multiple first weight parameters until the multiple first weight parameters meet the convergence condition, thus obtaining multiple second weight parameters.

[0048] In another possible implementation, obtaining the first relational data includes:

[0049] Determine the first vector matrix and the Laplacian matrix corresponding to each of the multiple association relationships. The first vector matrix is ​​composed of the first eigenvectors corresponding to multiple first nodes, and each association relationship corresponds to one Laplacian matrix.

[0050] Using the first vector matrix as the independent variable, determine the feature term relation data in the first relation data;

[0051] Based on the first vector matrix and the Laplace matrix corresponding to various association relationships, the first parameter item relationship data in the first relationship data is determined by taking the first weight parameter corresponding to various association relationships as independent variables. One association relationship corresponds to one Laplace matrix.

[0052] Using the first weight parameters corresponding to various relationships as independent variables, determine the second parameter item relationship data in the first relationship data;

[0053] Based on the feature term relationship data, the first parameter term relationship data, and the second parameter term relationship data, the first relationship data is determined.

[0054] In another possible implementation, the process of iteratively optimizing the first weight parameters using the current layer neural network based on the second relation data until the first weight parameters meet the convergence condition to obtain the second weight parameters includes:

[0055] The second relation data is optimized to obtain the third relation data. The third relation data is used to represent the relationship optimized based on the weight ratio coefficient, the first optimization matrix, the first optimization parameter, and the first quantity. The first weight parameter is optimized as the number of iterations increases. The weight ratio coefficient is determined based on the first weight and the second weight. The first weight is the weight of the first parameter item relation data in the first relation data, and the second weight is the weight of the second parameter item relation data in the first relation data. The first optimization matrix and the first optimization parameter are both used to optimize the first weight parameter corresponding to the association relationship.

[0056] Based on the first vector matrix and the Laplacian matrix corresponding to various association relationships, the first optimization matrix corresponding to various association relationships is determined, with one first optimization matrix corresponding to one association relationship.

[0057] The first optimization parameters are determined based on the weight ratio coefficients and the first optimization matrices corresponding to various correlation relationships.

[0058] Substitute the first quantity, weight ratio coefficient, first optimization parameter, and first optimization matrix corresponding to the association relationship into the third relationship data. Based on the number of iterations, iteratively optimize the first weight parameter through the current layer neural network until the first weight parameter meets the convergence condition, and obtain the second weight parameter.

[0059] In another possible implementation, the first quantity, weight ratio coefficient, first optimization parameter, and the first optimization matrix corresponding to the association relationship are substituted into the third relationship data. Based on the number of iterations, the first weight parameter is iteratively optimized through the current layer neural network until the first weight parameter satisfies the convergence condition, thus obtaining the second weight parameter, including:

[0060] Substitute the first quantity, weight ratio coefficient, first optimization parameter, first optimization matrix corresponding to the correlation relationship, and the current iteration optimization number into the third relation data, and optimize the current first weight parameter through the current layer neural network to obtain the optimized first weight parameter;

[0061] Increment the current iteration count by 1 to obtain the updated iteration count;

[0062] The optimized first weight parameter is used as the current first weight parameter, and the updated iteration optimization number is used as the current iteration optimization number. The first quantity, weight ratio coefficient, first optimization parameter, first optimization matrix corresponding to the association relationship and the current iteration optimization number are respectively substituted into the third relationship data. The current first weight parameter is optimized through the current layer neural network to obtain the optimized first weight parameter.

[0063] The second weight parameter is obtained when the optimized first weight parameter meets the convergence condition.

[0064] In another possible implementation, the first optimization parameters are determined based on the weight ratio coefficients and the first optimization matrices corresponding to various correlation relationships, including:

[0065] Determine the traces of the first optimization matrix corresponding to various correlation relationships to obtain multiple traces;

[0066] Determine the norm of a trace matrix consisting of multiple traces;

[0067] The sum of the norm and the weighting coefficient is determined as the first optimization parameter.

[0068] In another possible implementation, the process of determining the first optimized matrix corresponding to the association relationship based on the first vector matrix and the Laplacian matrix corresponding to the association relationship includes:

[0069] Determine the transpose of the first vector matrix to obtain the second vector matrix;

[0070] The product of the second vector matrix, the first vector matrix, and the Laplace matrix is ​​determined to obtain the first optimized matrix.

[0071] In another possible implementation, the model is trained based on multiple second weight parameters and the first feature vectors corresponding to multiple first nodes to obtain the target neural network model, including:

[0072] Based on the first relational data, the fourth relational data is obtained by using the first feature vector as the independent variable and keeping multiple second weight parameters unchanged. The first relational data is used to represent the relationship between two independent variables, namely the first feature vector and the first weight parameter.

[0073] Based on the fourth relation data, the first feature vectors corresponding to multiple first nodes are optimized by the current neural network to obtain the second feature vectors corresponding to multiple first nodes.

[0074] Based on the second feature vectors corresponding to multiple first nodes, the next layer of the neural network iteratively optimizes the multiple second weight parameters to obtain multiple third weight parameters, until the last layer of the neural network outputs the third feature vectors corresponding to multiple first nodes.

[0075] The target neural network model is obtained by training the model based on the third feature vectors corresponding to multiple first nodes.

[0076] In another possible implementation, the process of optimizing the first feature vector corresponding to the first node through the current neural network based on the fourth relation data to obtain the second feature vector corresponding to the first node includes:

[0077] The fourth relation data is optimized to obtain the fifth relation data. The fifth relation data is used to represent the relationship based on the first weight, the fourth feature vector corresponding to at least one second node, and the second weight parameter corresponding to at least one association relationship. The first feature vector is optimized as the number of neural network layers increases. The first weight is the weight of the first parameter item relation data in the first relation data. At least one second node is a node among multiple first nodes that has at least one association relationship with the first node that is the center node.

[0078] Based on the fourth feature vector corresponding to at least one second node and at least one second weight parameter corresponding to at least one association relationship, the fifth feature vector corresponding to the first node under at least one association relationship is output through the current layer neural network. The fifth feature vector is used to represent the node information of at least one second node. The fourth feature vector and at least one second weight parameter are both output through the previous layer neural network of the current layer neural network.

[0079] Substitute the first weight, the first feature vector corresponding to the first node, and the fifth feature vector into the fifth relation data, and output the second feature vector corresponding to the first node through the current layer neural network.

[0080] In another possible implementation, the first weight, the first feature vector corresponding to the first node, and the fifth feature vector are substituted into the fifth relation data, and the second feature vector corresponding to the first node is output through the current layer neural network, including:

[0081] Based on the first weight, the third and fourth weights are determined. The third weight is used to represent the importance of the first eigenvector in the second eigenvector, and the fourth weight is used to represent the importance of the fifth eigenvector in the second eigenvector.

[0082] Substitute the third weight, fourth weight, first feature vector, and fifth feature vector into the fifth relation data, and output the second feature vector through the current layer neural network.

[0083] In another possible implementation, the target neural network model is obtained by training the model based on the third feature vectors corresponding to multiple first nodes, including:

[0084] Based on the third feature vectors corresponding to multiple first nodes, the predicted labels corresponding to multiple first nodes are determined.

[0085] Based on the label loss values ​​between the predicted labels and the actual labels corresponding to multiple first nodes, the model is trained to obtain the target neural network model.

[0086] In another possible implementation, the process of integrating the weight matrix based on a first number of multiple associations to obtain the first weight parameters includes:

[0087] The first weight parameter is obtained by integrating multiple weight parameters in the weight matrix into the reciprocal of the first number.

[0088] In another possible implementation, the method also includes:

[0089] Obtain the sixth feature vector of the target node to be classified;

[0090] The sixth feature vector of the target node is extracted to obtain the seventh feature vector of the target node;

[0091] Based on the target neural network model, the predicted label of the target node is determined.

[0092] The target nodes are classified based on their predicted labels.

[0093] This application provides a method for training a neural network model. When training the neural network model, this method integrates multiple weight matrices based on the number of various correlations to obtain multiple first weight parameters. Since one correlation corresponds to one weight matrix, and one weight matrix corresponds to one first weight parameter, each correlation corresponds to one first weight parameter. Therefore, this method integrates a weight matrix into a single weight parameter, which significantly reduces the number of parameters, alleviates the problem of overparameterization, and thus improves model training efficiency.

[0094] Figure 3 This is a flowchart illustrating a neural network model training method provided in an embodiment of this application, executed by an electronic device. See also... Figure 3 The method includes:

[0095] Step 301: The electronic device obtains the first feature vectors corresponding to the multiple first nodes and the weight matrices corresponding to the various correlation relationships between the multiple first nodes.

[0096] In this step, for any first node, the process of the electronic device obtaining the first feature vector corresponding to the first node can be as follows: the electronic device obtains the initial feature vector of the first node, inputs the initial feature vector into a multilayer perceptron (MLP), and performs feature extraction on the initial feature vector through the multilayer perceptron to obtain the first feature vector.

[0097] In this implementation, the electronic device determines the product of the initial feature vector and the first weight matrix in the multilayer perceptron to obtain the first feature vector. This process can be expressed by the following formula: h i =MLP(x i , W1), where h i x is used to represent the first feature vector corresponding to the first node i. i W1 is used to represent the initial feature vector corresponding to the first node i, and W1 is used to represent the first weight matrix, where i is an integer greater than 0.

[0098] It's important to note that if the first node possesses attribute information, the electronic device can use this information to characterize its initial feature vector. For example, if the first node is an image with attributes such as color, texture, and resolution, this attribute information can be used to characterize its initial feature vector. If the first node lacks attribute information, a normal distribution can be used to initialize it and obtain the initial feature vector. For instance, if the first node is a user, and user data is sensitive, the electronic device may not be able to access relevant user information. In this case, a normal distribution can be used to initialize the first node and obtain its initial feature vector. This normal distribution method can be set and modified as needed; for example, the normal distribution could be the Kaiming normal distribution.

[0099] The process by which an electronic device obtains multiple weight matrices can be as follows: The electronic device first determines various relationships between multiple first nodes. For any relationship, it determines the similarity between two first nodes with that relationship and determines weight parameters based on this similarity. Since there are generally multiple first nodes with that relationship, the electronic device obtains multiple weight parameters and determines a weight matrix composed of these multiple weight parameters. One relationship corresponds to one weight matrix.

[0100] Step 302: The electronic device integrates multiple weight matrices based on the first quantity of multiple correlation relationships to obtain multiple first weight parameters.

[0101] Each weight matrix corresponds to a first weight parameter, which is used to represent the importance of the association.

[0102] For any weight matrix, the electronic device can first determine the reciprocal of a first quantity, and then integrate multiple weight parameters in the weight matrix into the reciprocal of that first quantity to obtain the first weight parameter. For example, if R represents the first quantity, then the first weight parameter can be expressed as...

[0103] In this embodiment, the first weight parameter obtained in step 302 is an initialized first weight parameter. This first weight parameter cannot accurately represent the importance of the correlation and needs to be optimized. In this embodiment, the electronic device can optimize the first weight parameter through the following steps.

[0104] Step 303: The electronic device acquires the first relationship data.

[0105] The first relational data is used to represent the relationship between two independent variables, namely the first eigenvector and the first weight parameter.

[0106] In this step, the electronic device can obtain the first relationship data through the following steps (1) to (5):

[0107] (1) The electronic device determines the first vector matrix and the Laplace matrix corresponding to various correlation relationships.

[0108] The first vector matrix is ​​composed of the first eigenvectors corresponding to multiple first nodes, and one association relationship corresponds to one Laplace matrix.

[0109] For any association, the process by which the electronic device determines the Laplacian matrix corresponding to the association can be as follows: Based on the graph structure data corresponding to the graph neural network, the electronic device determines the degree matrix and adjacency matrix corresponding to the association, determines the difference between the degree matrix and the adjacency matrix, and obtains the Laplacian matrix corresponding to the association.

[0110] (2) The electronic device uses the first vector matrix as the independent variable to determine the characteristic relation data in the first relation data.

[0111] The electronic device determines a first vector matrix composed of first feature vectors corresponding to multiple first nodes. Since the first feature vectors are continuously optimized through the neural network during the model training process, the first vector matrix changes continuously with the optimization of the first feature vectors.

[0112] The electronic device calculates the difference between the first and third vector matrices to obtain the fourth vector matrix. It then determines the F-norm of the fourth vector matrix to obtain the characteristic term relationship data. This third vector matrix is ​​also composed of the first eigenvectors corresponding to multiple first nodes, but here it is a constant and does not change with the changes in the first eigenvectors.

[0113] This feature relation data can be represented by the following formula: Where Z represents the first vector matrix and H represents the third vector matrix.

[0114] Since both the first and third vector matrices are composed of the first eigenvector, they are identical without optimization of the first eigenvector using a neural network. However, after optimization of the first eigenvector using a neural network, the third vector matrix remains unchanged, while the first vector matrix changes with the change of the first eigenvector.

[0115] (3) The electronic device determines the first parameter item relationship data in the first relationship data based on the first vector matrix and the Laplace matrix corresponding to the various relationships, with the first weight parameter corresponding to the various relationships as the independent variable.

[0116] The electronic device determines the transpose of the first vector matrix to obtain the second vector matrix. For any association relationship, the electronic device determines the product of the first vector matrix, the second vector matrix, and the Laplacian matrix corresponding to the association relationship to obtain the first optimization matrix corresponding to the association relationship. The first optimization matrix is ​​used to optimize the first weight parameter corresponding to the association relationship. The electronic device determines the trace of the first optimization matrix, and determines the sum of the products of the traces of the first optimization matrices corresponding to various association relationships and the first weight parameters corresponding to various association relationships to obtain the first parameter term relationship data.

[0117] The relationship data for the first parameter can be represented by the following formula: Where, μ r Z represents the first weight parameter corresponding to the association r. T Let L represent the second vector matrix. r Let tr(·) represent the Laplacian matrix corresponding to the correlation r, and let tr(·) represent the trace of the first optimization matrix.

[0118] (4) The electronic device uses the first weight parameter corresponding to each of the multiple relationships as the independent variable to determine the second parameter item relationship data in the first relationship data.

[0119] The electronic device determines a weight parameter matrix consisting of multiple first weight parameters, determines the second norm of the weight parameter matrix, and obtains the relational data of the second parameter terms.

[0120] The relationship data for the second parameter can be represented by the following formula: Where μ represents a weight parameter matrix consisting of multiple first weight parameters.

[0121] (5) The electronic device determines the first relation data based on the feature item relation data, the first parameter item relation data, and the second parameter item relation data.

[0122] The electronic device obtains a first weight and a second weight. The first weight is the weight of the first parameter item relationship data, and the second weight is the weight of the second parameter item relationship data. The electronic device determines the product of the first weight and the first parameter item relationship data to obtain the third parameter item relationship data. It determines the product of the second weight and the second parameter item relationship data to obtain the fourth parameter item relationship data. It determines the minimum value of the sum of the third parameter item relationship data, the fourth parameter item relationship data, and the feature item relationship data to obtain the first relationship data.

[0123] This first relational data can be represented by the following formula:

[0124] Here, argmin represents the values ​​of Z and μ that minimize the sum of the third parameter relation data, the fourth parameter relation data, and the feature relation data.

[0125] For this first relational data, its constraints can be expressed by the following formula:

[0126] in, This indicates that the sum of multiple first weight parameters is 1, and the values ​​of all multiple first weight parameters are greater than or equal to 0.

[0127] The magnitudes of the first and second weights can be set and changed as needed, without any specific limitations.

[0128] In this embodiment, the feature term relationship data can be used to encode the original features of nodes, while the first parameter term relationship data and the second parameter term relationship data can utilize the smoothing effect of the Laplacian matrix under multiple association relationships to capture the matching of graph structure data under multiple association relationships and measure the importance of different association relationships. Therefore, it can be seen that the first relationship data can simultaneously learn the feature vectors of nodes and the weight parameters of association relationships.

[0129] Step 304: Based on the first relation data, the electronic device uses the first weight parameter as the independent variable and keeps the first feature vectors corresponding to the multiple first nodes unchanged to obtain the second relation data.

[0130] As shown in step 303, the first relational data contains two independent variables: the first feature vector and the first weight parameter. Based on the first relational data, the electronic device first uses the first weight parameter as the independent variable, keeping the first feature vectors corresponding to the multiple first nodes unchanged, to obtain the second relational data.

[0131] Since the third vector matrix is ​​the same as the first vector matrix when the first feature vector is not optimized through a neural network, the second relation data can be represented by the following formula while keeping the first feature vectors corresponding to multiple first nodes unchanged:

[0132] The constraints for this second relational data can still be expressed by the following formula:

[0133]

[0134] Step 305: Based on the second relation data, the electronic device iteratively optimizes multiple first weight parameters through the current layer neural network until the multiple first weight parameters meet the convergence condition, thereby obtaining multiple second weight parameters.

[0135] The process of iteratively optimizing a first weight parameter to obtain a second weight parameter in an electronic device can be achieved through the following steps (1) to (4):

[0136] (1) The electronic device optimizes the second relation data to obtain the third relation data.

[0137] The third relation data is used to represent the relationship optimized based on the weight ratio coefficient, the first optimization matrix, the first optimization parameter, and the first quantity. The first weight parameter is optimized as the number of iterations increases. The weight ratio coefficient is determined based on the first weight and the second weight. The first weight is the weight of the first parameter item relation data in the first relation data, and the second weight is the weight of the second parameter item relation data in the first relation data. The first optimization matrix and the first optimization parameter are both used to optimize the first weight parameter corresponding to the association relationship.

[0138] In this step, the electronic device can optimize the second relation data using the mirror entropic descent algorithm to obtain the third relation data. This third relation data can be represented by the following formula: in, This represents the weight parameter of the first weight parameter corresponding to the association relationship r at the iteration optimization number t+1. The first weight parameter corresponding to the association r is the weight parameter when the number of iterations is t, where t ≥ 0 and is an integer. Where Φ represents the first optimization parameter, which can be expressed by the following formula: s = {s1, s2, ... s} r ...s R}, s r Let s represent the trace of the first optimization matrix. r =tr(Z) T Lr Z),

[0139] According to the expression of the third relation data, the first weight parameter changes with the increase of the number of iterations.

[0140] (2) The electronic device determines the first optimization matrix corresponding to the various correlation relationships based on the first vector matrix and the Laplace matrix corresponding to the various correlation relationships.

[0141] The process of determining the first optimization matrix for electronic devices has been described in step 303 above, and will not be repeated here.

[0142] (3) The electronic device determines the first optimization parameters based on the weight ratio coefficient and the first optimization matrix corresponding to various correlation relationships.

[0143] In this step, the electronic device determines the traces of the first optimization matrix corresponding to various correlation relationships, obtains multiple traces, determines the norm of the trace matrix composed of multiple traces, and determines the sum of the norm and the weight scaling factor as the first optimization parameter.

[0144] These multiple traces can be represented as s1, s2...s respectively. r ...s R A trace matrix consisting of multiple traces can be represented as s = {s1, s2, ... s}. r ...s R The norm of the trace matrix is ​​a 1-norm, which can be represented as ||s||1.

[0145] The electronic device determines a weighting ratio coefficient based on a first weight and a second weight. This weighting ratio coefficient can be expressed as: The electronic device determines the sum of the weight ratio coefficient and the norm to obtain the first optimization parameter Φ, which is used to optimize the first weight parameter.

[0146] (4) The electronic device substitutes the first quantity, weight ratio coefficient, first optimization parameter and the first optimization matrix corresponding to the relationship into the third relationship data respectively. Based on the number of iterations, the first weight parameter is iteratively optimized through the current layer neural network until the first weight parameter meets the convergence condition, and the second weight parameter is obtained.

[0147] This step can be achieved through the following steps (4-1) to (4-3), including:

[0148] (4-1) The electronic device substitutes the first quantity, weight ratio coefficient, first optimization parameter, first optimization matrix corresponding to the relationship and the current iteration optimization number into the third relationship data, and optimizes the current first weight parameter through the current layer neural network to obtain the optimized first weight parameter.

[0149] The electronic device, based on the expression of the third relation data in step (1), will convert the first quantity R and the weight ratio coefficient. First optimization parameter Φ, first optimization matrix Z T L r Z, the current iteration number t, and the current first weight parameter Substituting these values ​​into the third relation data, we obtain the optimized first weight parameters.

[0150] (4-2) The electronic device increments the current iteration optimization number by 1 to obtain the updated iteration optimization number.

[0151] For example, if the current iteration number is t, then the updated iteration number will be t+1.

[0152] (4-3) The electronic device takes the optimized first weight parameter as the current first weight parameter and the updated iteration optimization number as the current iteration optimization number, and executes step (4-1) until the optimized first weight parameter meets the convergence condition, and obtains the second weight parameter.

[0153] In this embodiment, after each optimization of the first weight parameter, the electronic device can determine whether the optimized first weight parameter meets the convergence condition. The convergence condition can be that the difference between the optimized first weight parameter and the previously optimized first weight parameter is less than a preset threshold, or that the number of iterations reaches the maximum number of iterations.

[0154] If the difference is greater than the preset threshold, or the number of iterations has not reached the maximum number of iterations, it indicates that the convergence condition is not met, and optimization continues. If the difference is less than or equal to the preset threshold, or the number of iterations has reached the maximum number of iterations, it indicates that the optimized first weight parameter meets the convergence condition, and the electronic device uses the optimized first weight parameter as the second weight parameter.

[0155] It should be noted that for each type of association, the electronic device optimizes the first weight parameter through the above steps (1) to (4) to obtain the second weight parameter corresponding to each type of association.

[0156] Step 306: Based on the first relation data, the electronic device uses the first feature vector as the independent variable and keeps multiple second weight parameters unchanged to obtain the fourth relation data.

[0157] The electronic device uses the first feature vector as the independent variable, while keeping several second weight parameters unchanged, to obtain fourth relational data, which can be represented by the following formula:

[0158] Where, μ′ r The second weight parameter is used to represent the association relationship r.

[0159] Step 307: Based on the fourth relation data, the electronic device optimizes the first feature vectors corresponding to multiple first nodes through the current layer neural network to obtain the second feature vectors corresponding to multiple first nodes.

[0160] For any first node, the electronic device can optimize the first feature vector of the first node through the following steps (1) to (3) to obtain the second feature vector corresponding to the first node:

[0161] (1) The electronic device optimizes the fourth relation data to obtain the fifth relation data.

[0162] The fifth relation data is used to represent the relationship based on the first weight, the fourth feature vector corresponding to at least one second node, and the second weight parameter corresponding to at least one association relationship. The first feature vector is optimized as the number of neural network layers increases. At least one second node is a node among multiple first nodes that has at least one association relationship with the first node that serves as the central node.

[0163] The electronic device can take the partial derivative of the first vector matrix in the fourth relational data, set the partial derivative to 0, and obtain the optimized expression of the first vector matrix: It can be seen that this expression requires inverting the matrix, but matrix inversion is very costly. Therefore, in this embodiment, the optimization process of the first vector matrix can be represented by the following formula:

[0164]

[0165] However, this formula is for the optimization process of the first vector matrix. For the first eigenvector corresponding to a first node, this formula can be transformed to obtain the fifth relation data. This fifth relation data can be used to represent the optimization process of the first eigenvector. This fifth relation data can be represented by the following formula:

[0166]

[0167] Where k represents the layer number of the neural network, h represents the weight parameter corresponding to the association r in the (k-1)th layer of the neural network. iThis represents the first feature vector corresponding to the first node i. Let $j$ represent the fourth feature vector of the second node $j$ in the (k-1)th layer of the neural network. The second node $j$ is a node that has a correlation $r$ with the first node $i$. Let N(i) represent the feature vector corresponding to the first node i in the k-th layer of the neural network. Let N(i) represent the neighboring nodes of the first node i. These neighboring nodes can be nodes directly connected to the first node i or nodes indirectly connected to the first node i. i, j and k are all integers greater than 0.

[0168] (2) The electronic device outputs the fifth feature vector corresponding to the first node under at least one association relationship through the current layer neural network based on the fourth feature vector corresponding to at least one second node and at least one second weight parameter corresponding to at least one association relationship.

[0169] The fifth feature vector is used to represent the node information of at least one second node. The fourth feature vector and at least one second weight parameter are both output by the previous layer of the neural network of the current layer.

[0170] In the embodiments of this application, there are multiple relationships between multiple first nodes. However, for any one first node, there may be multiple relationships between the first node and other first nodes, or there may be some of the multiple relationships. However, it is certain that there must be at least one relationship between the first node and other first nodes, and one relationship corresponds to at least one other first node. For ease of distinction, the other first nodes are referred to as second nodes.

[0171] For each of the at least one association relationships, the electronic device determines the fourth feature vector of at least one second node corresponding to the association relationship in the previous layer of the neural network, determines the product of the second weight parameter corresponding to the association relationship and the fourth feature vector corresponding to the at least one second node, obtains the eighth feature vector corresponding to the association relationship, and determines the sum of the eighth feature vectors corresponding to each of the at least one association relationships, obtains the fifth feature vector.

[0172] (3) The electronic device substitutes the first weight, the first feature vector corresponding to the first node and the fifth feature vector into the fifth relation data respectively, and outputs the second feature vector corresponding to the first node through the current layer neural network.

[0173] The electronic device first determines the third and fourth weights based on the first weight. The third weight is used to represent the importance of the first feature vector in the second feature vector, and the fourth weight is used to represent the importance of the fifth feature vector in the second feature vector. The third weight, the fourth weight, the first feature vector, and the fifth feature vector are substituted into the fifth relation data respectively, and the second feature vector is output through the current layer neural network.

[0174] In this implementation, the process of the electronic device determining the third weight and the fourth weight can be as follows: the electronic device determines the sum of the first preset value and the first weight, determines the ratio of the first preset value to the sum to obtain the third weight, and determines the ratio of the first weight to the sum to obtain the fourth weight.

[0175] This first preset value can be set and changed as needed. For example, if the first preset value is 1, then the third weight can be represented as: The fourth weight can be represented as:

[0176] The electronic device substitutes the third weight, fourth weight, first feature vector, and fifth feature vector into the fifth relation data, and determines the product of the third weight and the first feature vector and the product of the fourth weight and the fifth feature vector through the current layer neural network. Then, it determines the sum of these two products to obtain the second feature vector.

[0177] For each first node, the electronic device is optimized through the above steps (1) to (3) to obtain the second feature vectors corresponding to multiple first nodes respectively.

[0178] It should be noted that as the number of layers in a graph neural network increases, the feature vectors of nodes converge to a steady state. The graph neural network gradually forgets the initial features of the nodes, resulting in oversmoothing and poor node classification performance. However, in this embodiment, the first relation data includes a first feature vector, which is extracted from the initial feature vector. When optimizing the feature vector, optimization is performed based on this first feature vector, as can also be seen from the fifth relation data. Therefore, the transmission mechanism of this multi-layer neural network is equivalent to a personalized PageRank (random walk) for multiple relations, which can return to the node itself with a certain restart probability, maintaining the original node information, alleviating the oversmoothing phenomenon, and thus improving the node classification performance. Therefore, the method provided in this embodiment can not only alleviate the problem of overparameterization but also the oversmoothing phenomenon, improving both model training efficiency and model accuracy.

[0179] Step 308: Based on the second feature vectors corresponding to the multiple first nodes, the electronic device iteratively optimizes the multiple second weight parameters through the next layer of the neural network to obtain multiple third weight parameters, until the third feature vectors corresponding to the multiple first nodes are output through the last layer of the neural network.

[0180] The electronic device optimizes the first feature vectors corresponding to multiple first nodes through the current layer neural network to obtain the second feature vectors corresponding to multiple first nodes. Then, the first feature vectors and first weight parameters in the first relation data are updated to the second feature vectors and second weight parameters, respectively. Using the second weight parameters as independent variables, the second feature vectors corresponding to multiple first nodes are kept unchanged to obtain the sixth relation data. Based on the sixth relation data, the next layer neural network iteratively optimizes the multiple second weight parameters until the multiple second weight parameters meet the convergence condition, obtaining multiple third weight parameters. This process is similar to steps 304 and 305 above and will not be repeated here.

[0181] After obtaining multiple third weight parameters, the electronic device uses the second feature vector as the independent variable, keeping the multiple third weight parameters unchanged, to obtain the seventh relation data. Based on the seventh relation data, the second feature vectors corresponding to the multiple first nodes are optimized through the next layer of the neural network to obtain the ninth feature vectors corresponding to the multiple first nodes. This process is similar to steps 306 to 307 above, and will not be repeated here.

[0182] This shows that the iterative optimization of weight parameters and the optimization of feature vectors correspond to a layer of a neural network in a multi-graph neural network. That is, the weight parameters are iteratively optimized first through a layer of neural network, and then the feature vectors are optimized based on the iteratively optimized weight parameters.

[0183] By executing steps 304 to 308 above, the electronic device continuously iterates and optimizes the weight parameters and feature vectors through layers of neural networks until the last layer of neural networks optimizes the feature vectors, thereby obtaining the third feature vectors corresponding to multiple first nodes.

[0184] Step 309: The electronic device trains the model based on the third feature vectors corresponding to the multiple first nodes to obtain the target neural network model.

[0185] In the embodiments of this application, the electronic device can train a neural network model to complete tasks such as classification, recommendation, prediction, and search.

[0186] This explanation uses the example of training a neural network model to complete a classification task. The training process can be as follows: the electronic device determines the predicted labels corresponding to multiple first nodes based on the third feature vectors corresponding to each first node; and trains the model based on the label loss value between the predicted labels and the actual labels corresponding to the multiple first nodes to obtain the target neural network model.

[0187] In this implementation, for any first node, the electronic device can input the third feature vector corresponding to that first node into a multilayer perceptron, using the multilayer perceptron as a classifier to determine the product of the third feature vector and the second weight matrix of the multilayer perceptron, thus obtaining the predicted label of the first node. This process can be expressed by the following formula: in, This is used to represent the third feature vector obtained after the first node i is optimized by a k-layer neural network. W1 represents the predicted label of the first node i, and W2 represents the second weight matrix. Both the first and second weight matrices are randomly initialized.

[0188] For any first node, the electronic device can determine the label loss value between the predicted label and the actual label corresponding to the first node based on the loss function. If the label loss value is greater than the preset loss value, the first weight matrix and the second weight matrix are adjusted by backpropagation until the label loss value is less than or equal to the preset loss value, or the model training times reach the maximum number of training times. The electronic device then determines that the model has converged and obtains the target neural network model.

[0189] The loss function can be expressed by the following formula: Where l(·) represents the loss function, D(·) represents the discrimination function (cross-entropy loss), and y i This represents the actual label of the first node i.

[0190] In this embodiment, a decoupling approach is used to design a multi-relationship graph neural network. See [link to relevant documentation]. Figure 4 This method separates the feature extraction and feature optimization processes. A multilayer perceptron is used to extract features from the initial feature vectors of the nodes. These extracted feature vectors are then input into a multilayer neural network. The network iteratively optimizes the weight parameters and feature vectors continuously until the final layer outputs a third feature vector. If the trained neural network model successfully completes the classification task, this third feature vector is then input into the classifier for classification. This demonstrates that the method iteratively optimizes the weight parameters during the forward propagation process.

[0191] This application provides a method for training a neural network model. When training the neural network model, this method integrates multiple weight matrices based on the number of various correlations to obtain multiple first weight parameters. Since one correlation corresponds to one weight matrix, and one weight matrix corresponds to one first weight parameter, each correlation corresponds to one first weight parameter. Therefore, this method integrates a weight matrix into a single weight parameter, which significantly reduces the number of parameters, alleviates the problem of overparameterization, and thus improves model training efficiency.

[0192] Because this embodiment integrates the weight matrix corresponding to a relationship into a learnable weight parameter, the number of parameters is greatly reduced. Therefore, the target neural network model is a lightweight relational neural network model. Furthermore, neural network models with fewer parameters have lower storage overhead and can significantly alleviate overfitting problems in practical applications, making them highly suitable for large-scale graph training in real-world scenarios. Additionally, because this target neural network model can mitigate oversmoothing, feature vectors can propagate through multiple layers, effectively capturing information from distant nodes in actual products and businesses. Moreover, this embodiment explores how to derive multi-relationship graph neural networks from the perspective of an optimization framework. Compared to the heuristic design and black-box structure of traditional multi-relationship graph neural networks, the multi-relationship graph neural network provided in this embodiment has a clearer optimization objective and is more reliable and reasonable.

[0193] The following explanation uses a target neural network model to complete a classification task as an example. For ease of distinction, the electronic device used to train the neural network model is referred to as the first electronic device, and the electronic device in this embodiment is referred to as the second electronic device. The first electronic device and the second electronic device can be the same electronic device or different electronic devices; there is no specific limitation on this.

[0194] Figure 5 This is a flowchart illustrating the classification of target nodes using a target neural network model, as provided in an embodiment of this application. It is executed by a second electronic device. See [link to relevant documentation]. Figure 5 The method includes:

[0195] Step 501: The second electronic device acquires the sixth feature vector of the target node to be classified.

[0196] The sixth feature vector is the initial feature vector of the target node. The target node can be a product, image, video, etc., without specific limitations.

[0197] The process by which the second electronic device obtains the sixth feature vector of the target node is similar to the process of obtaining the initial feature vector of the first node in step 301, and will not be described again here.

[0198] Step 502: The second electronic device extracts features from the sixth feature vector of the target node to obtain the seventh feature vector of the target node.

[0199] The process by which the second electronic device extracts features from the sixth feature vector of the target node to obtain the seventh feature vector of the target node is similar to the process by which the second electronic device extracts features from the initial feature vector of the first node to obtain the first feature vector of the first node, and will not be described again here.

[0200] Step 503: The second electronic device determines the predicted label of the target node based on multiple target weight parameters and the seventh feature vector corresponding to the target neural network model.

[0201] The target weight parameters are the weight parameters corresponding to various relationships when the target neural network model has been trained.

[0202] The second electronic device inputs the seventh feature vector into the target neural network model. Based on multiple target weight parameters, it realizes message passing under multiple relationship graphs through a multi-layer neural network to obtain the target feature vector of the target node. The target feature vector is then input into the classifier to obtain the predicted label of the target node.

[0203] Step 504: The second electronic device classifies the target node based on the predicted label of the target node.

[0204] The second electronic device classifies the target node based on its predicted label and determines the category to which the target node belongs.

[0205] For example, if the target node is a product, the product category could be food, clothing, etc. If the target node is a video, the video category could be food videos, music videos, etc.

[0206] In this embodiment of the application, target nodes are classified by a target neural network model. Since a weight matrix is ​​integrated into a single weight parameter when training the target neural network model, the number of parameters can be greatly reduced, alleviating the problem of over-parameterization. Therefore, when classifying target nodes by this target neural network model, the classification speed is faster and the efficiency is higher.

[0207] Figure 6 This application provides a training device for a neural network model, see [link to relevant documentation]. Figure 6 The device includes:

[0208] The first acquisition module 601 is used to acquire the first feature vectors corresponding to multiple first nodes and the weight matrices corresponding to various association relationships between multiple first nodes, with one association relationship corresponding to one weight matrix.

[0209] The integration module 602 is used to integrate multiple weight matrices based on the first number of multiple association relationships to obtain multiple first weight parameters. One weight matrix corresponds to one first weight parameter, and the first weight parameter is used to represent the importance of the association relationship.

[0210] Optimization module 603 is used to iteratively optimize multiple first weight parameters based on the first feature vectors corresponding to multiple first nodes until the multiple first weight parameters meet the convergence condition, and obtain multiple second weight parameters.

[0211] The training module 604 is used to train the model based on multiple second weight parameters and the first feature vectors corresponding to multiple first nodes, so as to obtain the target neural network model.

[0212] In one possible implementation, the optimization module 603 is used to obtain first relation data, which represents the relationship between two independent variables, namely a first feature vector and a first weight parameter; based on the first relation data, with the first weight parameter as the independent variable, the first feature vectors corresponding to multiple first nodes are kept unchanged to obtain second relation data; based on the second relation data, the multiple first weight parameters are iteratively optimized through the current layer neural network until the multiple first weight parameters meet the convergence condition to obtain multiple second weight parameters.

[0213] In another possible implementation, optimization module 603 is used to determine the first vector matrix and the Laplace matrices corresponding to various association relationships. The first vector matrix is ​​composed of the first eigenvectors corresponding to multiple first nodes, and one association relationship corresponds to one Laplace matrix. Using the first vector matrix as the independent variable, feature term relationship data in the first relationship data is determined. Based on the first vector matrix and the Laplace matrices corresponding to various association relationships, and using the first weight parameters corresponding to various association relationships as independent variables, the first parameter term relationship data in the first relationship data is determined, and one association relationship corresponds to one Laplace matrix. Using the first weight parameters corresponding to various association relationships as independent variables, the second parameter term relationship data in the first relationship data is determined. Based on the feature term relationship data, the first parameter term relationship data, and the second parameter term relationship data, the first relationship data is determined.

[0214] In another possible implementation, optimization module 603 is used to optimize the second relation data to obtain third relation data. The third relation data represents a relationship optimized based on weight ratio coefficients, a first optimization matrix, a first optimization parameter, and a first quantity. The first weight parameter is optimized as the number of iterations increases. The weight ratio coefficient is determined based on the first weight and the second weight. The first weight is the weight of the first parameter item relation data in the first relation data, and the second weight is the weight of the second parameter item relation data in the first relation data. The first optimization matrix and the first optimization parameter are both used to optimize the first weight parameter corresponding to the association relationship. Based on the first vector matrix and the Laplacian matrix corresponding to each of the multiple association relationships, the first optimization matrix corresponding to each of the multiple association relationships is determined, with one first optimization matrix corresponding to each association relationship. Based on the weight ratio coefficients and the first optimization matrices corresponding to each of the multiple association relationships, the first optimization parameter is determined. The first quantity, weight ratio coefficients, first optimization parameter, and the first optimization matrix corresponding to the association relationship are substituted into the third relation data. Based on the number of iterations, the first weight parameter is iteratively optimized through the current layer neural network until the first weight parameter meets the convergence condition, thus obtaining the second weight parameter.

[0215] In another possible implementation, the optimization module 603 is used to substitute the first quantity, weight ratio coefficient, first optimization parameter, first optimization matrix corresponding to the association relationship, and the current iteration optimization number into the third relationship data, and optimize the current first weight parameter through the current layer neural network to obtain the optimized first weight parameter; increment the current iteration optimization number by 1 to obtain the updated iteration optimization number; use the optimized first weight parameter as the current first weight parameter, and use the updated iteration optimization number as the current iteration optimization number, and execute the steps of substituting the first quantity, weight ratio coefficient, first optimization parameter, first optimization matrix corresponding to the association relationship, and the current iteration optimization number into the third relationship data, and optimizing the current first weight parameter through the current layer neural network to obtain the optimized first weight parameter; until the optimized first weight parameter satisfies the convergence condition, the second weight parameter is obtained.

[0216] In another possible implementation, optimization module 603 is used to determine the traces of the first optimization matrix corresponding to various correlation relationships, thereby obtaining multiple traces; determine the norm of the trace matrix composed of multiple traces; and determine the sum of the norm and the weight scaling factor as the first optimization parameter.

[0217] In another possible implementation, optimization module 603 is used to determine the transpose matrix corresponding to the first vector matrix to obtain the second vector matrix; and to determine the product of the second vector matrix, the first vector matrix, and the Laplace matrix to obtain the first optimized matrix.

[0218] In another possible implementation, the training module 604 is used to obtain fourth relation data based on the first relation data, using the first feature vector as the independent variable and keeping multiple second weight parameters unchanged. The first relation data is used to represent the relationship between two independent variables, namely the first feature vector and the first weight parameter. Based on the fourth relation data, the first feature vectors corresponding to multiple first nodes are optimized through the current neural network to obtain second feature vectors corresponding to multiple first nodes. Based on the second feature vectors corresponding to multiple first nodes, the multiple second weight parameters are iteratively optimized through the next layer of the neural network to obtain multiple third weight parameters, until the third feature vectors corresponding to multiple first nodes are output through the last layer of the neural network. Based on the third feature vectors corresponding to multiple first nodes, the model is trained to obtain the target neural network model.

[0219] In another possible implementation, training module 604 is used to optimize the fourth relation data to obtain fifth relation data. The fifth relation data represents a relationship based on a first weight, a fourth feature vector corresponding to at least one second node, and a second weight parameter corresponding to at least one association relationship. The first feature vector is optimized as the number of neural network layers increases. The first weight is the weight of the first parameter item relation data in the first relation data. The at least one second node is a node among multiple first nodes that has at least one association relationship with the first node that is the center node. Based on the fourth feature vector corresponding to at least one second node and at least one second weight parameter corresponding to at least one association relationship, the fifth feature vector corresponding to the first node under at least one association relationship is output through the current layer neural network. The fifth feature vector is used to represent the node information of at least one second node. The fourth feature vector and at least one second weight parameter are both output by the previous layer neural network of the current layer neural network. The first weight, the first feature vector corresponding to the first node, and the fifth feature vector are substituted into the fifth relation data respectively, and the second feature vector corresponding to the first node is output through the current layer neural network.

[0220] In another possible implementation, training module 604 is used to determine third and fourth weights based on the first weight. The third weight is used to represent the importance of the first feature vector in the second feature vector, and the fourth weight is used to represent the importance of the fifth feature vector in the second feature vector. The third weight, fourth weight, first feature vector, and fifth feature vector are substituted into the fifth relation data respectively, and the second feature vector is output through the current layer neural network.

[0221] In another possible implementation, the training module 604 is used to determine the predicted labels corresponding to the multiple first nodes based on the third feature vectors corresponding to the multiple first nodes respectively; and to train the model based on the label loss value between the predicted labels corresponding to the multiple first nodes and the actual labels to obtain the target neural network model.

[0222] In another possible implementation, the integration module 602 is used to integrate multiple weight parameters in the weight matrix into a first number of reciprocals to obtain the first weight parameter.

[0223] In another possible implementation, the device also includes:

[0224] The second acquisition module is used to acquire the sixth feature vector of the target node to be classified;

[0225] The feature extraction module is used to extract features from the sixth feature vector of the target node to obtain the seventh feature vector of the target node.

[0226] The determination module is used to determine the predicted label of the target node based on multiple target weight parameters and the seventh feature vector corresponding to the target neural network model.

[0227] The classification module is used to classify target nodes based on their predicted labels.

[0228] This application provides a training device for a neural network model. When training the neural network model, this device integrates multiple weight matrices based on the number of various correlations to obtain multiple first weight parameters. Since one correlation corresponds to one weight matrix, and one weight matrix corresponds to one first weight parameter, each correlation corresponds to one first weight parameter. Therefore, this device integrates a weight matrix into one weight parameter, which greatly reduces the number of parameters, alleviates the problem of overparameterization, and thus improves model training efficiency.

[0229] It should be noted that the neural network model training device provided in the above embodiments is only illustrated by the division of the above functional modules when training the neural network model. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the electronic device can be divided into different functional modules to complete all or part of the functions described above. In addition, the neural network model training device and the neural network model training method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0230] If the electronic device is provided as a terminal, please refer to... Figure 7 , Figure 7This illustration shows a structural block diagram of a terminal 700 provided in an exemplary embodiment of this application. The terminal 700 may be a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. The terminal 700 may also be referred to as a user device, portable terminal, laptop terminal, desktop terminal, or other names.

[0231] Typically, terminal 700 includes a processor 701 and a memory 702.

[0232] Processor 701 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 701 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 701 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 701 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 701 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0233] The memory 702 may include one or more computer-readable storage media, which may be non-transitory. The memory 702 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 702 are used to store at least one program code instruction, which is executed by the processor 701 to implement the training method of the neural network model provided in the method embodiments of this application.

[0234] In some embodiments, the terminal 700 may also optionally include a peripheral device interface 703 and at least one peripheral device. The processor 701, memory 702, and peripheral device interface 703 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 703 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 704, a touch display screen 705, a camera 706, an audio circuit 707, a positioning component 708, and a power supply 709.

[0235] Peripheral device interface 703 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 701 and memory 702. In some embodiments, processor 701, memory 702 and peripheral device interface 703 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 701, memory 702 and peripheral device interface 703 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0236] The radio frequency (RF) circuit 704 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 704 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 704 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 704 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 704 can communicate with other terminals through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: metropolitan area networks (MANs), various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks (WLANs), and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 704 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0237] Display screen 705 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 705 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 701 for processing. In this case, display screen 705 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 705, which serves as the front panel of terminal 700; in other embodiments, there may be at least two display screens 705, respectively disposed on different surfaces of terminal 700 or in a folded design; in still other embodiments, display screen 705 may be a flexible display screen, disposed on a curved or folded surface of terminal 700. Furthermore, display screen 705 may be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. Display screen 705 may be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).

[0238] The camera assembly 706 is used to acquire images or videos. Optionally, the camera assembly 706 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 706 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0239] The audio circuit 707 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 701 for processing, or input to the radio frequency circuit 704 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each located at a different part of the terminal 700. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert the electrical signals from the processor 701 or the radio frequency circuit 704 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 707 may also include a headphone jack.

[0240] The positioning component 708 is used to determine the current geographic location of the terminal 700 in order to enable navigation or LBS (Location Based Service). The positioning component 708 can be a positioning component based on the US GPS (Global Positioning System), China's BeiDou system, Russia's Granas system, or the EU's Galileo system.

[0241] The power supply 709 is used to power the various components in the terminal 700. The power supply 709 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When the power supply 709 includes a rechargeable battery, the rechargeable battery can support wired or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0242] In some embodiments, the terminal 700 further includes one or more sensors 710. The one or more sensors 710 include, but are not limited to: an acceleration sensor 711, a gyroscope sensor 712, a pressure sensor 713, an optical sensor 715, and a proximity sensor 716.

[0243] Accelerometer 711 can detect the magnitude of acceleration along the three coordinate axes of a coordinate system established by terminal 700. For example, accelerometer 711 can be used to detect the components of gravitational acceleration along the three coordinate axes. Processor 701 can control touchscreen 705 to display the user interface in landscape or portrait view based on the gravitational acceleration signal acquired by accelerometer 711. Accelerometer 711 can also be used for games or for acquiring user motion data.

[0244] The gyroscope sensor 712 can detect the orientation and rotation angle of the terminal 700. The gyroscope sensor 712, in conjunction with the accelerometer sensor 711, can collect 3D motion data from the user on the terminal 700. Based on the data collected by the gyroscope sensor 712, the processor 701 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0245] The pressure sensor 713 can be disposed on the side bezel of the terminal 700 and / or on the lower layer of the touch display screen 705. When the pressure sensor 713 is disposed on the side bezel of the terminal 700, it can detect the user's grip signal on the terminal 700, and the processor 701 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 713. When the pressure sensor 713 is disposed on the lower layer of the touch display screen 705, the processor 701 can control the operable controls on the UI interface based on the user's pressure operation on the touch display screen 705. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0246] An optical sensor 715 is used to collect ambient light intensity. In one embodiment, the processor 701 can control the display brightness of the touch screen 705 based on the ambient light intensity collected by the optical sensor 715. Specifically, when the ambient light intensity is high, the display brightness of the touch screen 705 is increased; when the ambient light intensity is low, the display brightness of the touch screen 705 is decreased. In another embodiment, the processor 701 can also dynamically adjust the shooting parameters of the camera assembly 706 based on the ambient light intensity collected by the optical sensor 715.

[0247] The proximity sensor 716, also known as a distance sensor, is typically located on the front panel of the terminal 700. The proximity sensor 716 is used to detect the distance between the user and the front of the terminal 700. In one embodiment, when the proximity sensor 716 detects that the distance between the user and the front of the terminal 700 is gradually decreasing, the processor 701 controls the touchscreen display 705 to switch from a screen-on state to a screen-off state; when the proximity sensor 716 detects that the distance between the user and the front of the terminal 700 is gradually increasing, the processor 701 controls the touchscreen display 705 to switch from a screen-off state to a screen-on state.

[0248] Those skilled in the art will understand that Figure 7 The structure shown does not constitute a limitation on terminal 700, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0249] If the electronic device is provided as a server, see [link to relevant documentation]. Figure 8 , Figure 8This is a schematic diagram of a server structure provided in an embodiment of this application. The server 800 can vary significantly due to different configurations or performance. It may include a central processing unit (CPU) 801 and a memory 802. The memory 802 stores at least one line of program code, which is loaded and executed by the processor 801 to implement the methods provided in the various method embodiments described above. Of course, the server 800 may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server 800 may also include other components for implementing device functions, which will not be elaborated upon here.

[0250] If the electronic device is provided as a terminal and a server, the block diagrams of the terminal and server can be found separately. Figure 7 and Figure 8 .

[0251] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one piece of program code that is loaded and executed by a processor to implement the training method of the neural network model in the above embodiments.

[0252] In an exemplary embodiment, a computer program product is also provided, which stores at least one piece of program code, which is loaded and executed by a processor to implement the training method of the neural network model in the above embodiments.

[0253] In some embodiments, the computer program involved in the present application embodiments may be deployed and executed on a computer device, or executed on multiple computer devices located in one location, or executed on multiple computer devices distributed in multiple locations and interconnected through a communication network. Multiple computer devices distributed in multiple locations and interconnected through a communication network may constitute a blockchain system.

[0254] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0255] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for training a neural network model, characterized in that, The method includes: Obtain the first feature vectors corresponding to multiple first nodes and the weight matrices corresponding to various association relationships between the multiple first nodes, with one association relationship corresponding to one weight matrix, and the first node being an image; Based on the first number of the various associations, the multiple weight matrices are integrated to obtain multiple first weight parameters. Each weight matrix corresponds to one first weight parameter, and the first weight parameter is used to represent the importance of the association. Based on the first feature vectors corresponding to the plurality of first nodes, the plurality of first weight parameters are iteratively optimized until the plurality of first weight parameters meet the convergence condition, thereby obtaining a plurality of second weight parameters. Based on the plurality of second weight parameters and the first feature vectors corresponding to the plurality of first nodes respectively, the model is trained to obtain the target neural network model; The process of integrating the weight matrix based on the first quantity of the various associations to obtain the first weight parameter includes: The multiple weight parameters in the weight matrix are integrated into the reciprocal of the first number to obtain the first weight parameter; The process involves iteratively optimizing the multiple first weight parameters based on the first feature vectors corresponding to the multiple first nodes until the multiple first weight parameters satisfy the convergence condition, thereby obtaining multiple second weight parameters, including: Obtain first relation data, which is used to represent the relationship between two independent variables, namely a first feature vector and a first weight parameter; Based on the first relational data, using the first weight parameter as the independent variable, and keeping the first feature vectors corresponding to the plurality of first nodes unchanged, the second relational data is obtained. Based on the second relational data, the plurality of first weight parameters are iteratively optimized through the current layer neural network until the plurality of first weight parameters meet the convergence condition, thereby obtaining the plurality of second weight parameters.

2. The method according to claim 1, characterized in that, The acquisition of the first relationship data includes: Determine the first vector matrix and the Laplacian matrix corresponding to each of the multiple association relationships. The first vector matrix is ​​composed of the first eigenvectors corresponding to the multiple first nodes, and each association relationship corresponds to one Laplacian matrix. Using the first vector matrix as the independent variable, determine the feature term relation data in the first relation data; Based on the first vector matrix and the Laplace matrices corresponding to the various association relationships, the first parameter item relationship data in the first relationship data is determined using the first weight parameters corresponding to the various association relationships as independent variables, and one association relationship corresponds to one Laplace matrix; Using the first weight parameters corresponding to the various association relationships as independent variables, the second parameter item relationship data in the first relationship data is determined; The first relationship data is determined based on the feature item relationship data, the first parameter item relationship data, and the second parameter item relationship data.

3. The method according to claim 2, characterized in that, The process of iteratively optimizing the first weight parameter based on the second relation data through the current layer neural network until the first weight parameter satisfies the convergence condition to obtain the second weight parameter includes: The second relation data is optimized to obtain third relation data. The third relation data is used to represent the relationship optimized based on the weight ratio coefficient, the first optimization matrix, the first optimization parameter, and the first quantity. The first weight parameter is optimized as the number of iterations increases. The weight ratio coefficient is determined based on the first weight and the second weight. The first weight is the weight of the first parameter item relation data in the first relation data, and the second weight is the weight of the second parameter item relation data in the first relation data. The first optimization matrix and the first optimization parameter are both used to optimize the first weight parameter corresponding to the association relationship. Based on the first vector matrix and the Laplacian matrix corresponding to the various association relationships, the first optimization matrix corresponding to the various association relationships is determined, with one first optimization matrix corresponding to one association relationship. Based on the weight ratio coefficients and the first optimization matrices corresponding to the various correlation relationships, the first optimization parameters are determined. Substitute the first quantity, the weight ratio coefficient, the first optimization parameter, and the first optimization matrix corresponding to the association relationship into the third relationship data. Based on the number of iterations, iteratively optimize the first weight parameter through the current layer neural network until the first weight parameter meets the convergence condition to obtain the second weight parameter.

4. The method according to claim 3, characterized in that, The step of substituting the first quantity, the weight ratio coefficient, the first optimization parameter, and the first optimization matrix corresponding to the association relationship into the third relationship data, and iteratively optimizing the first weight parameter through the current layer neural network based on the number of iterations until the first weight parameter satisfies the convergence condition to obtain the second weight parameter includes: Substitute the first quantity, the weight ratio coefficient, the first optimization parameter, the first optimization matrix corresponding to the association relationship, and the current iteration optimization number into the third relationship data, and optimize the current first weight parameter through the current layer neural network to obtain the optimized first weight parameter; Increment the current iteration count by 1 to obtain the updated iteration count; The optimized first weight parameter is used as the current first weight parameter, and the updated iteration optimization number is used as the current iteration optimization number. The steps of substituting the first quantity, the weight ratio coefficient, the first optimization parameter, the first optimization matrix corresponding to the association relationship, and the current iteration optimization number into the third relationship data, and optimizing the current first weight parameter through the current layer neural network to obtain the optimized first weight parameter are performed. The second weight parameter is obtained when the optimized first weight parameter meets the convergence condition.

5. The method according to claim 3, characterized in that, The determination of the first optimization parameters based on the weight ratio coefficients and the first optimization matrices corresponding to the various correlation relationships includes: Determine the trace number of the first optimization matrix corresponding to each of the various association relationships to obtain multiple trace numbers; Determine the norm of the trace matrix composed of the plurality of traces; The sum of the norm and the weighting coefficient is determined to be the first optimization parameter.

6. The method according to claim 3, characterized in that, The process of determining the first optimization matrix corresponding to the association relationship based on the first vector matrix and the Laplacian matrix corresponding to the association relationship includes: Determine the transpose matrix corresponding to the first vector matrix to obtain the second vector matrix; The first optimized matrix is ​​obtained by determining the product of the second vector matrix, the first vector matrix, and the Laplace matrix.

7. The method according to claim 1, characterized in that, The step of training the model based on the plurality of second weight parameters and the first feature vectors corresponding to the plurality of first nodes to obtain the target neural network model includes: Based on the first relational data, with the first feature vector as the independent variable and keeping the multiple second weight parameters unchanged, the fourth relational data is obtained. The first relational data is used to represent the relationship between two independent variables, namely the first feature vector and the first weight parameter. Based on the fourth relation data, the first feature vectors corresponding to the plurality of first nodes are optimized by the current layer neural network to obtain the second feature vectors corresponding to the plurality of first nodes. Based on the second feature vectors corresponding to the plurality of first nodes, the plurality of second weight parameters are iteratively optimized through the next layer of neural network to obtain a plurality of third weight parameters, until the third feature vectors corresponding to the plurality of first nodes are output through the last layer of neural network. Based on the third feature vectors corresponding to the multiple first nodes, the model is trained to obtain the target neural network model.

8. The method according to claim 7, characterized in that, Based on the fourth relation data, the process of optimizing the first feature vector corresponding to the first node through the current layer neural network to obtain the second feature vector corresponding to the first node includes: The fourth relation data is optimized to obtain the fifth relation data. The fifth relation data is used to represent the relationship based on the first weight, the fourth feature vector corresponding to at least one second node, and the second weight parameter corresponding to at least one association relationship. The first feature vector is optimized as the number of neural network layers increases. The first weight is the weight of the first parameter item relation data in the first relation data. The at least one second node is a node among the plurality of first nodes that has at least one association relationship with the first node that is the center node. Based on the fourth feature vector corresponding to the at least one second node and the at least one second weight parameter corresponding to the at least one association relationship, the fifth feature vector corresponding to the first node under the at least one association relationship is output by the current layer neural network. The fifth feature vector is used to represent the node information of the at least one second node. The fourth feature vector and the at least one second weight parameter are both output by the previous layer neural network of the current layer neural network. The first weight, the first feature vector corresponding to the first node, and the fifth feature vector are respectively substituted into the fifth relation data, and the second feature vector corresponding to the first node is output through the current layer neural network.

9. The method according to claim 8, characterized in that, The step of substituting the first weight, the first feature vector corresponding to the first node, and the fifth feature vector into the fifth relation data, and outputting the second feature vector corresponding to the first node through the current layer neural network, includes: Based on the first weight, a third weight and a fourth weight are determined. The third weight is used to represent the importance of the first feature vector in the second feature vector, and the fourth weight is used to represent the importance of the fifth feature vector in the second feature vector. The third weight, the fourth weight, the first feature vector, and the fifth feature vector are respectively substituted into the fifth relation data, and the second feature vector is output through the current layer neural network.

10. The method according to claim 7, characterized in that, The step of training the model based on the third feature vectors corresponding to the plurality of first nodes to obtain the target neural network model includes: Based on the third feature vectors corresponding to the plurality of first nodes, the predicted labels corresponding to the plurality of first nodes are determined. Based on the label loss values ​​between the predicted labels and the actual labels corresponding to the multiple first nodes, the model is trained to obtain the target neural network model.

11. The method according to claim 1, characterized in that, The method further includes: Obtain the sixth feature vector of the target node to be classified; The sixth feature vector of the target node is extracted to obtain the seventh feature vector of the target node; Based on the multiple target weight parameters corresponding to the target neural network model and the seventh feature vector, the predicted label of the target node is determined; The target node is classified based on its predicted label.

12. A training device for a neural network model, characterized in that, The device includes: The first acquisition module is used to acquire the first feature vectors corresponding to multiple first nodes and the weight matrices corresponding to various association relationships between the multiple first nodes, wherein one association relationship corresponds to one weight matrix, and the first node is an image. An integration module is used to integrate multiple weight matrices based on a first number of the various associations to obtain multiple first weight parameters. Each weight matrix corresponds to one first weight parameter, and the first weight parameter is used to represent the importance of the association. The optimization module is used to iteratively optimize the multiple first weight parameters based on the first feature vectors corresponding to the multiple first nodes, until the multiple first weight parameters meet the convergence condition, and obtain multiple second weight parameters. The training module is used to train the model based on the plurality of second weight parameters and the first feature vectors corresponding to the plurality of first nodes respectively, so as to obtain the target neural network model; The process of integrating the weight matrix based on the first quantity of the various associations to obtain the first weight parameter includes: The multiple weight parameters in the weight matrix are integrated into the reciprocal of the first number to obtain the first weight parameter; The process involves iteratively optimizing the multiple first weight parameters based on the first feature vectors corresponding to the multiple first nodes until the multiple first weight parameters satisfy the convergence condition, thereby obtaining multiple second weight parameters, including: Obtain first relation data, which is used to represent the relationship between two independent variables, namely a first feature vector and a first weight parameter; Based on the first relational data, using the first weight parameter as the independent variable, and keeping the first feature vectors corresponding to the plurality of first nodes unchanged, the second relational data is obtained. Based on the second relational data, the plurality of first weight parameters are iteratively optimized through the current layer neural network until the plurality of first weight parameters meet the convergence condition, thereby obtaining the plurality of second weight parameters.

13. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing at least one piece of program code, which is loaded and executed by the processor to implement the training method of the neural network model as described in any one of claims 1 to 11.

Images