Method and device for evaluating credible result of graph neural network
By constructing an objective function and augmented Lagrangian function based on spatiotemporal relationships, combining the gradient alternating direction multiplier method, optimizing the node feature mask and motion rotation model, the interpretation and evaluation problems of spatiotemporal graph neural networks in human skeleton behavior recognition are solved, and highly sparse and consistent interpretation results are achieved.
Patent Information
- Application Number
- CN202510723176.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-30
- Publication Date
- 2025-09-05
AI Technical Summary
Existing spatiotemporal graph neural networks have difficulty interpreting and evaluating the complexity and dynamics of geometric spatiotemporal features in human skeleton behavior recognition, which increases the difficulty of interpreting and evaluating the behavior recognition decision process.
An objective function based on spatiotemporal relationships is constructed. The augmented Lagrangian function and the gradient alternating direction multiplier method are used to iteratively optimize the node feature mask and the human skeleton motion rotation model to interpret and evaluate the prediction results of the spatiotemporal graph neural network.
It realizes the quantitative and visual interpretation of the decision results of the spatiotemporal graph network, generates interpretation results with high sparsity and consistent with the original spatiotemporal structure, and improves the interpretability and generalization ability of the model.
Smart Images
Figure CN120599659A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to a method and device for evaluating the credible results of a graph neural network. Background Art
[0002] Human skeleton behavior recognition refers to the identification and classification of human behavior by analyzing human skeleton information. By integrating graph neural networks and various temporal learning methods, spatiotemporal graph neural networks can extract complex spatiotemporal dependencies. Spatiotemporal graph neural networks can be used for human skeleton behavior recognition and obtain prediction results. Understanding and interpreting the prediction results generated by spatiotemporal graph neural networks is crucial to improving the security and reliability of human skeleton behavior recognition using spatiotemporal graph neural networks. However, the complexity of the geometric spatiotemporal features in the spatiotemporal graphs generated by spatiotemporal graph neural networks for human skeleton behavior recognition, as well as the dynamic nature of spatiotemporal graph neural networks, poses challenges to existing spatiotemporal graph neural network trusted computing methods. This makes it difficult to interpret the intertwined geometric spatiotemporal features in spatiotemporal graph neural networks, increasing the difficulty of explaining and evaluating the behavior recognition decision process and prediction results. Summary of the Invention
[0003] To solve the above technical problems, the present invention proposes a method and device for evaluating the credible results of graph neural networks.
[0004] According to a first aspect of the present invention, a method for evaluating a credible result of a graph neural network is provided, the method comprising the following steps:
[0005] Step S1: Construct an objective function based on the spatiotemporal relationship. The objective function is:
[0006]
[0007] Among them, M n is the node feature mask matrix, It is a human skeleton motion rotation model. is the objective function based on the spatiotemporal relationship, m kj is the element in row k and column j of the node feature mask matrix, r kj is the element in row k and column j of the human skeleton motion rotation model matrix;
[0008] Step S2: Obtain the recognition result of the spatiotemporal graph neural network for skeleton behavior recognition, construct the augmented Lagrangian function corresponding to the objective function, initialize the number of iterations k = 1, and randomly initialize M n and Initial value of and M n and The initial value of is input into the augmented Lagrangian function;
[0009] Step S3: If the accuracy of the recognition result of the spatiotemporal graph neural network to be evaluated has not been improved for five consecutive times or the number of iterations k is greater than the preset value, the obtained and Assign values to M respectively n and Go to step S4; otherwise, based on and Calculate the augmented Lagrangian function about The gradient of The gradient of the augmented Lagrangian function is based on The gradient of the augmented Lagrangian function is about The gradient is updated:
[0010]
[0011] in, are the node feature mask matrix and human skeleton motion rotation model obtained in the kth iteration respectively, are the node feature mask matrix and human skeleton motion rotation model obtained in the k+1th iteration, Proj [0,1] The function represents projecting the result to [0,1], α is the update step size, α=0.01, is the augmented Lagrangian function about M n The gradient, is the augmented Lagrangian function about The gradient, is the augmented Lagrangian function corresponding to the objective function at the kth iteration, Z (k) ,U (k) ,V (k) are the auxiliary variable, the first Lagrangian multiplier and the second Lagrangian multiplier in the augmented Lagrangian function obtained at the kth iteration respectively;
[0012] Let k be equal to k+1, and go to step S3;
[0013] Step S4: M n and Evaluation results as prediction results of spatiotemporal graph neural networks.
[0014] Preferably, the human skeleton motion rotation model The expression is:
[0015]
[0016]
[0017] Among them, i is the i-th node of the human skeleton, The motion vector of frame t Motion vector to frame t+1 The rotation angle, is the motion vector of frame t Motion vector to frame t+1 The axis of rotation, is the rotation matrix of the space-time graph corresponding to two consecutive frames, t is the frame number of the video frame input when the space-time graph neural network to be evaluated is used for skeleton behavior recognition, T is the number of video frames input when the space-time graph neural network to be evaluated is used for skeleton behavior recognition, N is the number of nodes of the human skeleton in each frame, is the first element of the rotation axis, The second element of the rotation axis, The third element of the rotation axis.
[0018] Preferably, in step S2, an augmented Lagrangian function corresponding to the objective function is constructed, and the augmented Lagrangian function is:
[0019]
[0020] Where ρ is the penalty parameter, ||·|| 2 It is a quadratic normal form.
[0021] Preferably, in step S3, based on and Calculate the augmented Lagrangian function about The gradient of The gradient includes:
[0022]
[0023] in, The augmented Lagrangian function is about The gradient, is the augmented Lagrangian function corresponding to the objective function, The augmented Lagrangian function is about gradient.
[0024] Preferably, in step S3, based on the augmented Lagrangian function The gradient of the augmented Lagrangian function is about The gradient of is updated, including the following updates:
[0025]
[0026] Among them, Z (k+1) 、U (k+1) 、V (k+1)are the auxiliary variable, the first Lagrangian multiplier, and the second Lagrangian multiplier in the augmented Lagrangian function obtained in the k+1th iteration respectively.
[0027] According to a second aspect of the present invention, a device for evaluating a credible result of a graph neural network is provided, the device comprising:
[0028] Objective function building module: configured to build an objective function based on spatiotemporal relationships. The objective function is:
[0029]
[0030] Among them, M n is the node feature mask matrix, It is a human skeleton motion rotation model. is the objective function based on the spatiotemporal relationship, m kj is the element in row k and column j of the node feature mask matrix, r kj is the element in row k and column j of the human skeleton motion rotation model matrix;
[0031] Conversion module: configured to obtain the recognition results of the spatiotemporal graph neural network for skeleton behavior recognition to be evaluated, construct the augmented Lagrangian function corresponding to the objective function, initialize the number of iterations k = 1, and randomly initialize M n and Initial value of and M n and The initial value of is input into the augmented Lagrangian function;
[0032] Iteration module: If the accuracy of the recognition result of the spatiotemporal graph neural network to be evaluated does not improve for five consecutive times or the number of iterations k is greater than the preset value, the obtained and Assign values to M respectively n and Trigger the evaluation module; otherwise, based on and Calculate the augmented Lagrangian function about The gradient of The gradient of the augmented Lagrangian function is based on The gradient of the augmented Lagrangian function is about The gradient is updated:
[0033]
[0034] in, are the node feature mask matrix and human skeleton motion rotation model obtained in the kth iteration respectively, are the node feature mask matrix and human skeleton motion rotation model obtained in the k+1th iteration, Proj [0,1] The function represents projecting the result to [0,1], α is the update step size, α=0.01, is the augmented Lagrangian function about M n The gradient, is the augmented Lagrangian function about The gradient, is the augmented Lagrangian function corresponding to the objective function at the kth iteration, Z (k) ,U (k) ,V (k) are the auxiliary variable, the first Lagrangian multiplier and the second Lagrangian multiplier in the augmented Lagrangian function obtained at the kth iteration respectively;
[0035] Let k equal to k+1, triggering the iteration module;
[0036] Evaluation Module: n and Evaluation results as prediction results of spatiotemporal graph neural networks.
[0037] According to a third aspect of the present invention, there is provided an electronic device, comprising:
[0038] A processor, which is used to execute multiple instructions;
[0039] A memory for storing a plurality of instructions;
[0040] The plurality of instructions are used to be stored by the memory and loaded and executed by the processor to implement the method as described above.
[0041] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein a plurality of instructions are stored in the storage medium; the plurality of instructions are used for a processor to load and execute the aforementioned method.
[0042] Based on the characteristics of human skeleton behavior recognition, the present invention constructs an objective function based on the spatiotemporal relationship, uses the gradient alternating direction multiplier method to solve the skeleton motion rotation model and the node feature mask corresponding to the human body obtained by the spatiotemporal graph neural network to be evaluated, and interprets and evaluates the spatiotemporal graph neural network to be evaluated based on the skeleton motion rotation model and the node feature mask corresponding to the human body. The interpretation result obtained by the present invention has a high degree of sparsity and is consistent with the spatiotemporal geometry of the input spatiotemporal graph neural network to be evaluated in terms of geometric consistency of the spatiotemporal structure. Figure 1 The present invention can quantitatively and visually explain the decision results of the spatiotemporal graph network, and the explanation results are highly sparse.
[0043] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] The accompanying drawings, which constitute part of the present invention, are used to provide a further understanding of the present invention. The present invention is described with the following accompanying drawings. In the accompanying drawings:
[0045] Figure 1 A flowchart of a method for evaluating credible results of a graph neural network according to one embodiment of the present invention. DETAILED DESCRIPTION
[0046] First combine Figure 1 The following describes a method for evaluating the credibility of graph neural network results according to an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:
[0047] Step S1: Construct an objective function based on the spatiotemporal relationship. The objective function is:
[0048]
[0049] Among them, M n is the node feature mask matrix, It is a human skeleton motion rotation model. is the objective function based on the spatiotemporal relationship, m kj is the element in row k and column j of the node feature mask matrix, r kj is the element in row k and column j of the human skeleton motion rotation model matrix;
[0050] Step S2: Obtain the recognition result of the spatiotemporal graph neural network for skeleton behavior recognition, construct the augmented Lagrangian function corresponding to the objective function, initialize the number of iterations k = 1, and randomly initialize M n and Initial value of and M n and The initial value of is input into the augmented Lagrangian function;
[0051] Step S3: If the accuracy of the recognition result of the spatiotemporal graph neural network to be evaluated has not been improved for five consecutive times or the number of iterations k is greater than the preset value, the obtained and Assign values to M respectively n and Go to step S4; otherwise, based on and Calculate the augmented Lagrangian function about The gradient of The gradient of the augmented Lagrangian function is based on The gradient of the augmented Lagrangian function is about The gradient is updated:
[0052]
[0053] in, are the node feature mask matrix and human skeleton motion rotation model obtained in the kth iteration respectively, are the node feature mask matrix and human skeleton motion rotation model obtained in the k+1th iteration, Proj [0,1] The function represents projecting the result to [0,1], α is the update step size, α=0.01, is the augmented Lagrangian function about M n The gradient, is the augmented Lagrangian function about The gradient, is the augmented Lagrangian function corresponding to the objective function at the kth iteration, Z (k) ,U (k) ,V (k) are the auxiliary variable, the first Lagrangian multiplier and the second Lagrangian multiplier in the augmented Lagrangian function obtained at the kth iteration respectively;
[0054] Let k be equal to k+1, and go to step S3;
[0055] Step S4: M n and Evaluation results as prediction results of spatiotemporal graph neural networks.
[0056] Furthermore, the human skeleton motion rotation model The expression is:
[0057]
[0058] Among them, i is the i-th node of the human skeleton, The motion vector of frame t Motion vector to frame t+1 The rotation angle, is the motion vector of frame t Motion vector to frame t+1 The axis of rotation, is the rotation matrix of the space-time graph corresponding to two consecutive frames, t is the frame number of the video frame input when the space-time graph neural network to be evaluated is used for skeleton behavior recognition, T is the number of video frames input when the space-time graph neural network to be evaluated is used for skeleton behavior recognition, N is the number of nodes of the human skeleton in each frame, is the first element of the rotation axis, The second element of the rotation axis, The third element of the rotation axis.
[0059] In the present invention, is the first element of the rotation axis, The value of The second element of the rotation axis, The value of is the third element of the rotation axis, The value of
[0060] Furthermore, in step S2, an augmented Lagrangian function corresponding to the objective function is constructed, and the augmented Lagrangian function is:
[0061]
[0062] Where ρ is the penalty parameter, ||·|| 2 It is a quadratic normal form.
[0063] Furthermore, in step S3, based on and Calculate the augmented Lagrangian function about The gradient of The gradient includes:
[0064]
[0065] in, The augmented Lagrangian function is about The gradient, is the augmented Lagrangian function corresponding to the objective function, The augmented Lagrangian function is about gradient.
[0066] Furthermore, in step S3, based on the augmented Lagrangian function The gradient of the augmented Lagrangian function is about The gradient of is updated, including the following updates:
[0067]
[0068] In the iterative process, the present invention uses the geometric transfer model and node feature mask in the updating process to geometrically perturb the spatiotemporal graph in the spatiotemporal graph network model, and uses the gradient alternating direction multiplier method to update the objective function based on the spatiotemporal relationship under the current perturbation, gradually updating and optimizing and M n When the algorithm reaches the maximum number of iterations, the final space-time dimensions and geometric space under perturbation conditions are generated. and M n These are used as the interpretation results of the geometric spatiotemporal features of the prediction results of the spatiotemporal graph neural network for skeleton behavior recognition. Experiments have shown that the present invention has an accurate evaluation effect on the ST-GCN network and the HD-GCN network.
[0069] In the present invention, the construction of the human skeleton motion rotation model is based on the following reasoning.
[0070] The human skeleton motion rotation model is used to describe the complex geometric topology and highly dependent time relationship in the space-time graph of the human skeleton. Specifically, if the node in the space-time graph of the tth frame is known The three-dimensional coordinates of Nodes in the spatiotemporal graph of frame t+1 The three-dimensional coordinates of Then the human skeleton motion vector in the space-time graph of the tth frame is The definition is shown below.
[0071]
[0072] Therefore, the human skeleton motion vector set is In the space-time diagram, there is a relationship between the current motion vector and the motion vector of the next state over time, which is called the rotation state matrix. According to the rotation state matrix of the human skeleton between different frames, a geometric transfer model is constructed. In the human skeleton motion vector set B t In the example, it is assumed that the motion vector of the t-th frame is The three-dimensional coordinates of The motion vector moving to the t+1th frame is The three-dimensional coordinates of According to the definition of dot product, the rotation angle of this motion state The definition is shown below.
[0073]
[0074] Then, the rotation angle The plane where it is located is and The plane formed by the two vectors, then the rotation axis must be perpendicular to the plane, then the rotation axis between the two vectors As shown in the following formula.
[0075]
[0076] Finally, according to the rotation angle in the current motion state and rotation axis Using the Rodrigues' rotation formula, the corresponding rotation matrix of two consecutive frames of space-time graphs can be obtained As shown in the following formula.
[0077]
[0078] Therefore, in the spatiotemporal graph of the T-frame human skeleton, the geometric transfer model of human motion in the spatiotemporal space is defined as follows.
[0079]
[0080] This geometric transfer model describes the complex geometric spatiotemporal dependencies in the spatiotemporal graph from a temporal perspective. This model, combined with node feature masks, can effectively utilize the geometric spatiotemporal relationships in the spatiotemporal graph to perform geometric perturbations on the spatiotemporal graph network model, which is used to understand the importance of geometric spatiotemporal features in behavior recognition models.
[0081] To quantify the importance of geometric spatiotemporal features in a spatiotemporal graph network to the prediction results of an action recognition model, this paper constructs an objective function based on spatiotemporal relationships. This function generates an explanatory analysis of the spatiotemporal importance of significant model impact when interpreting a spatiotemporal graph network model. This objective function integrates the spatiotemporal constraints of the geometric transfer model and the spatiotemporal graph model, exploring key geometric spatiotemporal features that influence the model's prediction results. This objective function further reveals the relative importance of geometry and spatiotemporal relationships within the geometric spatiotemporal features in action recognition network models.
[0082] First, according to the human skeleton motion rotation model and node feature mask of the spatiotemporal graph, and combined with the node feature F(V) in the spatiotemporal graph, the spatiotemporal structure of the input spatiotemporal graph in the spatiotemporal graph network model is perturbed. The spatiotemporal structure perturbation prediction loss term is The definition is shown below.
[0083]
[0084] Among them, Φ(F(V)) is the prediction result of the behavior recognition network model Φ, M n ∈(0,1) is a randomly generated node feature mask, It measures the mean square error between the prediction result after geometric perturbation and the original prediction Φ(F(V)).
[0085] Secondly, in order to effectively capture the importance of complex geometric spatiotemporal features in the spatiotemporal graph to the model prediction results, a time constraint loss term is constructed. and spatial constraint loss These two loss terms constrain the geometric perturbation process of the action recognition network model from the aspects of time and space respectively. The definition is shown below.
[0086]
[0087] Used to encourage geometric transfer masks and node feature mask M n The sparsity of ‖·‖1 represents the l1 norm, which can further promote the sparsity and stability of the model interpretation. The sparsity mask helps to identify the geometric features that have the greatest impact on the prediction results, thereby improving the model's interpretability and generalization ability.
[0088] Then, in the process of geometric perturbation of the spatiotemporal graph neural network model, a spatial constraint loss term is constructed to ensure the consistency between the interpretation result and the original geometric structure of the spatiotemporal graph. As shown in the following formula.
[0089]
[0090] in, represents the Kullback–Leibler divergence (KL divergence) related to the critical structural consistency in the network, which is used to penalize the importance weight of the edge. A is the adjacency matrix of the spatiotemporal graph, W edge Behavior recognition network model after geometric perturbation By minimizing the KL divergence, the model's interpretation results are constrained to maintain a faithful representation of the original spatiotemporal structure.
[0091] Finally, the spatiotemporal constraint objective function The definition is shown below.
[0092]
[0093] Among them, the hyperparameters α and β control the influence of spatiotemporal constraints respectively.
[0094] The present invention adopts a decomposition-coordination strategy based on the Alternating Direction Method of Multipliers (ADMM) to decompose and optimize the importance distribution of the time dimension and the geometric space dimension in the geometric space-time features. The large-scale global space-time graph trustworthy computation problem is decomposed into two smaller and relatively easier to solve sub-problems of the importance distribution of time and geometric space. Through this strategy, the global trustworthy computation of the space-time graph model can be achieved by coordinating the solutions of these sub-problems. This will gradually improve the accuracy of the space-time graph interpretation generated by the algorithm.
[0095] First, the general form of the objective function optimization problem is shown in the following formula.
[0096]
[0097] Among them, the constraint condition represents the matrix M n and Each element value m in ij and r ij The distribution must be between 0 and 1. In order to use the gradient ADMM algorithm, auxiliary variables need to be introduced to make the optimization problem more suitable for decomposition and solution. Define auxiliary variables The augmented Lagrangian function of the above equation can be restated as shown below.
[0098]
[0099] Where U and V represent the Lagrange multipliers introduced during the decomposition of the ADMM optimization problem. The penalty parameter ensures that constraint violations are penalized. Gradient descent is used to optimize the node feature masks and the geometric transfer model, which only requires computing the gradient of the objective function, without the need for complex second-order derivatives.
[0100] Then, update the gradient of the augmented Lagrangian function. Calculate the augmented Lagrangian function with respect to M n and The gradient of is shown below.
[0101]
[0102] During the ADMM iterative optimization process, each iteration calculates About M n and The gradient of M is updated according to the gradient descent method n and Specifically, at the k+1th iteration, M n and The update rule is shown below.
[0103]
[0104] Among them, k is the number of iterations, and in this algorithm the number of iterations is set to 50. [0,1] (·) indicates that the result is projected into the range [0, 1] at each iteration, and the update step size α is set to 0.01. Next, the auxiliary variable Z is updated as shown in the following formula.
[0105]
[0106] Finally, update the Lagrange multipliers U and V as shown below.
[0107]
[0108] By substituting Lagrange multipliers and penalty parameters, ADMM simplifies complex iterative optimization into multiple simple iterative steps with fewer parameters, thereby solving the optimization problem more efficiently. This process gives mask optimization stronger computational reliability. By using an optimization algorithm based on the gradient alternating direction multiplier method to optimize the geometric spatiotemporal relationship in the spatiotemporal graph model, the optimal explanation, that is, the optimal node feature mask, can be obtained. and optimal geometric relationship distribution This not only ensures the sparsity of the explanation results, but also enables in-depth exploration of the impact of geometric spatiotemporal features on human behavior recognition and prediction based on the spatiotemporal graph network model.
[0109] The present invention provides a device for evaluating credible results of a graph neural network, the device comprising:
[0110] Objective function building module: configured to build an objective function based on spatiotemporal relationships. The objective function is:
[0111]
[0112] Among them, M n is the node feature mask matrix, It is a human skeleton motion rotation model. is the objective function based on the spatiotemporal relationship, m kj is the element in row k and column j of the node feature mask matrix, r kj is the element in row k and column j of the human skeleton motion rotation model matrix;
[0113] Conversion module: configured to obtain the recognition results of the spatiotemporal graph neural network for skeleton behavior recognition to be evaluated, construct the augmented Lagrangian function corresponding to the objective function, initialize the number of iterations k = 1, and randomly initialize M n and Initial value of and M n and The initial value of is input into the augmented Lagrangian function;
[0114] Iteration module: If the accuracy of the recognition result of the spatiotemporal graph neural network to be evaluated does not improve for five consecutive times or the number of iterations k is greater than the preset value, the obtained and Assign values to M respectively n and Trigger the evaluation module; otherwise, based on and Calculate the augmented Lagrangian function about The gradient of The gradient of the augmented Lagrangian function is based on The gradient of the augmented Lagrangian function is about The gradient is updated:
[0115]
[0116] in, are the node feature mask matrix and human skeleton motion rotation model obtained in the kth iteration respectively, are the node feature mask matrix and human skeleton motion rotation model obtained in the k+1th iteration, Proj [0,1] The function represents projecting the result to [0,1], α is the update step size, α=0.01, is the augmented Lagrangian function about M n The gradient, is the augmented Lagrangian function about The gradient, is the augmented Lagrangian function corresponding to the objective function at the kth iteration, Z (k) ,U (k) ,V (k) are the auxiliary variable, the first Lagrangian multiplier and the second Lagrangian multiplier in the augmented Lagrangian function obtained at the kth iteration respectively;
[0117] Let k equal to k+1, triggering the iteration module;
[0118] Evaluation Module: n and Evaluation results as prediction results of spatiotemporal graph neural networks.
[0119] An embodiment of the present invention further provides an electronic device, including:
[0120] A processor, which is used to execute multiple instructions;
[0121] A memory for storing a plurality of instructions;
[0122] The plurality of instructions are used to be stored by the memory and loaded and executed by the processor to implement the method as described above.
[0123] An embodiment of the present invention further provides a computer-readable storage medium, wherein a plurality of instructions are stored in the storage medium; the plurality of instructions are used for a processor to load and execute the method described above.
[0124] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.
[0125] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, which may be electrical, mechanical or other forms.
[0126] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.
[0127] In addition, the functional units in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional units.
[0128] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a physical server, or a network cloud server, etc., and requires the Ubuntu operating system to be installed) to perform some of the steps of the method described in various embodiments of the present invention. The aforementioned storage medium includes: a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.
[0129] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Any simple modifications, equivalent changes and modifications made to the above embodiment based on the technical essence of the present invention still fall within the scope of the technical solution of the present invention.
Claims
1. A method for evaluating the credibility of graph neural network results, characterized in that: Methods include: Step S1: Construct an objective function based on the spatiotemporal relationship. The objective function is: Among them, M n is the node feature mask matrix, It is a human skeleton motion rotation model. is the objective function based on the spatiotemporal relationship, m kj is the element in row k and column j of the node feature mask matrix, r kj is the element in row k and column j of the human skeleton motion rotation model matrix; Step S2: Obtain the recognition result of the spatiotemporal graph neural network for skeleton behavior recognition, construct the augmented Lagrangian function corresponding to the objective function, initialize the number of iterations k = 1, and randomly initialize M n and Initial value of and M n and The initial value of is input into the augmented Lagrangian function; Step S3: If the accuracy of the recognition result of the spatiotemporal graph neural network to be evaluated has not been improved for five consecutive times or the number of iterations k is greater than the preset value, the obtained and Assign values to M respectively n and Go to step S4; otherwise, based on and Calculate the augmented Lagrangian function about The gradient of The gradient of the augmented Lagrangian function is based on The gradient of the augmented Lagrangian function is about The gradient is updated: in, are the node feature mask matrix and human skeleton motion rotation model obtained in the kth iteration respectively, are the node feature mask matrix and human skeleton motion rotation model obtained in the k+1th iteration, Proj [0,1] The function represents projecting the result to [0,1], α is the update step size, α=0.01, is the augmented Lagrangian function about M n The gradient, is the gradient of the augmented Lagrangian function with respect to R, is the augmented Lagrangian function corresponding to the objective function at the kth iteration, Z (k) ,U (k) ,V (k) are the auxiliary variable, the first Lagrangian multiplier and the second Lagrangian multiplier in the augmented Lagrangian function obtained at the kth iteration respectively; Let k be equal to k+1, and go to step S3; Step S4: M n and Evaluation results as prediction results of spatiotemporal graph neural networks.
2. The method according to claim 1, wherein Human skeleton motion rotation model The expression is: Among them, i is the i-th node of the human skeleton, The motion vector b of the tth frame i t Motion vector to frame t+1 The rotation angle, is the motion vector of frame t Motion vector to frame t+1 The axis of rotation, is the rotation matrix of the space-time graph corresponding to two consecutive frames, t is the frame number of the video frame input when the space-time graph neural network to be evaluated is used for skeleton behavior recognition, T is the number of video frames input when the space-time graph neural network to be evaluated is used for skeleton behavior recognition, N is the number of nodes of the human skeleton in each frame, is the first element of the rotation axis, The second element of the rotation axis, The third element of the rotation axis.
3. The method according to claim 2, wherein In step S2, an augmented Lagrangian function corresponding to the objective function is constructed, and the augmented Lagrangian function is: Where ρ is the penalty parameter, ||·|| 2 It is a quadratic normal form.
4. The method according to claim 3, wherein In step S3, based on and Calculate the augmented Lagrangian function about The gradient of The gradient includes: in, The augmented Lagrangian function is about The gradient, is the augmented Lagrangian function corresponding to the objective function, The augmented Lagrangian function is about gradient.
5. The method according to claim 4, wherein In step S3, based on the augmented Lagrangian function The gradient of the augmented Lagrangian function is about The gradient of is updated, including the following updates: Among them, Z (k+1) 、U (k+1) 、V (k+1) are the auxiliary variable, the first Lagrangian multiplier, and the second Lagrangian multiplier in the augmented Lagrangian function obtained in the k+1th iteration respectively.
6. A device for evaluating the credibility of graph neural network results, characterized in that: The device includes: Objective function building module: configured to build an objective function based on spatiotemporal relationships. The objective function is: Among them, M n is the node feature mask matrix, It is a human skeleton motion rotation model. is the objective function based on the spatiotemporal relationship, m kj is the element in row k and column j of the node feature mask matrix, r kj is the element in row k and column j of the human skeleton motion rotation model matrix; Conversion module: configured to obtain the recognition results of the spatiotemporal graph neural network for skeleton behavior recognition to be evaluated, construct the augmented Lagrangian function corresponding to the objective function, initialize the number of iterations k = 1, and randomly initialize M n and Initial value of and M n and The initial value of is input into the augmented Lagrangian function; Iteration module: If the accuracy of the recognition result of the spatiotemporal graph neural network to be evaluated does not improve for five consecutive times or the number of iterations k is greater than the preset value, the obtained and Assign values to M respectively n and Trigger the evaluation module; otherwise, based on and Calculate the augmented Lagrangian function about The gradient of The gradient of the augmented Lagrangian function is based on The gradient of the augmented Lagrangian function is about The gradient is updated: in, are the node feature mask matrix and human skeleton motion rotation model obtained in the kth iteration respectively, are the node feature mask matrix and human skeleton motion rotation model obtained in the k+1th iteration, Proj [0,1] The function represents projecting the result to [0,1], α is the update step size, α=0.01, is the augmented Lagrangian function about M n The gradient, is the augmented Lagrangian function about The gradient, is the augmented Lagrangian function corresponding to the objective function at the kth iteration, Z (k) ,U (k) ,V (k) are the auxiliary variable, the first Lagrangian multiplier and the second Lagrangian multiplier in the augmented Lagrangian function obtained at the kth iteration respectively; Let k equal to k+1, triggering the iteration module; Evaluation Module: n and Evaluation results as prediction results of spatiotemporal graph neural networks.
7. An electronic device comprising: A processor, which is used to execute multiple instructions; A memory for storing a plurality of instructions; The plurality of instructions are used to be stored in the memory and loaded and executed by the processor according to any one of claims 1 to 5.
8. A computer-readable storage medium, wherein a plurality of instructions are stored in the storage medium; the plurality of instructions are used for a processor to load and execute the method according to any one of claims 1 to 5.