A gesture recognition method, system, device and medium

Abstract

This invention discloses a gesture recognition method, system, device, and medium, relating to the field of gesture recognition technology. The method includes: collecting hand surface point cloud data of various gestures from an individual sample; performing point cloud sparsification on the hand surface point cloud data to generate a variable-density hand surface point cloud; selecting different target points in the variable-density hand surface point cloud as center nodes, and constructing a local graph for each node using the nodes closest to the center node as neighbor nodes; obtaining the changing features of the connection relationship between nodes and edges in the local graph by changing the geometric features of the hand; extracting local features at different scales using edge convolution; stitching together the local features extracted at different scales to obtain a multi-scale local feature representation; and recognizing the gesture of the individual sample based on the local feature representation. This method accurately distinguishes subtle differences in complex gestures, improving the accuracy and robustness of gesture recognition.

Description

Technical Field

[0001] This invention relates to the field of gesture recognition technology, specifically to a gesture recognition method, system, device, and medium. Background Technology

[0002] In gesture recognition, existing methods are easily affected by lighting, occlusion, and changes in viewing angle. Currently, multi-view methods and voxel methods are commonly used, such as... Figure 1 As shown: The multi-view method converts point clouds into a two-dimensional image sequence and captures three-dimensional projected images from multiple perspectives for analysis. The multi-view method relies on multi-view projected image analysis, but because it depends on precise perspectives, even small deviations in perspective can reduce recognition performance. In addition, spatial information is lost during the projection process, making it difficult to distinguish subtle differences when recognizing complex gestures. The voxel method converts point clouds into three-dimensional voxel meshes. However, due to the uneven distribution of point clouds on the hand surface and the presence of a large number of blank areas, direct use will result in a large number of empty voxels, which has a sparsity problem, increases the difficulty of data processing and introduces interference. The quantization process will also produce quantization errors, affecting the accuracy of gesture recognition.

[0003] In summary, existing methods suffer from problems such as information loss, sparsity, and quantization errors when processing point cloud data, resulting in low accuracy in processing point cloud data and poor intelligent rehabilitation effects for hand functions. Summary of the Invention

[0004] To address the shortcomings of existing technologies in processing point cloud data, such as information loss, sparsity, and quantization errors, which result in low accuracy, this invention proposes a gesture recognition method, system, device, and medium. By collecting point cloud data from the hand surface and aggregating the point cloud based on the KNN Graph algorithm, multi-scale edge convolution is achieved, thereby solving the problems existing in the prior art.

[0005] A gesture recognition method includes the following steps:

[0006] Collect hand surface point cloud data of the individual gesture to be identified, and perform point cloud sparsification processing on the hand surface point cloud data to generate variable density hand surface point cloud.

[0007] The center point of the variable density hand surface point cloud is selected as the center node, and the nodes closest to the center node are selected as neighbor nodes. Undirected edges are established between the center node and each neighbor node, and the Euclidean distance between each node is used as the weight of the edge to construct a local graph structure.

[0008] By changing the geometric characteristics of the hand, the change characteristics of the connection relationship between the center node and its neighbor nodes in the local graph structure are obtained; and the feature information of the center node and its neighbor nodes around it is integrated to obtain the local feature; based on the local graph structure, the local features under different scales are extracted by adjusting the number of neighbor nodes around the center node; the local features extracted under different scales are spliced to obtain the multi-scale local feature representation;

[0009] According to the local feature representation, the gesture of the individual to be recognized is recognized.

[0010] Further, after generating the variable-density hand surface point cloud, Fourier transform is performed on the variable-density hand surface point cloud in the graph structure, and the point cloud data is converted from the spatial domain to the frequency domain, which is specifically represented as:

[0011] ;

[0012] Wherein, is the Fourier transform result, is a set of feature values associated with the graph node, corresponds to the frequency of the frequency domain, is the first feature of the node, is the first feature vector, is the first component of the first feature vector; represents the number of points in the hand surface point cloud; is an index variable of the summation operation; n is the m feature value corresponding to the frequency domain representation; represents a parameter related to the sampling interval of the frequency; N The transformation result is converted into a matrix form by matrix multiplication, which is represented as:

[0013]

[0014] ;

[0015] Wherein, is the feature vector of the first node on the graph.

[0016] Further, the KNN Graph algorithm is used to find the K nearest points to the center node as the neighbor nodes in the local graph structure.

[0017] Further, the change characteristics of the connection relationship between the center node and its neighbor nodes in the local graph structure are obtained by performing convolution operation on the adjacency matrix, the degree matrix and the feature matrix in the local graph structure, and the change characteristics are specifically represented as: ​​​

[0018] ;

[0019] wherein, is an activation function, is a square root of an inverse of a degree matrix is a convolution kernel; is a feature matrix; the degree matrix is represented as:

[0020] ;

[0021] wherein, is a sum of weights of edges connected to the point ;

[0022] the feature matrix is represented as:

[0023] ;

[0024] wherein, is the i-th column of the feature matrix;

[0025] the adjacency matrix is a matrix of , which is represented as:

[0026] .

[0027] Further, the multi-scale local feature representation is obtained by employing multiple edge convolutions of different scales to extract local features of each center node at different scales and concatenating them, and the generation process is specifically represented as:

[0028] ;

[0029] wherein, S represents the number of scales, is a set of neighborhood points of the center node at scale s ; and are weight matrices for relationships between the center node and the neighborhood nodes and their own features at the corresponding scales, is element-wise multiplication; b is a bias term, represents a feature representation of the center node, represents a feature representation of the neighborhood node.

[0030] The application also includes a gesture recognition system, comprising:

[0031] ​​The data acquisition module is configured to acquire hand surface point cloud data of a gesture of an individual to be recognized, and perform point cloud sparsification processing on the hand surface point cloud data to generate a variable-density hand surface point cloud.

[0032] The graph structure construction module is configured to select a center point in the variable-density hand surface point cloud as a center node, select a plurality of nodes closest to the center node as neighbor nodes, establish an undirected edge between the center node and each neighbor node, and construct a local graph structure by taking the Euclidean distance between each node as the weight of the edge.

[0033] The feature extraction module is configured to obtain a change feature of a connection relationship between the center node and the neighbor nodes in the local graph structure by changing the geometric feature of the hand, integrate feature information of the center node and the neighbor nodes around the center node to obtain a local feature, extract the local feature at different scales by adjusting the number of the neighbor nodes around the center node based on the local graph structure, and splice the local features extracted at different scales to obtain a multi-scale local feature representation.

[0034] The gesture recognition module is configured to recognize the gesture of the individual to be recognized according to the local feature representation.

[0035] The present application also includes a gesture recognition computer device, which comprises a memory, a processor, and a computer program stored in the memory, and the processor implements the steps of the gesture recognition method when executing the computer program.

[0036] The present application also includes a readable storage medium, which stores a computer program, and the computer program comprises program instructions, and the program instructions are used to execute the steps of the gesture recognition method when executed by a processor.

[0037] The present application provides a gesture recognition method, which has the following beneficial effects:

[0038] The application solves the problem of large difference in distribution density of hand surface point cloud data by sparse processing, increasing the density of points in sparse areas to supplement key information, reasonably reducing the number of points in dense areas to remove redundant information, and making the point cloud distribution more uniform and reasonable; by constructing a local structure diagram, various features of the hand surface point cloud in the local area are effectively organized and represented, which helps to improve the accuracy and efficiency of gesture recognition; a multi-scale extraction method is also used to obtain more rich local features, so that the extracted features have different emphases, small scales can capture detailed features such as subtle shape changes at finger joints, and large scales pay more attention to overall geometric structure features such as the general shape of the palm and the relative position relationship of the fingers, the different scale features are spliced together, which can greatly enrich the dimension and information content of the features, improve the processing ability of the model to complex geometric structures, accurately distinguish the subtle differences of complex gestures, and improve the accuracy and robustness of gesture recognition. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 A framework diagram of the two methods of multi-view method and voxel method in the background art of the application is shown in the figure.

[0040] Figure 2 A structure diagram of the gesture recognition system in the embodiment of the application is shown in the figure.

[0041] Figure 3 A flowchart of the gesture recognition method in the embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all the embodiments.

[0043] The application provides a gesture recognition method, as shown in the figure, specifically comprising the following steps: Figure 3

[0044] S1, data acquisition and preprocessing: build a data acquisition environment with uniform light and less interference, reasonably set the acquisition range, comprehensively acquire hand surface point cloud data in different directions, environments and experimenters, and then perform unified formatting and standardization processing to construct a point cloud data set containing various gestures and different experimenters.

[0045] ​To address the problem of large differences in the distribution density of hand surface point cloud data (less information in sparse areas and redundant information in dense areas), a variable density hand surface point cloud is first generated by an innovative method, which targets different sparse areas. In sparse areas, the density of points is appropriately increased to supplement key information, and in dense areas, the number of points is reasonably reduced to remove redundant information, making the point cloud distribution more uniform and reasonable, providing high-quality data for model training, and enabling the model to better adapt to input data of different sparsity. In the training stage, a mixed density training strategy is used to train the model using point cloud data of multiple different densities. This enables the model to learn more general feature representations, rather than being limited to features of specific density data. For example, when facing data with varying sparsity in a new environment, the model can still accurately recognize gestures based on the learned general features, thereby improving the adaptability and accuracy of the model in complex and unknown environments. The model is trained based on variable density point cloud data, using a mixed density training strategy to enable the model to learn general feature representations to adapt to data of different sparsity, achieving accurate gesture recognition.

[0046] The processing of hand surface point cloud sparsity includes:

[0047] (1) Data problem analysis: The hand surface point cloud data collected by the device has large differences in distribution density due to environmental and user habit influences. There is less information in sparse areas and redundant information in dense areas, which not only interferes with the model's accurate recognition of gestures, but also increases the difficulty and computational overhead of model training, hindering the efficient implementation of gesture recognition.

[0048] (2) Variable density point cloud generation: To overcome the problem of point cloud sparsity, a variable density hand surface point cloud is generated. This method is based on the sparsity characteristics of point clouds and targets different sparse areas for processing. In sparse areas, the density of points is appropriately increased to supplement key information, and in dense areas, the number of points is reasonably reduced to remove redundant information, making the point cloud distribution more uniform and reasonable, providing high-quality data for subsequent processing.

[0049] (3) Sparsity processing and model optimization: With the help of variable density point cloud, the model can better adapt to input data of different sparsity. In the training stage, a mixed density training strategy is used to train the model using point cloud data of multiple different densities. This enables the model to learn more general feature representations, rather than being limited to features of specific density data. For example, when facing data with varying sparsity in a new environment, the model can still accurately recognize gestures based on the learned general features, thereby improving the adaptability and accuracy of the model in complex and unknown environments.

[0050] (4) Application of Fourier transform: Fourier transform is applied to the graph structure of the variable density hand surface point cloud: ;

[0051] wherein, is the Fourier transform result, is a set of feature values associated with the graph nodes, corresponds to the frequency domain, is the first feature of the node, is the first component of the first eigenvector; represents the number of points in the hand surface point cloud; n is the index variable of the summation operation; m is the is the representation of the frequency domain corresponding to the feature value is obtained by operating the features of the nodes on the graph and the eigenvector components ; N represents a parameter related to the sampling interval of the frequency.

[0052] It is extended to matrix form by matrix multiplication, and the formula is:

[0053] ;

[0054] where, is the eigenvector of the node on the graph. The Fourier transform converts the point cloud data from the spatial domain to the frequency domain, allowing the model to analyze the data features from different frequency angles and mine the local domain features hidden in the data, further improving the accuracy of gesture recognition and the performance of the model.

[0055] S2, aggregation of hand surface point cloud on graph structure:

[0056] a) Node determination:

[0057] Select the target point as the center node: in the local graph, it is in the core position, and it is the reference point for observing and analyzing the features of the local area. All information related to this local graph is developed around this center node, which is like a "hub" and other point information will interact and associate with it.

[0058] The nearest K points are selected as neighbor nodes: the K nearest points to the target point are found by the KNN Graph algorithm, which become the neighbor nodes in the local graph. If the hand surface point cloud contains points, then the adjacency matrix is a matrix, which represents the connection relationship between points. The KNN Graph is used to construct the adjacency matrix, and the expression is These neighbor nodes are spatially close to the target node, and they contain important information about the environment and gesture features around the target node. Each neighbor node has a certain spatial relationship with the target node, and together they form the local neighborhood of the target node.

[0059] b) Construction of edges:

[0060] Connection of edges: After determining the center node and neighbor nodes, edges need to be established between them to connect these nodes and form the structure of the graph. Usually, edges are undirected, i.e., the connection from the center node to the neighbor node and from the neighbor node to the center node is equivalent. This is because in the context of constructing a local graph based on point cloud data, the relationship between points is more based on spatial proximity rather than specific semantics in the direction. For example, in a local graph representing a hand surface point cloud, the relationship between a point and its neighbor points is mainly their spatial proximity, forming a degree matrix is a diagonal matrix, represented as:

[0061] ;

[0062] In the formula, is the sum of the weights of the edges connected to the point , and there is no obvious directionality, so undirected edges are used to connect them.

[0063] Determination of edge weights: The weight of an edge can be determined according to the distance between nodes or other related features. A common method is. For example, the Euclidean distance between nodes can be used as the weight of the edge, i.e., where represents the weight of the edge between node and node, and is the Euclidean distance between node and node. Of course, in addition to distance, the weight of the edge can also be determined according to other factors, such as the similarity of the normal vectors of the points, etc. Assuming that each node (each point in the hand surface point cloud) has a feature vector, these feature vectors form a feature matrix , represented as:

[0064] ;

[0065] In the formula, is the column of the feature matrix, and the convolution operation is performed on the graph structure, and the output feature matrix depends on the requirements of the gesture recognition task and the characteristics of the data.

[0066] c) Spatial relationship representation of local graph:

[0067] Spatial Structure Representation: The local graph is not merely a simple collection of nodes and edges; it also visually represents the spatial structure between the target point and its neighbors. Through edge connections, the relative positions of each point in space can be clearly seen. For example, if two neighboring nodes are also close together, they may be connected by an edge in the local graph, forming a more complex but more accurate graphical structure reflecting spatial relationships. This representation of spatial structure is crucial for capturing the geometric features of gestures, i.e., according to the formula... In the formula, It is an activation function. yes The inverse square root, It is the convolution kernel. For example, the bending and stretching of the hand will cause changes in the spatial distribution of the point cloud, and the local map can record and reflect these changes very well.

[0068] Local Feature Integration: The local graph integrates information about the target point and its surrounding neighbors. Each node carries its own attribute information, such as coordinates and normal vectors, while edges convey information about the relationships between nodes. The way edges connect determines the association between nodes, and the weight of the edges quantifies the tightness of this relationship. The local graph presents the spatial structure of the target point and its neighbors. Through the connection of edges, the relative spatial positions of each point can be seen. This spatial structure can capture the geometric features of gestures; for example, changes in the spatial distribution of the point cloud caused by hand movements can be reflected in the local graph. (Center point of the hand surface point cloud) and distance point nearest point It forms a local graph structure with undirected edges. The characteristics can be expressed as a formula In the formula, i , j These represent the row and column indices of the matrix, corresponding to different nodes in the graph; , These are the hyperparameters that need to be learned in the graph structure formed by the point cloud on the hand surface. These are values ​​from the KNNGraph. In this way, the local graph can effectively organize and represent various features of the hand surface point cloud within a local region. In subsequent deep learning model processing, these integrated local features can be more easily extracted and utilized, helping to improve the accuracy and efficiency of gesture recognition. For example, when classifying gestures, the model can determine the category of a gesture by analyzing the features of nodes and edges in the local graph; different gestures such as clenching a fist and extending fingers will exhibit different feature patterns in the local graph.

[0069] Through the above steps, a complete local graph can be constructed based on the hand surface point cloud data and the KNN Graph algorithm. By adjusting the K value (i.e., the parameter for determining the number of neighbor nodes), different local features can be obtained. Using the relevant matrix in graph theory (adjacency matrix, degree matrix, etc.), convolution operations are performed on the graph structure to update the features, where the activation function, convolution kernel, and other parameters participate in feature extraction, and different K values affect the feature extraction effect. An effective data representation form is provided for subsequent gesture recognition tasks, which helps to more deeply mine the features of gestures and improve the performance of recognition.

[0070] S3, the neighborhood structure around the center point in the hand surface point cloud:

[0071] a) Single-scale edge convolution to extract local features: The hand surface point cloud data is a low-resolution sampling of the three-dimensional physical world of the hand surface, with irregular distribution and different point densities in different regions. In this case, single-scale edge convolution is used to preliminarily extract local features. It is assumed that the center point of the feature is represented as , the features of the neighborhood points are represented as , and the aggregated feature of the center point under single-scale edge convolution is calculated as follows:

[0072] ;

[0073] wherein and are learnable weight matrices, used to describe the relationship between the center point and the neighborhood points, reflecting the influence of its own features; is an element-wise multiplication; b is a bias term; is the neighborhood point set of the center point . Through this formula, the model can calculate an aggregated feature based on the feature information of the neighborhood points and the center point itself, and preliminarily extract local features. However, the features obtained by single-scale edge convolution are limited and cannot fully and deeply reflect the complex features of the hand surface point cloud.

[0074] ​​b) Multi-scale extraction method to obtain more abundant local features: In order to obtain the local features of the hand surface point cloud in a more in-depth and multi-level manner, a multi-scale extraction method is constructed. This method uses three edge convolutions of different scales to extract the local neighborhood features of each center point. In point cloud data, the connection relationship between points can be regarded as the edge in the graph structure, and multi-scale analysis is to observe the local neighborhood features represented by these edges at different scales. At different scales, the neighborhood point set is different, and the extracted features also have different emphases. At a smaller scale, it focuses on capturing the detailed features of the hand surface point cloud, such as the subtle shape changes at the finger joints; at a larger scale, it pays more attention to the overall geometric structure features, such as the general shape of the palm and the relative position relationship of the fingers. Then, the local features extracted at different scale regions are spliced together. The center point Aggregated features under multi-scale edge convolution The calculation formula is:

[0075] ;

[0076] Here S represents the number of scales, which is 3 in the present application; is the neighborhood point set of the center point at scale s; and are the weight matrices for the relationship between the center point and the neighborhood points and the features of the center point at the corresponding scale.

[0077] Through this multi-scale extraction and splicing feature method, the local neighborhood features represented by the edges in the graph structure can be observed at different scales. The neighborhood point set is different at different scales, and the extracted features also have different emphases. Small scales can capture detailed features such as subtle shape changes at finger joints, and large scales pay more attention to overall geometric structure features such as the general shape of the palm and the relative position relationship of the fingers. Splicing different scale features can greatly enrich the dimension and information content of the features, improve the model's processing ability for complex geometric structures, and enhance the accuracy and robustness of gesture recognition.

[0078] S4, after splicing the local features, the trained classifier is used to analyze and predict the features, so as to realize accurate recognition of gestures. The specific process includes the following steps:

[0079] Feature classifier training: Using a labeled dataset containing multiple gestures, the concatenated multi-scale local features are used as input, and the corresponding gesture class is used as the output label to train a suitable classifier such as a support vector machine (SVM), a multi-layer perceptron (MLP), or a convolutional neural network (CNN). During the training process, the classifier learns the mapping relationship between the local feature patterns of different gestures and the gesture class. For example, for a fist gesture, its local features will exhibit specific combination patterns at different scales. By learning a large number of fist gesture samples, the classifier remembers the correspondence between these feature patterns and the fist gesture class.

[0080] Feature input and prediction: For the gesture to be recognized, the same data collection, point cloud aggregation, and multi-scale feature extraction steps are performed to obtain the concatenated multi-scale local features. These features are input into the trained classifier, which analyzes and judges the input features based on the mapping relationship learned during the training phase, and predicts which gesture class the gesture belongs to. For example, if the input features have a high degree of matching with the feature patterns of the fist gesture in the training set, the classifier predicts that the gesture is a fist gesture. For example Figure 2 In deep learning processing of gesture recognition tasks, the collected hand point cloud data is processed by "hidden layer + Offset". The hidden layer is the hidden layer in the neural network, which is responsible for feature extraction and transformation of the input data. Adding Offset (offset) is to adjust the calculation of neurons in the hidden layer, such as in graph convolution operations, by introducing an offset to optimize feature extraction and representation, so that the model can better learn the feature patterns in the hand point cloud data, thereby improving the accuracy of gesture recognition and preparing for subsequent input to the input layer of the neural network.

[0081] Result evaluation and optimization: The prediction results of the classifier are evaluated, and accuracy, recall rate, F1 value, and other indicators are used to measure the accuracy of recognition. If the recognition result is not satisfactory, the performance of gesture recognition can be improved by adjusting the parameters of the classifier, increasing the amount of training data, optimizing the feature extraction process, etc. For example, if it is found that the recognition accuracy for some gestures is low, further analysis of the features of these gestures can be performed to determine whether there are problems such as insufficient feature extraction or the classifier is not sensitive to certain features, and then targeted improvements can be made.

[0082] The application directly processes unstructured point cloud data without conversion, reduces information loss, improves processing efficiency and recognition accuracy, and proposes a multi-scale feature extraction and sparsity processing method to enhance the adaptability and processing capacity of the model to complex data and improve the performance of the model. It is suitable for intelligent rehabilitation systems in Internet of Things environment and can process data with different sparsity and quality. The application realizes non-contact gesture recognition, improves the comfort and safety of the rehabilitation process, and avoids discomfort and safety hazards caused by physical contact. Through the innovative point cloud processing method, the influence of light, occlusion and view angle change is effectively overcome, the subtle differences of complex gestures are accurately distinguished, and the gesture recognition accuracy and robustness are improved.

[0083] Based on the same inventive concept, the application also provides a gesture recognition system, comprising:

[0084] A data acquisition module is configured to acquire hand surface point cloud data of a gesture of an individual to be recognized, and perform point cloud sparsification processing on the hand surface point cloud data to generate variable-density hand surface point cloud.

[0085] A point cloud aggregation module is configured to select a target point in the variable-density hand surface point cloud as a center node, select a plurality of nodes closest to the center node as neighbor nodes, establish an undirected edge between the center node and each neighbor node, and construct a local graph structure with the Euclidean distance between each node as the weight of the edge. By changing the geometric features of the hand, the change features of the connection relationship between the center node and its neighbor nodes in the local graph structure are obtained. The feature information of the center node and its neighbor nodes around the center node is integrated to obtain local features. Based on the local graph structure, the number of neighbor nodes around the center node is adjusted, and convolution operation is used to extract local features at different scales. The local features extracted at different scales are spliced to obtain multi-scale local feature representation.

[0086] A gesture recognition module is configured to recognize the gesture of the individual to be recognized based on the local feature representation.

[0087] The application also provides a gesture recognition computer device, which comprises a memory, a processor and a computer program stored in the memory, and the processor executes the computer program to realize the steps of the gesture recognition method.

[0088] The application also provides a readable storage medium, which stores a computer program, and the computer program comprises program instructions, which are executed by a processor to perform the steps of the gesture recognition method.

[0089] The above merely describes preferred specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art, according to the technical solution and inventive concept of the present application, makes equivalent replacement or change within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A gesture recognition method, characterized by, The method comprises the following steps: Collecting hand surface point cloud data of a gesture of an individual to be identified, and performing point cloud sparsification processing on the hand surface point cloud data to generate a variable-density hand surface point cloud; the point cloud sparsification processing is performed by processing different sparsity regions, specifically including appropriately increasing the density of points in sparse regions to supplement key information, and reasonably reducing the number of points in dense regions to remove redundant information, to generate the variable-density hand surface point cloud; Selecting a center point in the variable-density hand surface point cloud as a center node, selecting a plurality of nodes closest to the center node as neighbor nodes, establishing an undirected edge between the center node and each neighbor node, and constructing a local graph structure with the Euclidean distance between each node as the weight of the edge; the local graph structure presents the spatial structure of the center node and its neighbor points, and the relative spatial positional relationship of each point can be seen through the connection of the edges, and this spatial structure can capture the geometric features of the gesture; By changing the geometric characteristics of the hand, the change characteristics of the connection relationship between the center node and its neighbor nodes in the local graph structure are obtained; and the feature information of the center node and its neighbor nodes around it is integrated to obtain local features; based on the local graph structure, by adjusting the number of neighbor nodes around the center node, local features under different scales are extracted; the local features extracted under different scales are spliced to obtain multi-scale local feature representation; the change characteristics of the connection relationship between the center node and its neighbor nodes in the local graph structure are obtained by performing convolution operation on the adjacency matrix, the degree matrix and the feature matrix on the local graph structure, and the change characteristics Specifically represented as: ; wherein, is an activation function, is the square root of the inverse of the degree matrix , and is a convolution kernel. is a feature matrix; the degree matrix is represented as: ; wherein, is the sum of the weights of the edges connected to the point ; the feature matrix is represented as: ; wherein, is the i-th column of the feature matrix ; the adjacency matrix is a matrix, which is represented as: , n represents the number of points in the hand surface point cloud; According to the local feature representation, the gesture of the individual to be identified is recognized.

2. The gesture recognition method of claim 1, wherein, After generating the variable-density hand surface point cloud, Fourier transform is performed on the variable-density hand surface point cloud in the graph structure to convert the point cloud data from the spatial domain to the frequency domain, and the Fourier transform is specifically represented as: ； in, It is the result of the Fourier transform. It is the set of feature values ​​associated with graph nodes. Corresponding frequency in the frequency domain It is the node's One characteristic, It is the first The th eigenvector of the th feature vector One component; n This indicates the number of points in the point cloud on the surface of the hand. m It is the index variable for the summation operation; yes In eigenvalues The corresponding frequency domain representation; N This represents a parameter related to the sampling interval of the frequency; the transformation result is converted into matrix form using matrix multiplication, and is expressed as: ； wherein, is the feature vector of the i-th node on the graph. is the feature vector of the i-th node on the graph.

3. The gesture recognition method of claim 2, wherein, The KNN Graph algorithm is used to find K points closest to the center node as neighbor nodes in the local graph structure.

4. The gesture recognition method of claim 1, wherein, The multi-scale local feature representation is obtained by using a plurality of edge convolutions of different scales to extract local features of each center node at different scales and splicing them, and the generation process is specifically represented as: ； wherein, S the number of representative scales, is a center node a set of neighborhood nodes at a scale s ; and is a weight matrix for the relationship between the center node and the neighborhood nodes and the self-characteristics at the corresponding scale, is an element-wise multiplication; b is a bias term, denotes a feature representation of the center node, denotes a feature representation of the neighborhood nodes.

5. A gesture recognition system, characterized by It comprises: A data acquisition module is configured to collect hand surface point cloud data of a gesture of an individual to be identified, and perform point cloud sparsification processing on the hand surface point cloud data to generate a variable-density hand surface point cloud; the point cloud sparsification processing is performed by processing different sparsity regions, specifically including appropriately increasing the density of points in sparse regions to supplement key information, and reasonably reducing the number of points in dense regions to remove redundant information, to generate the variable-density hand surface point cloud; A graph structure construction module is configured to select a center point in the variable-density hand surface point cloud as a center node, select a plurality of nodes closest to the center node as neighbor nodes, establish an undirected edge between the center node and each neighbor node, and construct a local graph structure with the Euclidean distance between each node as the weight of the edge; the local graph structure presents the spatial structure of the center node and its neighbor points, and the relative spatial positional relationship of each point can be seen through the connection of the edges, and this spatial structure can capture the geometric features of the gesture; The feature extraction module is used to obtain the changing features of the connection relationship between the central node and its neighboring nodes in the local graph structure by changing the geometric features of the hand; and to integrate the feature information of the central node and its surrounding neighboring nodes to obtain local features; based on the local graph structure, by adjusting the number of neighboring nodes around the central node, local features at different scales are extracted; and the local features extracted at different scales are concatenated to obtain a multi-scale local feature representation; the changing features of the connection relationship between the central node and its neighboring nodes in the local graph structure are obtained by performing convolution operations on the adjacency matrix, degree matrix, and feature matrix on the local graph structure. Specifically, it is expressed as follows: ;in, It is an activation function. It is a degree matrix The inverse square root, It is a convolution kernel; The feature matrix; the degree matrix is ​​represented as: ;in, Is and point The sum of the weights of the connected edges; the feature matrix is ​​represented as: ;in, It is the first characteristic matrix Columns; Adjacency Matrix It is The matrix is ​​represented as: , n represents the number of points in the hand surface point cloud; A gesture recognition module is configured to recognize the gesture of the individual to be identified according to the local feature representation.

6. A gesture recognizing computer device, characterized by It comprises: A memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the gesture recognition method of any one of claims 1-4.

7. A readable storage medium, characterized by, The readable storage medium stores a computer program, and the computer program comprises program instructions, which are executed by a processor to perform the steps of the gesture recognition method of any one of claims 1-4.

Images