A gesture recognition method and system based on depth camera and contour extraction

Abstract

The application belongs to the technical field of optical information processing and computer vision, and particularly relates to a gesture recognition method and system based on a depth camera and contour extraction, which comprises the following steps: acquiring gesture point cloud data; using a gesture point cloud data processing algorithm to pre-process the gesture point cloud data; performing principal component analysis and contour extraction on the pre-processed gesture point cloud data to obtain compressed gesture point cloud data; dividing the processed gesture point cloud data into multiple data sets, training and testing the data sets by using a gesture point cloud data recognition algorithm, training a gesture recognition model, wherein the input of the gesture recognition model is gesture point cloud data, and the output is category information of a gesture, and testing to obtain a gesture recognition result. The application greatly improves the processing speed of the gesture point cloud data recognition algorithm, and has great application prospects in the fields of mobile devices, smart home, intelligent control and the like.

Description

Technical Field

[0001] This invention belongs to the fields of optical information processing and computer vision technology, and specifically relates to a gesture recognition method and system based on depth camera and contour extraction. Background Technology

[0002] Gesture recognition technology has wide applications in human-computer interaction, smart homes, robot control, and communication for the deaf and mute. Practical gesture recognition algorithms typically possess characteristics such as wide applicability, robustness, fast processing speed, and strong scalability. Currently, most recognition algorithms use grayscale or color images as raw data; therefore, the robustness of gesture recognition is significantly affected by changes in ambient lighting or complete darkness. Depth cameras, as active measurement devices, are less affected by changes in ambient lighting and can operate normally in dark environments. In conclusion, depth cameras have great application potential as data acquisition devices.

[0003] Gesture point cloud data acquired directly by depth cameras typically contains background, noise, and outliers, requiring the removal of this redundant information. Furthermore, the large amount of data in the raw gesture point cloud data results in slow computation speeds for existing methods while maintaining recognition accuracy, hindering the widespread adoption of these methods. Summary of the Invention

[0004] The purpose of this invention is to propose a gesture recognition method and system based on depth camera and contour extraction, so as to achieve accurate gesture recognition in complex lighting environments and dark conditions, and greatly improve the algorithm processing speed while ensuring high recognition accuracy.

[0005] The technical solution adopted in this invention is as follows:

[0006] A gesture recognition method based on depth camera and contour extraction includes:

[0007] Acquire gesture point cloud data;

[0008] Gesture point cloud data is preprocessed using a gesture point cloud data processing algorithm;

[0009] Gesture point cloud data compression involves performing principal component analysis and contour extraction on the preprocessed gesture point cloud data to obtain compressed gesture point cloud data.

[0010] The processed gesture point cloud data is divided into multiple datasets. The gesture point cloud data recognition algorithm is used to train and test the above datasets to obtain a gesture recognition model. The input of the gesture recognition model is the gesture point cloud data, and the output is the gesture category information. The gesture recognition result is obtained by testing.

[0011] As a further improvement of the present invention, the acquisition of gesture point cloud data includes:

[0012] Based on the gesture recognition task, provide the gesture to be recognized;

[0013] Use a depth camera to collect point cloud data of the gesture to be recognized;

[0014] Repeatedly collect and save all types of gesture point cloud data.

[0015] As a further improvement of the present invention, the depth camera is a time-of-flight depth camera or a structured light depth camera.

[0016] As a further improvement of the present invention, the preprocessing of gesture point cloud data using a gesture point cloud data processing algorithm includes:

[0017] Organize and classify the collected gesture point cloud data of different types;

[0018] The gesture point cloud data is cropped: a depth threshold is set, and background and redundant gesture point cloud data information are cropped according to the depth threshold to obtain gesture point cloud data containing only key gestures.

[0019] The cropped gesture point cloud data is filtered: two parameters are set for the number of nearest neighbors and the standard deviation ratio of the gesture point cloud data. The preprocessing is completed by filtering based on the number of nearest neighbors and the standard deviation ratio.

[0020] As a further improvement of the present invention, the step of performing principal component analysis and contour extraction on the preprocessed gesture point cloud data, i.e., compressing the gesture point cloud data to obtain compressed gesture point cloud data, includes:

[0021] The first step in the gesture point cloud data compression process is as follows: The gesture point cloud data contains three dimensions, represented by (x, y, z) in a three-dimensional Cartesian coordinate system. Principal component analysis is performed in two steps: The first step is to reduce the dimensionality of the gesture point cloud data to obtain two-dimensional gesture point cloud data and principal component vectors; the second step is to use the principal component vectors to convert the two-dimensional gesture point cloud data into three-dimensional planar gesture point cloud data.

[0022] Step 2 of gesture point cloud data compression: Extract the contour of the gesture point cloud data based on the 3D planar gesture point cloud data.

[0023] As a further improvement of the present invention, the processed gesture point cloud data is divided into multiple datasets, and the above datasets are trained and tested using a gesture point cloud data recognition algorithm to obtain a gesture recognition model. The input of the gesture recognition model is the gesture point cloud data, and the output is the gesture category information. The test obtains the gesture recognition result, including:

[0024] The processed gesture point cloud data is divided into multiple datasets using the following method:

[0025] Based on the gesture category and sample size, all gesture point cloud data are divided into training set, validation set and test set;

[0026] The method for training and validating the gesture recognition model is as follows:

[0027] The training dataset and validation dataset are input into the gesture point cloud data recognition algorithm to train and obtain the gesture recognition model;

[0028] The method for testing the gesture point cloud data recognition algorithm is as follows:

[0029] The test set is input into various pre-trained gesture recognition models, and the different models are used to recognize the gestures to obtain the gesture recognition results.

[0030] As a further improvement of the present invention, the gesture point cloud data recognition algorithm includes a support vector machine machine learning algorithm, a decision tree machine learning algorithm, and a deep learning algorithm.

[0031] A gesture recognition system based on depth camera and contour extraction includes:

[0032] The acquisition module is used to acquire gesture point cloud data;

[0033] The preprocessing module is used to preprocess the gesture point cloud data using gesture point cloud data processing algorithms;

[0034] The data compression module is used to perform principal component analysis and contour extraction on the preprocessed gesture point cloud data to obtain the processed gesture point cloud data.

[0035] The recognition module is used to divide the processed gesture point cloud data into multiple datasets, and use the gesture point cloud data recognition algorithm to train and test the above datasets to obtain a gesture recognition model. The input of the gesture recognition model is the gesture point cloud data, the output is the gesture category information, and the test obtains the gesture recognition result.

[0036] A gesture recognition device based on a depth camera and contour extraction, comprising:

[0037] memory,

[0038] processor,

[0039] The processor is configured to execute the gesture recognition method based on depth camera and contour extraction.

[0040] A computer-readable storage medium, characterized in that, when the instructions in the storage medium are executed by a processor, the processor is able to execute the gesture recognition method based on a depth camera and contour extraction.

[0041] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows:

[0042] This invention employs preprocessing methods such as gesture point cloud data cropping and filtering to remove background and noise from the data. Principal component analysis and contour extraction are then used to further compress the gesture point cloud data, significantly reducing its size. The gesture point cloud data recognition algorithm used is highly scalable, enabling rapid and accurate recognition. The gesture recognition algorithm based on depth cameras and contour extraction proposed in this invention not only has a wider range of applicability but also greatly improves processing speed while maintaining high recognition accuracy. This invention can recognize various types of gestures and has significant application prospects in fields such as human-computer interaction, smart homes, and robot control.

[0043] Furthermore, this invention uses a depth camera as a data acquisition device, which is not easily affected by changes in ambient light and can work in dark environments. Attached Figure Description

[0044] Figure 1 This is one embodiment of the present invention. 101 The gesture is the object of data acquisition; 102 The gesture example is the shape example of the number 5; 103 The depth camera is the data acquisition device; 104 The data acquisition process; 105 The gesture point cloud data is a type of data acquired by the camera; 106 The gesture point cloud data example is the gesture point cloud data map of the number 5; 107 The data preprocessing process; 108 The gesture point cloud data example of the number 5 after preprocessing; 109 The subsequent data processing process.

[0045] Figure 2 This is a flowchart of the present invention, which specifically involves using a depth camera to collect gesture point cloud data, performing preprocessing operations such as cropping and filtering on the gesture point cloud data, performing principal component analysis and contour extraction on the preprocessed gesture point cloud data, and finally using a gesture point cloud data recognition algorithm to classify gestures.

[0046] Figure 3 This is a flowchart of the gesture point cloud data preprocessing of the present invention, which specifically includes steps such as gesture point cloud data cropping, gesture point cloud data filtering, gesture point cloud data principal component analysis, and gesture point cloud data contour extraction.

[0047] Figure 4This is a schematic diagram of one embodiment of the present invention. 401 is the raw gesture point cloud data acquired by the depth camera, 402 is an example of gesture point cloud data cropping, 403 is an example of gesture point cloud data filtering, 404 is an example of principal component analysis of gesture point cloud data, and 405 is an example of contour extraction.

[0048] Figure 5 This is a flowchart of the gesture point cloud data processing algorithm of the present invention. Specifically, the algorithm uses a gesture point cloud data recognition algorithm to train the processed gesture point cloud data, identify the type of gesture, and provide the recognition accuracy.

[0049] Figure 6 This is a gesture recognition confusion matrix diagram of the present invention, where the recognized gesture categories are the numbers 0-9. (a)-(g) are confusion matrix diagrams of the recognition results of the models PointNet, PointNet++ (SSG), PointNet++ (MSG), PointConv, PointCNN, DGCNN and PCT, respectively, from which the recognition accuracy of different categories of gestures can be analyzed.

[0050] Figure 7 The above are bar charts showing the gesture recognition accuracy of this invention. (a) is a bar chart showing the gesture recognition accuracy without gesture point cloud data compression, and (b) is a bar chart showing the gesture recognition accuracy after principal component analysis and contour extraction of the gesture point cloud. From these, we can obtain... Figure 6 Average accuracy of gesture recognition for different models.

[0051] Figure 8 This is a schematic diagram of the structure of a gesture recognition system based on a depth camera and contour extraction according to the present invention;

[0052] Figure 9 This is a schematic diagram of an electronic device structure according to the present invention. Detailed Implementation

[0053] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0054] Machine learning, as an important research area in computer vision, has significant applications in image classification, image segmentation, object detection, and object tracking. Machine learning and deep learning algorithms can assist in gesture recognition, enabling gesture recognition tasks with large sample sizes and high accuracy.

[0055] The fundamental objective of this invention is to use a depth camera as a data acquisition device to collect gesture point cloud data. Combining this data with processing methods such as cropping, filtering, principal component analysis, and contour extraction, a gesture point cloud data recognition algorithm is further employed to achieve accurate gesture recognition. The gesture point cloud data processing methods, including cropping, filtering, principal component analysis, and contour extraction, as well as the gesture point cloud data recognition algorithm used in this invention, can achieve gesture recognition in dark environments and under complex lighting conditions. The introduced contour extraction algorithm significantly reduces the amount of data, greatly improving the speed of gesture recognition while maintaining accuracy.

[0056] Specifically, this invention relates to using a depth camera to acquire gesture point cloud data; using a gesture point cloud data preprocessing algorithm to perform processing on the gesture point cloud data, such as cropping, filtering, principal component analysis, and contour extraction; and using a gesture point cloud data recognition algorithm to identify the gesture type.

[0057] The method proposed in this invention comprises four parts: gesture point cloud data acquisition, gesture point cloud data preprocessing, gesture point cloud data processing, and gesture point cloud data recognition. Specifically, it includes:

[0058] Acquire gesture point cloud data;

[0059] Gesture point cloud data is preprocessed using a gesture point cloud data processing algorithm;

[0060] Gesture point cloud data compression involves performing principal component analysis and contour extraction on the preprocessed gesture point cloud data to obtain compressed gesture point cloud data.

[0061] The processed gesture point cloud data is divided into multiple datasets. The gesture point cloud data recognition algorithm is used to train and test the above datasets to obtain a gesture recognition model. The input of the gesture recognition model is the gesture point cloud data, and the output is the gesture category information. The gesture recognition result is obtained by testing.

[0062] The descriptions of each part are as follows:

[0063] First, gesture point cloud data is acquired using a depth camera. A depth camera is an active measurement device whose core function is to acquire depth information of an object, thereby obtaining gesture point cloud data. The depth cameras used in this invention include, but are not limited to, time-of-flight depth cameras and structured light depth cameras.

[0064] Secondly, gesture point cloud data processing algorithms are used to preprocess the gesture point cloud data acquired by the depth camera to remove redundant information such as background and noise. The algorithms used in gesture point cloud data preprocessing include, but are not limited to, gesture point cloud data cropping and gesture point cloud data filtering algorithms.

[0065] Secondly, gesture point cloud data processing involves further compressing the gesture point cloud data based on preprocessing to reduce its dimensionality and volume. The algorithms used in gesture point cloud data compression include, but are not limited to, principal component analysis, contour extraction, and downsampling.

[0066] Finally, gesture recognition algorithms are used to obtain the gesture recognition results and accuracy. These algorithms include, but are not limited to, machine learning algorithms such as support vector machines and decision trees for classification tasks, as well as deep learning algorithms for gesture point cloud data classification, such as PointNet and PointNet++ deep learning networks.

[0067] Among them, gesture recognition methods based on depth cameras and contour extraction include:

[0068] Gestures: The object of data collection; as the object of gesture recognition, different gesture shapes are provided for data collection;

[0069] Depth camera: a gesture point cloud data acquisition device; it uses the principle of active measurement to obtain the depth information of an object and collect the gesture point cloud data of the gesture object;

[0070] Gesture point cloud data acquisition: Use a depth camera to acquire gesture point cloud data containing gesture information; record various types of gesture information captured by the depth camera as gesture point cloud data;

[0071] Gesture point cloud data processing: Preprocessing operations such as cropping and filtering are performed on the gesture point cloud data to remove background and noise. Then, principal component analysis and contour extraction are performed on the preprocessed data to compress the gesture point cloud data and further reduce the amount of gesture point cloud data. The collected gesture point cloud data is processed, including preprocessing and subsequent gesture point cloud data compression.

[0072] Gesture recognition: The processed gesture point cloud data is recognized using a gesture point cloud data recognition algorithm.

[0073] The depth cameras used include, but are not limited to, time-of-flight depth cameras and structured light depth cameras.

[0074] The following will provide a more detailed explanation of each step:

[0075] 1) This gesture acquisition mainly involves using a depth camera to collect a series of gesture point cloud data to be recognized. The working steps are as follows:

[0076] Step 1: Based on the gesture recognition task, provide a specific type of gesture to be recognized;

[0077] Step 2: Use a depth camera to collect the corresponding gesture point cloud data;

[0078] Step 3: Repeat steps 1 and 2 to collect and save all types of gesture point cloud data.

[0079] The above steps enable the implementation of a gesture point cloud data acquisition system based on a depth camera.

[0080] 2) A method for processing gesture point cloud data; this processing system is located after the gesture point cloud data acquisition step, performing preprocessing such as cropping and filtering on the acquired gesture point cloud data; and performing gesture point cloud data compression operations such as principal component analysis and contour extraction on the preprocessed gesture point cloud data. The specific working steps are as follows:

[0081] Step 1: Organize and classify the saved series of gesture point cloud data according to categories;

[0082] Step 2: Crop the gesture point cloud data;

[0083] Use gesture point cloud data processing software or programming to crop the gesture point cloud data. Set an appropriate depth threshold, then crop away the background and redundant gesture point cloud data information to finally obtain gesture point cloud data containing only the key gestures.

[0084] Step 3: Filter the cropped gesture point cloud data;

[0085] Filtering gesture point cloud data can remove noise and reduce data volume. Algorithms that can be used include, but are not limited to, Statistical Outlier Removal (SOR) filtering, pass-through filtering, voxel filtering, statistical filtering, conditional filtering, and bilateral filtering.

[0086] Step 4: Compression of gesture point cloud data. Principal component analysis and contour extraction are performed on the cropped and filtered gesture point cloud data.

[0087] The original gesture point cloud data contains three dimensions, represented by (x, y, z) in a three-dimensional Cartesian coordinate system. Principal component analysis (PCA) is performed in two steps: first, the dimensionality of the gesture point cloud data is reduced to two-dimensional gesture point cloud data and principal component vectors; second, the principal component vectors are used to convert the two-dimensional gesture point cloud data into three-dimensional planar gesture point cloud data. Contour extraction is performed on the three-dimensional planar gesture point cloud data to extract its contours. The combination of PCA and contour extraction significantly reduces the amount of gesture point cloud data. Algorithms for gesture point cloud data compression include, but are not limited to, PCA and contour extraction algorithms.

[0088] The above steps enable the preprocessing and compression of gesture point cloud data.

[0089] Specifically, the present invention preprocesses the raw gesture point cloud data, and the data preprocessing methods used include, but are not limited to, cropping and filtering.

[0090] Gesture point cloud data compression reduces the amount of gesture point cloud data. Methods for reducing the amount of gesture point cloud data include, but are not limited to, principal component analysis and contour extraction.

[0091] 3) Gesture point cloud data-based recognition method: Different types of gesture point cloud data recognition algorithms are used to train, verify, and test the processed gesture point cloud data to achieve rapid and accurate gesture type recognition. This system uses gesture point cloud data recognition algorithms to train the processed gesture point cloud data, enabling rapid and accurate gesture recognition. Its working steps are as follows:

[0092] Step 1: Divide the data processed by the gesture point cloud data processing system into multiple datasets;

[0093] Based on the gesture category and sample size, all gesture point cloud data are divided into training set, validation set and test set;

[0094] Step 2: Train and validate the gesture recognition model. The method is as follows:

[0095] The training and validation datasets are input into the gesture recognition model to train and select the gesture recognition model with the best recognition performance.

[0096] One part of the dataset is used for training and validating the gesture point cloud data recognition algorithm, and the other part of the dataset is used for testing the algorithm.

[0097] The input to the gesture point cloud data recognition algorithm is gesture point cloud data, and the output is the gesture category information.

[0098] Step 3: Test the gesture point cloud data recognition algorithm. The method is as follows:

[0099] The test data is input into the trained gesture recognition model, and the recognition accuracy and other results of different models are further analyzed.

[0100] Step 4: Use gesture point cloud data recognition algorithms to obtain the accuracy of gesture recognition;

[0101] The above steps enable a gesture recognition system based on gesture point cloud data.

[0102] Gesture point cloud data recognition algorithms include, but are not limited to, machine learning algorithms such as support vector machines and decision trees that perform classification tasks, as well as deep learning algorithms for gesture point cloud data classification, such as PointNet, PointNet++ and other deep learning networks.

[0103] Gesture recognition is performed using different types of gesture point cloud data recognition algorithms, including but not limited to machine learning algorithms such as support vector machines and decision trees for classification tasks, as well as deep learning algorithms for gesture point cloud data classification, such as PointNet and PointNet++.

[0104] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0105] Figure 1 This is one embodiment of the present invention. The processing steps are as follows:

[0106] Step 1: Depth camera 103 sequentially captures various hand gesture shapes shown in 101, and 102 is an example of the hand gesture shape for the number 5;

[0107] Step 2: Data Acquisition Process 104 Acquires a series of gesture point cloud data through depth camera 103; 106 is an example of gesture point cloud data for the number 5.

[0108] Step 3: The gesture point cloud data 105 undergoes data preprocessing process 107 to obtain the preprocessed gesture point cloud data example shown in 108;

[0109] Step 4: All preprocessed gesture point cloud data are transmitted to data processing 109.

[0110] Figure 3 This is a detailed flowchart of one embodiment of the present invention, involving the processing procedures and... Figure 4 Corresponding to the example. Combined Figure 1 The preprocessing steps for data 107 and data processing for data 109 are as follows:

[0111] Step 1: Figure 1 The 107 data preprocessing steps involve cropping the gesture point cloud data.

[0112] The specific method involves cropping the gesture point cloud data to remove background and invalid gesture point cloud data. The cropping algorithm works by setting an appropriate depth threshold to remove redundant information outside the gesture point cloud data. In this embodiment, the gesture point cloud data cropping algorithm used includes, but is not limited to, depth threshold cropping algorithms. Figure 4 The data in section 402 is the cropped point cloud data of the gesture.

[0113] Step Two: Figure 1 The 107 data preprocessing steps involve filtering the gesture point cloud data.

[0114] The specific method is as follows: while preserving the structural information of the gesture point cloud data to the maximum extent, the influence of noise and outliers is eliminated. For gesture point cloud data, this invention uses the SOR filtering method, setting two parameters: the number of nearest neighbors and the standard deviation ratio of the gesture point cloud data. The filtering result is as follows: Figure 4 As shown in Figure 403. The gesture point cloud data filtering algorithms used in this embodiment include, but are not limited to, SOR filtering, pass-through filtering, voxel filtering, statistical filtering, conditional filtering, bilateral filtering, and other filtering algorithms.

[0115] Step 3: Compression of gesture point cloud data. Figure 1 The data processing in the 109 data set performs principal component analysis and contour extraction on the gesture point cloud data.

[0116] The specific method is as follows: Principal component analysis and contour extraction are performed on the gesture point cloud data to reduce the data volume. In this embodiment, principal component analysis is first used to reduce the 3D gesture point cloud data to 2D, and then principal component vector analysis is used to increase the dimensionality of the 2D gesture point cloud data to obtain 3D planar gesture point cloud data. The result is as follows. Figure 4 As shown in Figure 404, contour extraction is performed on the planar gesture point cloud data to obtain the following... Figure 4 The results are shown in Figure 405. The gesture point cloud data compression algorithm used in this embodiment includes, but is not limited to, an algorithm combining principal component analysis and contour extraction.

[0117] according to Figure 5 The detailed flowchart shown illustrates how the processed gesture point cloud data is divided into multiple datasets. A gesture point cloud data recognition algorithm is then used to train and test these datasets to obtain gesture recognition results and accuracy. Figure 6 The gesture recognition confusion matrix for the seven models used. Figure 7 To assess the gesture recognition accuracy of the seven models, the specific processing steps are as follows:

[0118] Step 1: Divide the processed gesture point cloud data into multiple datasets;

[0119] according to Figure 5 The flowchart shows that the division ratio of the dataset is determined according to the sample size of each category of gestures. In this invention, all gesture point cloud data are divided into three datasets in a ratio of 8:1:1, namely the training set, the validation set, and the test set.

[0120] Step 2: Train the gesture recognition model;

[0121] according to Figure 5The flowchart illustrates how gesture point cloud data from the training and validation sets are input into a gesture recognition model to train and select the model with the best recognition performance. This invention trained seven different gesture recognition models: PointNet, PointNet++ (SSG), PointNet++ (MSG), PointConv, PointCNN, DGCNN, and PCT.

[0122] Among them, gesture point cloud data recognition algorithms include, but are not limited to, machine learning algorithms such as support vector machines and decision trees that perform classification tasks, as well as deep learning algorithms for gesture point cloud data classification, such as PointNet, PointNet++ and other deep learning networks.

[0123] Step 3: Test the trained gesture recognition model and output the confusion matrix of the gesture point cloud data gesture recognition model;

[0124] according to Figure 5 The flowchart shows how to test seven gesture recognition point cloud data models using a test dataset. Figure 6 The confusion matrix is ​​shown below. Based on the confusion matrix, the classification accuracy of different models for the ten types of gestures can be obtained.

[0125] Step 4: Test the trained gesture recognition model, output and summarize the gesture recognition accuracy of various models;

[0126] like Figure 7 The bar chart shown summarizes the average gesture recognition accuracy of the seven models. (a) is the bar chart without gesture point cloud data compression, and (b) is the bar chart after gesture point cloud principal component analysis and contour extraction. It can be seen from the figure that gesture point cloud data compression does not affect the accuracy of gesture recognition, which reflects the stability of gesture point cloud data compression algorithm.

[0127] like Figure 8 As shown, the present invention also provides a gesture recognition system based on a depth camera and contour extraction, comprising:

[0128] The acquisition module is used to acquire gesture point cloud data;

[0129] The preprocessing module is used to preprocess the gesture point cloud data using gesture point cloud data processing algorithms;

[0130] The data compression module is used to perform principal component analysis and contour extraction on the preprocessed gesture point cloud data to obtain compressed gesture point cloud data.

[0131] The recognition module is used to divide the processed gesture point cloud data into multiple datasets, and use the gesture point cloud data recognition algorithm to train and test the above datasets to obtain a gesture recognition model. The input of the gesture recognition model is the gesture point cloud data, the output is the gesture category information, and the test obtains the gesture recognition result.

[0132] like Figure 9 As shown, another object of the present invention is to provide a gesture recognition device based on a depth camera and contour extraction, comprising:

[0133] memory,

[0134] processor,

[0135] The processor is configured to execute the gesture recognition method based on depth camera and contour extraction.

[0136] The present invention also provides a computer-readable storage medium that, when the instructions in the storage medium are executed by a processor, enables the processor to perform a gesture recognition method based on a depth camera and contour extraction.

[0137] The gesture recognition method based on depth camera and contour extraction includes:

[0138] Acquire gesture point cloud data;

[0139] Gesture point cloud data is preprocessed using a gesture point cloud data processing algorithm;

[0140] Principal component analysis and contour extraction were performed on the preprocessed gesture point cloud data to obtain compressed gesture point cloud data.

[0141] The processed gesture point cloud data is divided into multiple datasets. The gesture point cloud data recognition algorithm is used to train and test the above datasets to obtain a gesture recognition model. The input of the gesture recognition model is the gesture point cloud data, and the output is the gesture category information. The gesture recognition result is obtained by testing.

[0142] In summary, this invention mainly includes using a depth camera to collect different types of gesture point cloud data; processing the collected gesture point cloud data such as cropping, filtering, principal component analysis, and contour extraction; and using a gesture point cloud data recognition algorithm to recognize the processed gesture point cloud data and obtain the classification accuracy of the gestures. This invention proposes using a depth camera as the data acquisition device, which is less affected by changes in ambient light and can achieve accurate gesture recognition in dark environments. The proposed contour extraction algorithm greatly reduces the amount of original gesture point cloud data and improves the processing speed of subsequent gesture point cloud data recognition algorithms. The technical solution of this invention uses gesture point cloud data collected by a depth camera as the original data. The proposed gesture recognition algorithm, while ensuring recognition accuracy, greatly improves the processing speed of gesture point cloud data recognition algorithms by introducing contour extraction, and has significant application prospects in mobile devices, smart homes, and intelligent control.

[0143] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0144] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0145] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0146] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0147] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A gesture recognition method based on depth camera and contour extraction, characterized in that, include: Acquire gesture point cloud data; Gesture point cloud data is preprocessed using a gesture point cloud data processing algorithm; Principal component analysis and contour extraction were performed on the preprocessed gesture point cloud data to obtain compressed gesture point cloud data. The process of performing principal component analysis and contour extraction on the preprocessed gesture point cloud data to obtain processed gesture point cloud data includes: The 3D gesture point cloud data contains three dimensions, represented by (x, y, z) in a 3D Cartesian coordinate system. Principal component analysis is performed in two steps: the first step is to reduce the dimensionality of the 3D gesture point cloud data to obtain 2D gesture point cloud data and principal component vectors; the second step is to use the principal component vectors to convert the 2D gesture point cloud data into 3D planar gesture point cloud data. Based on 3D planar gesture point cloud data, the contour of the gesture point cloud data is extracted; The processed gesture point cloud data is divided into multiple datasets. A gesture point cloud data recognition algorithm is used to train and test these datasets to obtain a gesture recognition model. The input of this model is the gesture point cloud data, and the output is the gesture category information. Testing yields the gesture recognition results. Specifically, this includes: The processed gesture point cloud data is divided into multiple datasets using the following method: Based on the gesture category and sample size, all gesture point cloud data are divided into training set, validation set and test set; The method for training and validating the gesture recognition model is as follows: The training dataset and validation dataset are input into the gesture point cloud data recognition algorithm to train and obtain the gesture recognition model; The acquisition of gesture point cloud data includes: Based on the gesture recognition task, provide the gesture to be recognized; Use a depth camera to collect point cloud data of the gesture to be recognized; Repeatedly collect and save all types of gesture point cloud data; The preprocessing of gesture point cloud data using a gesture point cloud data processing algorithm includes: Organize and classify the collected gesture point cloud data of different types; The gesture point cloud data is cropped: a depth threshold is set, and background and redundant gesture point cloud data information are cropped according to the depth threshold to obtain gesture point cloud data containing only key gestures. The cropped gesture point cloud data is filtered: two parameters are set for the number of nearest neighbors and the standard deviation ratio of the gesture point cloud data. The preprocessing is completed by filtering based on the number of nearest neighbors and the standard deviation ratio.

2. The gesture recognition method based on depth camera and contour extraction according to claim 1, characterized in that, The depth camera is a time-of-flight depth camera or a structured light depth camera.

3. The gesture recognition method based on depth camera and contour extraction according to claim 1, characterized in that, The method for testing the gesture point cloud data recognition algorithm is as follows: The test set is input into various pre-trained gesture recognition models, and the different models are used to recognize the gestures to obtain the gesture recognition results.

4. The gesture recognition method based on depth camera and contour extraction according to claim 1, characterized in that, The gesture point cloud data recognition algorithm includes support vector machine machine learning algorithm, decision tree machine learning algorithm, and deep learning algorithm.

5. A gesture recognition system based on a depth camera and contour extraction, based on the gesture recognition method based on a depth camera and contour extraction as described in any one of claims 1 to 4, characterized in that, include: The acquisition module is used to acquire gesture point cloud data; The preprocessing module is used to preprocess the gesture point cloud data using gesture point cloud data processing algorithms; The data compression module is used to perform principal component analysis and contour extraction on the preprocessed gesture point cloud data to obtain compressed gesture point cloud data. The recognition module is used to divide the processed gesture point cloud data into multiple datasets, and use the gesture point cloud data recognition algorithm to train and test the above datasets to obtain a gesture recognition model. The input of the gesture recognition model is the gesture point cloud data, the output is the gesture category information, and the test obtains the gesture recognition result.

6. A gesture recognition device based on a depth camera and contour extraction, characterized in that, include: memory, processor, The processor is configured to execute the gesture recognition method based on depth camera and contour extraction as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor, the processor is able to perform the gesture recognition method based on depth camera and contour extraction as described in any one of claims 1 to 4.

Images