Gesture recognition method, device, storage medium and equipment based on neural network
By performing binary processing of gesture images and identifying the dual-channel neural network model, the problem of low gesture recognition accuracy in the prior art is solved, and high-accuracy gesture recognition is achieved in complex scenarios.
Patent Information
- Application Number
- CN201910493340.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-06-06
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2039-06-06
AI Technical Summary
In scenarios such as insufficient lighting, occlusion, insufficient resolution, and incorrect posture, existing gesture recognition technology can easily lead to a decrease in the accuracy of gesture recognition, making it difficult to effectively recognize user gestures.
Using a neural network-based gesture recognition method, the original gesture image is binarized, and the gesture feature information is extracted using a dual-channel neural network model, and the European-style distance between the feature information and the positive sample gesture feature information in the database is calculated to determine the gesture type.
It improves the accuracy of gesture recognition, can accurately identify user gestures in complex scenarios, and enhances the robustness of the recognition algorithm.
Smart Images

Figure CN110334605B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image recognition technology. Specifically, the present application relates to a gesture recognition method, device, storage medium and equipment based on a neural network. Background Art
[0002] Gesture recognition is to use certain algorithms to enable computers to recognize human gestures in pictures or shots, and then understand the meaning of the gestures to achieve mutual communication between users and computers. With the development of machine learning and deep learning, gesture recognition has been widely used in games, shopping and other scenarios.
[0003] In the prior art, gesture types are generally obtained by performing corresponding image processing and recognition on gesture images. However, due to different shooting environments, there are often scenes such as insufficient lighting, occlusion, insufficient resolution, incorrect posture, etc. The use of the above prior art is prone to problems such as decreased gesture recognition accuracy, which poses a great challenge to the gesture recognition process. Summary of the invention
[0004] The present application provides a gesture recognition method based on a neural network, a gesture recognition device based on a neural network, a computer-readable storage medium and a computer device to solve the problem of low accuracy of gesture recognition and to improve the accuracy of gesture recognition.
[0005] The present application embodiment first provides a gesture recognition method based on a neural network, comprising:
[0006] Acquire an original gesture image, and perform binarization processing on the original gesture image to obtain a binarized gesture image;
[0007] Inputting the original gesture image and the binarized gesture image into two channels of a neural network model for recognition respectively, to obtain gesture feature information of the original gesture image;
[0008] The Euclidean distance between the gesture feature information and each positive sample gesture feature information in the database is calculated, and the gesture type in the original gesture image is determined according to the Euclidean distance.
[0009] In one embodiment, the step of binarizing the original gesture image to obtain a binarized gesture image includes:
[0010] Divide the original gesture image into several sub-regions;
[0011] The following operations are performed on the pixel window in each sub-region: the gray value of the pixel at the center of the window is used as the threshold, and the gray values of the adjacent pixels are compared with it to obtain the LBP value of the pixel window;
[0012] The original grayscale value of the pixel window is replaced by the LBP value of the pixel window to obtain a binary gesture image corresponding to the original gesture image.
[0013] In one embodiment, before the step of inputting the original gesture image and the binarized gesture image into two channels of a neural network model for recognition, the method further includes:
[0014] A neural network model is established according to the training gesture image; wherein the steps of establishing the neural network model include:
[0015] Obtaining a training gesture image from a preset training image set, performing feature extraction on the training gesture image and its binarized gesture image, respectively obtaining N-dimensional feature vectors corresponding to the training gesture image and the binarized gesture image, and integrating the N-dimensional feature vectors to obtain a 2N-dimensional feature vector;
[0016] The feature vectors are compared based on the 2N-dimensional feature vectors, and the weights of the feature vectors are adjusted using the comparison results of the positive sample gesture images to obtain a neural network model.
[0017] In one embodiment, the step of adjusting the weight of the feature vector using the comparison result of the positive sample gesture image to obtain the neural network model includes:
[0018] The basic network structure of Inception-Resnet-V2 is used to build an initial dual-channel neural network model;
[0019] Extracting the 2N-dimensional feature vector from the positive sample gesture image, inputting the 2N-dimensional feature vector and the gesture type corresponding to the positive sample gesture image into an initial dual-channel neural network model, and obtaining an initial weight value of the 2N-dimensional feature vector;
[0020] All positive sample gesture images and corresponding gesture types in the positive sample gesture image set are used to continuously adjust the initial weight value of each feature vector in the initial dual-channel neural network model. After the weight value of each feature vector is determined, the dual-channel neural network model is obtained.
[0021] In one embodiment, the step of obtaining an N-dimensional feature vector corresponding to the binary gesture image includes:
[0022] Dividing the binary gesture image into N sub-regions;
[0023] Obtaining LBP histograms of the N sub-regions, and performing normalization processing on the LBP histograms;
[0024] The normalized histograms of the N sub-regions are connected to obtain an N-dimensional feature vector corresponding to the binary gesture image.
[0025] In one embodiment, the step of calculating the Euclidean distance between the gesture feature information and each positive sample gesture feature information in the database, and determining the gesture type in the original gesture image according to the Euclidean distance includes:
[0026] Obtain the feature vector of the original gesture image and the feature vector of the same dimension of each positive sample gesture image in the database, and calculate the Euclidean distance between the feature vector of the original gesture image and the feature vector of each positive sample gesture image;
[0027] The confidence between the original gesture image and each positive sample gesture image is obtained according to the Euclidean distance, and the positive sample gesture type corresponding to the highest confidence is output as the gesture type in the original gesture image.
[0028] In one embodiment, before the step of binarizing the original gesture image to obtain a binarized gesture image, the method further includes:
[0029] The original gesture image is denoised using the edge-preserving denoising algorithm.
[0030] Furthermore, the embodiment of the present application also provides a gesture recognition device based on a neural network, comprising:
[0031] A binarization processing module, used to obtain an original gesture image, and perform binarization processing on the original gesture image to obtain a binarized gesture image;
[0032] A recognition module, used to input the original gesture image and the binarized gesture image into two channels of a neural network model for recognition, respectively, to obtain gesture feature information of the original gesture image;
[0033] The gesture type determination module is used to calculate the Euclidean distance between the gesture feature information and each positive sample gesture feature information in the database, and determine the gesture type in the original gesture image according to the Euclidean distance.
[0034] Furthermore, an embodiment of the present application also provides a computer-readable storage medium, which is used to store computer instructions. When the computer-readable storage medium is run on a computer, it enables the computer to execute the steps of the neural network-based gesture recognition method described in any of the above technical solutions.
[0035] Furthermore, an embodiment of the present application further provides a computer device, the computer device comprising:
[0036] one or more processors;
[0037] a storage device for storing one or more programs,
[0038] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the gesture recognition method based on a neural network described in any of the above technical solutions.
[0039] Compared with the prior art, the solution provided by this application has at least the following advantages:
[0040] The gesture recognition method based on a neural network provided in the embodiment of the present application obtains feature information of the original gesture image by inputting the original gesture image and its corresponding binarized gesture image into a neural network model for recognition, and then determines the gesture type in the original gesture image based on the Euclidean distance between the feature information of the original gesture image and the feature information of the positive sample gesture image stored in the database. Since the binarized gesture image can reflect the texture features of the original gesture image, the neural network model of multiple channels extracts the gesture features and texture feature information of the original gesture image, which improves the recognition accuracy of the original gesture image compared with the traditional gesture recognition scheme using a single-channel neural network.
[0041] Additional aspects and advantages of the present application will be partially given in the following description, which will become apparent from the following description, or will be understood through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0043] Figure 1 A diagram of an implementation environment of a gesture recognition method based on a neural network provided in one embodiment of the present application;
[0044] Figure 2 A flowchart of a gesture recognition method based on a neural network provided in one embodiment of the present application;
[0045] Figure 3 A flowchart of performing binarization processing on an original gesture image to obtain a binarized gesture image provided by an embodiment of the present application;
[0046] Figure 4 A flowchart of establishing a neural network model provided for one embodiment of the present application;
[0047] Figure 5 A flowchart of establishing a dual-channel neural network model provided by another embodiment of the present application;
[0048] Figure 6A flowchart for calculating the Euclidean distance between the gesture feature information and each positive sample gesture feature information in a database, and determining the gesture type in the original gesture image according to the Euclidean distance, provided by an embodiment of the present application;
[0049] Figure 7 A schematic diagram of the structure of a gesture recognition device based on a neural network provided in one embodiment of the present application;
[0050] Figure 8 A structural block diagram of a computer device provided in accordance with an embodiment of the present application. DETAILED DESCRIPTION
[0051] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.
[0052] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.
[0053] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first live video image may be referred to as a second live video image, and similarly, a second live video image may be referred to as a first live video image. Both the first live video image and the second live video image are live video images, but they are not the same live video image.
[0054] Figure 1 The diagram is an implementation environment diagram of a gesture recognition method based on a neural network provided in an embodiment. In the implementation environment, a user terminal and a server are included.
[0055] The gesture recognition method based on neural network provided in this embodiment can be executed on the server side, and the execution process is as follows: obtaining an original gesture image, and binarizing the original gesture image to obtain a binarized gesture image; inputting the original gesture image and the binarized gesture image into two channels of a neural network model respectively for recognition, to obtain gesture feature information of the original gesture image, calculating the Euclidean distance between the gesture feature information and each positive sample gesture feature information in a database, and determining the gesture type in the original gesture image according to the Euclidean distance.
[0056] It should be noted that the user terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., and the server side can be implemented by a computer device with processing functions, but is not limited to this. The server side and the user terminal can be connected to the network via Bluetooth, USB (Universal Serial Bus) or other communication connection methods, and this application does not limit this.
[0057] In one embodiment, Figure 2 The flowchart of the gesture recognition method based on a neural network provided in the embodiment of the present application is as follows. The gesture recognition method based on a neural network can be applied to the above-mentioned server side, and includes the following steps:
[0058] Step S210, obtaining an original gesture image, and performing binarization processing on the original gesture image to obtain a binarized gesture image;
[0059] Step S220, inputting the original gesture image and the binarized gesture image into two channels of a neural network model for recognition, respectively, to obtain gesture feature information of the original gesture image;
[0060] Step S230, calculating the Euclidean distance between the gesture feature information and each positive sample gesture feature information in the database, and determining the gesture type in the original gesture image according to the Euclidean distance.
[0061] The gesture recognition solution provided in this application can be applied to the following scenarios: during the identity authentication process, the user's verification gesture image is captured. Due to the complexity of the actual situation, the captured verification gesture image may be blurred and difficult to recognize; or in games, videos and other files, the gesture image is only a small part of the entire frame of the picture. Due to the lack of storage technology or shooting technology, it is impossible to clearly identify the gesture type in the image.
[0062] In order to solve the above problems, the present application provides a gesture recognition method based on a neural network, which performs binarization processing on the acquired original gesture image to obtain its binarized gesture image, uses a dual-channel neural network model to identify gesture feature information, determines the Euclidean distance between the gesture feature information and each positive sample gesture feature information, and determines the gesture type of the original gesture image according to the size of the Euclidean distance. For example, the positive sample gesture with the smallest Euclidean distance can be used as the gesture type of the original gesture image.
[0063] After the gesture type in the original gesture image is identified using the above solution, the identification result can be used to perform the following operations, such as: performing verification analysis in identity authentication, or returning the recognition result of the gesture image to the user.
[0064] The original gesture image is binarized to obtain a binary gesture image. The binary gesture image extracts the texture features of the original gesture image, especially the local texture feature information of the original gesture image. The texture features recognize the user gesture and can distinguish the user gesture from the background image. The gesture category is recognized based on the binary gesture image, which is conducive to improving the accuracy of gesture recognition.
[0065] The solution provided in the present application is applicable to static gesture recognition scenarios. In order to solve the difficulty or failure of gesture recognition caused by the unclear collected gesture images, the solution provided in the present application proposes a gesture recognition method based on a neural network model. The neural network model has two input channels. The dual-channel convolutional neural network can simultaneously accept different features of the image as input. There are two features in the solution provided in the present application, one feature is a gesture feature, such as gesture posture information, and the other feature is a texture feature. Convolution processing is performed on each of them, and then these features are combined to extract more original gesture feature information for image recognition and classification, which is conducive to improving the recognition accuracy of gesture images.
[0066] In order to make the neural network-based gesture recognition solution and its technical effects provided by the present application more clear, the specific solution is described in detail with multiple embodiments below.
[0067] In one embodiment, the step of binarizing the original gesture image to obtain a binarized gesture image in step S210 may be processed in the following manner, as shown in the flowchart. Figure 3 As shown, it includes the following sub-steps:
[0068] S211, dividing the original gesture image into a plurality of sub-regions;
[0069] S212, performing the following operation on each pixel window of each sub-region: taking the gray value of the central pixel of the window as a threshold, comparing the gray values of the adjacent pixels with it, and obtaining the LBP value of the pixel window;
[0070] S213, replacing the original grayscale value of the pixel window with the LBP value of the pixel window to obtain a binary gesture image corresponding to the original gesture image.
[0071] LBP refers to Local Binary Patterns, an operator used to describe the local texture features of an image. The extracted features are the local texture features of the original gesture image. This solution converts the original gesture image into a binary gesture image in order to extract the texture feature information of the original gesture image.
[0072] Specifically, the original gesture image is divided into several sub-regions, and the several sub-regions include one or more sub-regions, and each sub-region includes multiple pixels. A suitable window size is selected, and the grayscale value of the pixel at the center of the window is used as the threshold. The grayscale values of the adjacent pixels are compared with the threshold to obtain the corresponding binary code to represent the local texture feature. The scheme is described by example. The window size is 3*3. If the pixel value around the center of the window is greater than the center pixel value, the position of the pixel is marked as 1, otherwise it is 0. In this way, the 8 pixels in the 3*3 neighborhood can generate an 8-bit binary number after comparison. The binary number is converted into a decimal number, that is, the LBP value of the pixel at the center of the window is obtained, and this value is used to reflect the texture feature information of the pixel window. The LBP value of each window pixel in the sub-region is obtained in the above manner, and the original grayscale value of the pixel window is replaced by the LBP value. After all the pixel windows are replaced, a binary sub-region gesture image is obtained. According to the scheme, the binary sub-region gesture images corresponding to all sub-regions are obtained, and the binary sub-region gesture images are spliced to obtain the binary gesture image of the original gesture image.
[0073] In one embodiment, before the step of inputting the original gesture image and the binarized gesture image into two channels of the neural network model for recognition respectively in step S220, the process further includes: establishing a neural network model according to the training gesture image. The process of establishing the neural network model can be performed in the following manner, see Figure 4 The process diagram shown includes the following steps:
[0074] S221, obtaining a training gesture image from a preset training image set, performing feature extraction on the training gesture image and its binarized gesture image, respectively obtaining N-dimensional feature vectors corresponding to the training gesture image and the binarized gesture image, and integrating the N-dimensional feature vectors to obtain a 2N-dimensional feature vector;
[0075] S222, performing feature vector comparison based on the 2N-dimensional feature vector, adjusting the weight of the feature vector using the comparison result of the positive sample gesture image, and obtaining a neural network model.
[0076] Wherein, N is any positive integer, N=1, 2, ... The training gesture image is extracted from a preset training image set, the training gesture image is binarized to obtain a binarized gesture image corresponding to the training gesture image, feature extraction is performed on the training gesture image and its binarized gesture image, N-dimensional feature vectors are extracted from both images, the obtained N-dimensional feature vectors are integrated to obtain a 2N-dimensional feature vector, the feature vectors of the positive sample gesture image are used for comparison, the weight of the 2N-dimensional feature vector is obtained, and a neural network model is established.
[0077] The positive sample gesture image refers to an image including a known gesture type, that is, the positive sample gesture image and the corresponding gesture type are pre-stored. The above 2N-dimensional feature vector of the positive sample gesture image is extracted, and the weight of the 2N-dimensional feature vector is obtained by using the positive sample gesture image as a training sample. The established neural network model can be described as follows:
[0078] P=A1*X1+A2*X2+…+A 2N *X 2N ,
[0079] Among them, X1, X2…X 2N is 2N eigenvectors, A1, A2…A 2N is the weight of the feature vector corresponding to the 2N feature vectors, and P is the corresponding gesture type. The weight of the feature vector corresponding to the 2N feature vectors is obtained by training a large number of positive sample gesture images in the positive sample gesture image set. The neural network model is obtained by this big data training method, which is conducive to calling the neural network model in the subsequent gesture image recognition to quickly obtain the accurate gesture type.
[0080] This embodiment further elaborates on how to obtain a neural network model. The neural network model described in the embodiment of the present application is preferably a dual-channel neural network model. Figure 5 The dual-channel neural network model is obtained in the manner shown, including the following sub-steps:
[0081] S2221, using the basic network structure of Inception-Resnet-V2 to build an initial dual-channel neural network model;
[0082] S2222, extracting the 2N-dimensional feature vector in the positive sample gesture image, inputting the 2N-dimensional feature vector and the gesture type corresponding to the positive sample gesture image into an initial dual-channel neural network model, and obtaining an initial weight value of the 2N-dimensional feature vector;
[0083] S2223, using all positive sample gesture images and corresponding gesture types in the positive sample gesture image set to continuously adjust the initial weight value of each feature vector in the initial dual-channel neural network model, after the weight value of each feature vector is determined, the dual-channel neural network model is obtained.
[0084] Specifically, the positive sample gesture image set includes a large number of positive sample gesture images of known gesture types. Specifically, the 2N-dimensional feature vector in the first positive sample gesture image is extracted, and the gesture type corresponding to the 2N-dimensional feature vector and the first positive sample gesture image is input into a dual-channel neural network model to obtain a first weight value of the 2N-dimensional feature vector, and the first weight value is an initial weight value; the 2N-dimensional feature vector in the second positive sample gesture image is extracted, and the gesture type corresponding to the 2N-dimensional feature vector and the second positive sample gesture image is input into a dual-channel neural network model in which the weight value of each feature vector is the first weight value, and the second weight value of the 2N-dimensional feature vector is obtained, and the second weight value is a weight value after adjusting the first weight value. In this way, the weight value of the 2N-dimensional feature vector is adjusted in turn using the positive sample gesture images in the positive sample gesture set, and after obtaining the final weight value of each feature vector in the dual-channel neural network model and determining the weight value corresponding to each feature vector, a dual-channel neural network model is established.
[0085] In step S2221, an initial dual-channel neural network model is constructed using the basic network structure of Inception-Resnet-V2. Inception-Resnet-V2 is a convolutional neural network, which is the neural network with the best image classification effect in current benchmark tests. The convolutional neural network model established using this network structure can improve the accuracy of gesture type recognition.
[0086] Taking N as 64 as an example, the process of obtaining a 2N-dimensional feature vector is explained: the positive sample gesture image is input into one channel of the dual-channel neural network model, and the binary gesture image corresponding to the positive sample gesture image is input into the other channel. Feature extraction is performed in the two channels respectively to obtain 64-dimensional feature vectors respectively. After L2 normalization, they are finally integrated and connected into a 128-dimensional vector.
[0087] The training process of the whole sample gesture image based on the integrated 128-dimensional vector is as follows: the first positive sample image is used to obtain the corresponding first weight value, the neural network model corresponding to the first weight value is used to calculate the value of the loss function between the output of the second positive sample gesture image and the preset gesture type, and the weight value of the neural network model is adjusted according to the value of the loss function to reduce the value of the loss function. The loss function between the output of the neural network model and the preset gesture type of the positive sample is continuously calculated. After a large number of sample training in the sample set, the value of the loss function is continuously reduced, and the accuracy of the gesture type output by the model is getting higher and higher. The 128-dimensional feature vector finally extracted can reflect the feature points of the gesture to be verified, and the weight of each feature vector can accurately reflect the image force of each feature point, which is conducive to fast and accurate gesture recognition.
[0088] The model is trained by the above method, and step 2 is used to extract gesture feature information. Then, gesture recognition is performed based on the extracted feature information. Preferably, 128-dimensional gesture features are extracted for gesture verification, which increases the robustness and accuracy of the gesture recognition algorithm.
[0089] Compared with the single-channel neural network model, the dual-channel neural network model established with the 2N-dimensional feature vector has more extracted feature information. The extracted feature vector is used to compare and identify the feature information, which is beneficial to improving the accuracy of gesture recognition.
[0090] The above embodiment introduces how to establish a neural network model based on the obtained 2N-dimensional feature vector. The following embodiment describes how to obtain the 2N-dimensional feature vector of a binary gesture image.
[0091] Furthermore, the following operations may be performed to obtain the LBP feature vector of the original gesture image: statistically analyzing the distribution of the LBP value of each sub-region to obtain the LBP histogram of each sub-region, and connecting the histograms of each sub-region to obtain the LBP texture feature vector of the original gesture image.
[0092] The N-dimensional feature vector corresponding to the binary gesture image can be obtained by the following operation, including the following sub-steps:
[0093] A1. Divide the binary gesture image into N sub-regions;
[0094] A2. Obtaining LBP histograms of the N sub-regions, and normalizing the LBP histograms;
[0095] A3. Connect the normalized histograms of the N sub-regions to obtain an N-dimensional feature vector corresponding to the binary gesture image.
[0096] In the embodiment of the present application, the process of obtaining the LBP histograms of N sub-regions is as follows: first, the LBP value of each pixel window in each sub-region is obtained using the scheme described in S212, and the distribution of the LBP value in each sub-region is counted to obtain the LBP histogram of each sub-region.
[0097] The LBP histograms of the N sub-regions are normalized, and the processed histograms of the sub-regions are arranged in a column according to the spatial order of the sub-regions to form an LBP feature vector. In this way, an N-dimensional feature vector corresponding to the binary gesture image is obtained.
[0098] Step S230 of calculating the Euclidean distance between the gesture feature information and each positive sample gesture feature information in the database, and determining the gesture type in the original gesture image according to the Euclidean distance, can be determined in the following manner, as shown in the flowchart. Figure 6 As shown, the following steps are included:
[0099] S231, obtaining a feature vector of the original gesture image and a feature vector of the same dimension of each positive sample gesture image in the database, and calculating the Euclidean distance between the feature vector of the original gesture image and the feature vector of each positive sample gesture image;
[0100] S232, obtaining the confidence between the original gesture image and each positive sample gesture image according to the Euclidean distance, and outputting the positive sample gesture type corresponding to the highest confidence as the gesture type in the original gesture image.
[0101] Step S230 determines the feature information in the original gesture image by calculating the Euclidean distance between the gesture feature information of the original gesture image and the feature information of each positive sample gesture in the database. The gesture feature information can be represented in a variety of forms. The embodiment of the present application preferably uses a feature vector to represent the feature information of the original gesture image. The original gesture image and its binarized gesture image are subjected to feature extraction according to the scheme provided in step S221 to obtain N-dimensional feature vectors corresponding to the original gesture image and the binarized gesture image, respectively. The N-dimensional feature vectors are integrated to obtain a 2N-dimensional feature vector. The 2N-dimensional feature vector of each positive sample gesture image is obtained by the same scheme. The Euclidean distance between the feature vector of the original gesture image and each positive sample gesture image is calculated. The confidence of each positive sample gesture image is obtained according to the size of the Euclidean distance, and the gesture type in the positive sample gesture image with the highest confidence is output as the gesture type of the original gesture image.
[0102] The solution provided in the embodiment of the present application uses the Euclidean distance between the feature vector corresponding to the original gesture image and the feature vector corresponding to the positive sample gesture image to determine the gesture type in the original gesture image. It can accurately judge the similarity between the gesture type in the original gesture image and the gesture type in the positive sample gesture image, and use the similarity to accurately judge the gesture type in the original gesture image in a short time.
[0103] Optionally, in order to improve the recognition accuracy of gesture types in the original gesture image, the number of samples of positive sample gesture images can be increased in the database, and the original gesture image can be pre-processed with image enhancement and / or image filtering, and the original gesture image can be processed using an edge-preserving denoising algorithm. Using the edge-preserving denoising algorithm to process the original gesture image is beneficial to highlighting the gesture part in the original gesture image and filtering out the noise in the original gesture image, which is beneficial to improving gesture recognition of the original gesture image.
[0104] The above is an embodiment of a gesture recognition method based on a neural network provided by the present application. With respect to the method, an embodiment of a gesture recognition device based on a neural network corresponding to the method is described below.
[0105] The present application also provides a gesture recognition device based on a neural network, and its structural diagram is shown in FIG. Figure 7 As shown, it includes: a binarization processing module 710, a recognition module 720, and a gesture type determination module 730, which are specifically as follows:
[0106] A binarization processing module 710 is used to obtain an original gesture image and perform binarization processing on the original gesture image to obtain a binarized gesture image;
[0107] A recognition module 720, configured to input the original gesture image and the binarized gesture image into two channels of a neural network model for recognition, respectively, to obtain gesture feature information of the original gesture image;
[0108] The gesture type determination module 730 is used to calculate the Euclidean distance between the gesture feature information and each positive sample gesture feature information in the database, and determine the gesture type in the original gesture image according to the Euclidean distance.
[0109] Regarding the gesture recognition device based on a neural network in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0110] Furthermore, the embodiment of the present application also provides a computer-readable storage medium on which computer instructions are stored, and when the computer instructions are executed by the processor, the steps of the gesture recognition method based on the neural network described in any one of the above are implemented. Wherein, the storage medium includes but is not limited to any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic card or optical card. That is, the storage medium includes any medium that stores or transmits information in a readable form by a device (for example, a computer). It can be a read-only memory, a disk or an optical disk, etc.
[0111] Furthermore, an embodiment of the present application further provides a computer device, the computer device comprising:
[0112] one or more processors;
[0113] a storage device for storing one or more programs,
[0114] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of any one of the above-mentioned neural network-based gesture recognition methods.
[0115] Figure 8 800 is a structural block diagram of a computer device 800 according to an exemplary embodiment. For example, the computer device 800 may be provided as a server. Figure 8 , the computer device 800 includes a processing component 822, which further includes one or more processors, and a memory resource represented by a memory 832 for storing instructions executable by the processing component 822, such as an application. The application stored in the memory 832 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 822 is configured to execute instructions to perform the steps of the above-mentioned neural network-based gesture recognition method based on the dual-channel neural network.
[0116] The computer device 800 may also include a power supply component 826 configured to perform power management of the computer device 800, a wired or wireless network interface 850 configured to connect the computer device 800 to a network, and an input / output (I / O) interface 858. The computer device 800 may operate based on an operating system stored in the memory 832, such as Windows Server TM, Mac OS X TM, Unix TM, Linux TM, FreeBSD TM or the like. It should be understood that although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in order and may be executed in other orders. Moreover, at least a portion of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but may be executed at different times, and their execution order is not necessarily sequential, but may be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
[0117] It should be understood that each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of software functional modules.
[0118] The above is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A gesture recognition method based on a neural network, characterized in that: include: Acquire an original gesture image, and perform binarization processing on the original gesture image to obtain a binarized gesture image; A neural network model is established according to the training gesture image; wherein the steps of establishing the neural network model include: Acquire training gesture images from a preset training image set, perform feature extraction on the training gesture images, and obtain training gesture images respectively; Dividing the binary gesture image of the training gesture image into N sub-regions; Obtaining LBP histograms of the N sub-regions, and performing normalization processing on the LBP histograms; Connecting the normalized histograms of the N sub-regions to obtain an N-dimensional feature vector corresponding to the binary gesture image of the training gesture image, and integrating the N-dimensional feature vectors to obtain a 2N-dimensional feature vector; Comparing feature vectors based on the 2N-dimensional feature vectors, adjusting the weights of the feature vectors using the comparison results of the positive sample gesture images, and obtaining a neural network model; Inputting the original gesture image and the binarized gesture image into two channels of a neural network model for recognition respectively, to obtain gesture feature information of the original gesture image; The Euclidean distance between the gesture feature information and each positive sample gesture feature information in the database is calculated, and the gesture type in the original gesture image is determined according to the Euclidean distance.
2. The gesture recognition method based on neural network according to claim 1, characterized in that: The step of binarizing the original gesture image to obtain a binarized gesture image comprises: Divide the original gesture image into several sub-regions; The following operations are performed on the pixel window in each sub-region: the gray value of the pixel at the center of the window is used as the threshold, and the gray values of the adjacent pixels are compared with it to obtain the LBP value of the pixel window; The original grayscale value of the pixel window is replaced by the LBP value of the pixel window to obtain a binary gesture image corresponding to the original gesture image.
3. The gesture recognition method based on neural network according to claim 1, characterized in that: The step of adjusting the weight of the feature vector using the comparison result of the positive sample gesture image to obtain the neural network model includes: The basic network structure of Inception-Resnet-V2 is used to build an initial dual-channel neural network model; Extracting the 2N-dimensional feature vector in the positive sample gesture image, inputting the 2N-dimensional feature vector and the gesture type corresponding to the positive sample gesture image into an initial dual-channel neural network model, and obtaining an initial weight value of the 2N-dimensional feature vector; All positive sample gesture images and corresponding gesture types in the positive sample gesture image set are used to continuously adjust the initial weight value of each feature vector in the initial dual-channel neural network model. After the weight value of each feature vector is determined, the dual-channel neural network model is obtained.
4. The gesture recognition method based on neural network according to claim 1, characterized in that: The step of calculating the Euclidean distance between the gesture feature information and each positive sample gesture feature information in the database, and determining the gesture type in the original gesture image according to the Euclidean distance comprises: Obtain the feature vector of the original gesture image and the feature vector of the same dimension of each positive sample gesture image in the database, and calculate the Euclidean distance between the feature vector of the original gesture image and the feature vector of each positive sample gesture image; The confidence between the original gesture image and each positive sample gesture image is obtained according to the Euclidean distance, and the positive sample gesture type corresponding to the highest confidence is output as the gesture type in the original gesture image.
5. The gesture recognition method based on neural network according to claim 1, characterized in that: Before the step of binarizing the original gesture image to obtain a binarized gesture image, the method further includes: The original gesture image is denoised using the edge-preserving denoising algorithm.
6. A gesture recognition device based on a neural network, characterized in that: include: A binarization processing module, used to obtain an original gesture image, and perform binarization processing on the original gesture image to obtain a binarized gesture image; A recognition module, used to input the original gesture image and the binarized gesture image into two channels of a neural network model for recognition, respectively, to obtain gesture feature information of the original gesture image; A gesture type determination module, used to calculate the Euclidean distance between the gesture feature information and each positive sample gesture feature information in the database, and determine the gesture type in the original gesture image according to the Euclidean distance; Wherein, the neural network model is established through the following steps: Obtaining training gesture images from a preset training image set, performing feature extraction on the training gesture images, and obtaining training gesture images respectively; Dividing the binary gesture image of the training gesture image into N sub-regions; Obtaining LBP histograms of the N sub-regions, and performing normalization processing on the LBP histograms; Connecting the normalized histograms of the N sub-regions to obtain an N-dimensional feature vector corresponding to the binary gesture image of the training gesture image, and integrating the N-dimensional feature vectors to obtain a 2N-dimensional feature vector; The feature vectors are compared based on the 2N-dimensional feature vectors, and the weights of the feature vectors are adjusted using the comparison results of the positive sample gesture images to obtain a neural network model.
7. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store computer instructions, and when the computer-readable storage medium is executed on a computer, the computer can execute the steps of the gesture recognition method based on a neural network as described in any one of claims 1 to 5.
8. A computer device, characterized in that: The computer device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the neural network-based gesture recognition method as described in any one of claims 1 to 5.
Citation Information
Patent Citations
Static gesture recognizing method
CN104020848A
Training method, recognition method and system for merging key feature points
CN106127170A
Convolutional neural network and user habitual behavior analysis combination-based AR system gesture identification method
CN108334814A
A monocular static gesture recognition method based on multi-feature fusion
CN109190496A
Pedestrian re-identification method and a related product
CN109657533A
