Gesture recognition method, system and gesture recognition device based on brain-like computing

Abstract

The present invention proposes a gesture recognition method, system and gesture recognition device based on brain-like computing, the method comprising: collecting multi-channel electromyographic signals of the user at the arm, filtering and processing, and obtaining a first data set; segmenting processing to extract effective gesture activity segments, and obtaining gesture features from the extracted gesture activity segments; inputting the test data set into the recognition model; in the recognition model, the first memory encodes and maps the test data set into a hyperdimensional vector, and performs similarity calculation with the sample hyperdimensional vector, finds the sample hyperdimensional vector with the highest similarity, and outputs the sample gesture corresponding to the sample hyperdimensional vector as a predicted gesture. The present invention recognizes gestures on the surface electromyographic signals of human arms, and only needs to learn on a small amount of data to achieve an ideal recognition accuracy, and the calculation is convenient, no multiple iterations are required, the training model is small, and it is suitable for direct learning and prediction on the hardware.

Description

Technical Field

[0001] The present invention relates to the field of gesture recognition, and in particular to a gesture recognition method, system and gesture recognition device based on brain-like computing. Background Art

[0002] In the concept of the metaverse, gesture recognition is an important medium for human-computer interaction. Gesture recognition technology can provide technical support for human-computer interaction in the metaverse, displaying the movements and trajectories of human hands in the real world in real time in the virtual space, and interacting with the virtual world.

[0003] In the existing technology, most gesture recognition technologies are based on computer vision algorithms. In dim light or when the human hand is blocked, the gesture recognition effect will become very poor, or even impossible to recognize. Moreover, this type of gesture recognition method often needs to be used with other peripheral devices, such as handles and other media devices, in order to obtain feedback information in the virtual world.

[0004] In addition, gesture recognition technology based on deep learning or machine learning algorithms often requires a large amount of data for learning and training, which requires multiple iterations. The trained algorithm model requires a large storage space and can only be stored on a large PC. It is difficult to learn and predict locally on a single-chip microcomputer or FPGA hardware, which greatly limits the application of gesture recognition technology in microprocessors.

[0005] Therefore, there is an urgent need for a gesture recognition solution that can learn and predict locally on hardware to solve the above problems. Summary of the invention

[0006] In view of this, the present invention proposes a gesture recognition method, system and gesture recognition device based on brain-like computing. The specific scheme is as follows:

[0007] A gesture recognition method based on brain-like computing, comprising the following:

[0008] Collect multi-channel electromyographic signals of the user at the arm to obtain a multi-channel electromyographic data set about the gesture;

[0009] Performing filtering processing on the multi-channel electromyography data set to obtain a first data set;

[0010] Processing the first data set in segments to extract valid gesture activity segments, and acquiring gesture features from the extracted gesture activity segments to obtain a test data set;

[0011] Obtaining a recognition model that has been trained for the user, the recognition model is an algorithm model built based on hyperdimensional computing, inputting the test data set into the recognition model for gesture prediction, the recognition model being preset with a first memory and a second memory, the second memory including a sample hyperdimensional vector carrying a sample gesture;

[0012] In the recognition model, the first memory encodes and maps the test data set into a hyperdimensional vector, and performs similarity calculation with the sample hyperdimensional vector to find the sample hyperdimensional vector with the highest similarity, and outputs the sample gesture corresponding to the sample hyperdimensional vector as the predicted gesture.

[0013] In a specific embodiment, the segmentation process specifically includes:

[0014] Before collecting the multi-channel electromyographic data set of the user's gesture, the multi-channel electromyographic signal of the user in a resting state is collected in advance to obtain a resting state threshold;

[0015] Valid gesture activity segments are extracted based on the resting state threshold.

[0016] In a specific embodiment, the training process of the recognition model includes:

[0017] Collect multi-channel electromyographic signals of a user at the arm to obtain a sample data set of multiple sample gestures of the user;

[0018] Performing filtering on the sample data set to obtain a first training data set;

[0019] Processing the first training data set in segments to extract valid gesture activity segments, extracting gesture features from the extracted gesture activity segments, and marking corresponding gesture categories to obtain training set data;

[0020] Build an initial algorithm model based on hyperdimensional computing, and set the data dimension of the model to D;

[0021] Initializing the first memory and the second memory in the initial algorithm model respectively;

[0022] Inputting the training data set into the initial algorithm model for training, obtaining a sample hyperdimensional vector, and storing it in the second memory;

[0023] After the training is completed, the algorithm model is output to obtain the recognition model for the user.

[0024] In a specific embodiment, the user's electromyographic signals are collected through a multi-channel bracelet arranged with an electrode array.

[0025] In a specific embodiment, the electrode array arrangement in the multi-channel bracelet includes a 4×8 array or a 4×16 array.

[0026] A gesture recognition system based on brain-like computing, comprising:

[0027] A collection unit, used to collect the electromyographic signals of the user at the arm, and obtain a multi-channel electromyographic data set about the gesture;

[0028] A filtering unit, used for filtering the multi-channel electromyography data set to obtain a first data set;

[0029] A preprocessing unit, configured to process the first data set in segments to extract valid gesture activity segments, and obtain gesture features from the extracted gesture activity segments to obtain a test data set;

[0030] A model acquisition unit, used to acquire a recognition model that has been trained for the user, wherein the recognition model is an algorithm model constructed based on hyperdimensional computing, and the test data set is input into the recognition model for gesture prediction. The recognition model is preset with a first memory and a second memory, and the second memory includes a sample hyperdimensional vector carrying a sample gesture;

[0031] Prediction unit, in the recognition model, the first memory encodes and maps the test data set into a hyperdimensional vector, and performs similarity calculation with the sample hyperdimensional vector, finds the sample hyperdimensional vector with the highest similarity, and outputs the sample gesture corresponding to the sample hyperdimensional vector as the predicted gesture.

[0032] In a specific embodiment, the pre-processing unit comprises:

[0033] Before collecting the multi-channel electromyographic data set of the user's gesture, the multi-channel electromyographic signal of the user in a resting state is collected in advance to obtain a resting state threshold;

[0034] Valid gesture activity segments are extracted based on the resting state threshold.

[0035] In a specific embodiment, it also includes a model training unit;

[0036] The model training unit specifically includes:

[0037] Collect multi-channel electromyographic signals of a user at the arm to obtain a sample data set of multiple sample gestures of the user;

[0038] Performing filtering on the sample data set to obtain a first training data set;

[0039] Processing the first training data set in segments to extract valid gesture activity segments, extracting gesture features from the gesture activity segments, and marking corresponding gesture categories to obtain training set data;

[0040] Construct an initial algorithm model based on hyperdimensional computing, and set the data dimension of the model to be a hyperdimensional vector;

[0041] Initializing the first memory and the second memory in the initial algorithm model respectively;

[0042] Inputting the training data set into the initial algorithm model for training, obtaining a sample hyperdimensional vector, and storing it in the second memory;

[0043] After the training is completed, the algorithm model is output to obtain the recognition model for the user.

[0044] In a specific embodiment, the acquisition unit acquires the user's electromyographic signals through a multi-channel bracelet arranged with an electrode array;

[0045] The electrode array arrangement in the multi-channel bracelet includes a 4×8 array or a 4×16 array.

[0046] A gesture recognition device, comprising:

[0047] An acquisition module, used to acquire multi-channel electromyographic signals from the user's arm;

[0048] The main control module is integrated on the FPGA and electrically connected to the acquisition module to control the acquisition module and execute a gesture recognition method based on brain-like computing as described in any one of the above items.

[0049] Beneficial effects: The present invention proposes a gesture recognition method, system and gesture recognition device based on brain-like computing. By performing gesture recognition on the surface electromyographic signals of human arms, poor recognition effects caused by light and occlusion problems can be avoided, and the gesture recognition accuracy based on electromyographic signals is very high. The scheme only needs to be learned on a small amount of data to achieve the ideal recognition accuracy; and the calculation is convenient, no multiple iterations are required, the training model is small, and it is suitable for direct learning and prediction on the local hardware. The proposed gesture recognition device integrates the gesture recognition method into the local hardware such as FPGA, which is convenient for direct learning and prediction on the local hardware. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A flowchart of a gesture recognition method according to an embodiment of the present invention;

[0051] Figure 2 A schematic diagram of the principle of a gesture recognition method according to an embodiment of the present invention;

[0052] Figure 3 Schematic diagram of myoelectric signal waveform according to an embodiment of the present invention;

[0053] Figure 4 Schematic diagram of the electromyographic signal waveform after segmented processing according to an embodiment of the present invention;

[0054] Figure 5 Schematic diagram of a gesture recognition system module according to an embodiment of the present invention.

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0056] Figure numerals: 1 - acquisition unit; 2 - filtering unit; 3 - preprocessing unit; 4 - model acquisition unit; 5 - prediction unit; 6 - model training unit. DETAILED DESCRIPTION

[0057] In the following, various embodiments of the present disclosure will be described more fully. The present disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of the present disclosure to the specific embodiments disclosed herein, but the present disclosure should be understood to cover all adjustments, equivalents and / or alternatives that fall within the spirit and scope of the various embodiments of the present disclosure.

[0058] The present invention proposes a gesture recognition method, system and gesture recognition device based on brain-like computing. By performing gesture recognition on the electromyographic signals on the surface of the human arm, poor recognition effect caused by light and occlusion problems can be avoided. The accuracy of gesture recognition based on electromyographic signals is very high, which is suitable for providing the metaverse with reliable and convenient gesture recognition technology to achieve a complete closed-loop interaction effect.

[0059] The solution of the present invention can achieve an ideal recognition accuracy rate by learning on only a small amount of data; it is also easy to calculate, does not require multiple iterations, has a small training model, and is suitable for direct learning and prediction on local hardware.

[0060] The terms used in the various embodiments disclosed in the present invention are only used for the purpose of describing specific embodiments and are not intended to limit the various embodiments disclosed in the present invention. As used herein, the singular form is intended to also include the plural form, unless the context clearly indicates otherwise. Unless otherwise specified, all terms used herein (including technical terms and scientific terms) have the same meaning as the meanings commonly understood by ordinary technicians in the field of the various embodiments disclosed in the present invention. The terms (such as the terms defined in the generally used dictionary) will be interpreted as having the same meaning as the contextual meaning in the relevant technical field and will not be interpreted as having an idealized meaning or an overly formal meaning, unless clearly defined in the various embodiments disclosed in the present invention.

[0061] Example 1

[0062] Embodiment 1 of the present invention discloses a gesture recognition method based on brain-like computing, which performs gesture recognition prediction based on the muscle electrical signals of human arm. Figure 1 The specific plan is as follows:

[0063] A gesture recognition method based on brain-like computing includes the following steps:

[0064] 101. Collect multi-channel electromyographic signals of the user at the arm to obtain a multi-channel electromyographic data set about gestures;

[0065] 102. Filter the multi-channel electromyography data set to obtain a first data set;

[0066] 103. Process the first data set in segments to extract valid gesture activity segments, and obtain gesture features from the extracted gesture activity segments to obtain a test data set;

[0067] 104. Obtain a recognition model that has been trained for the user, and input the test data set into the recognition model for gesture prediction, wherein the recognition model is preset with a first memory and a second memory, and the second memory includes a sample hyperdimensional vector carrying the sample gesture;

[0068] 105. In the recognition model, the first memory encodes and maps the test data set into a hyperdimensional vector, and performs similarity calculation with the sample hyperdimensional vector to find the sample hyperdimensional vector with the highest similarity, and outputs the sample gesture corresponding to the sample hyperdimensional vector as the predicted gesture.

[0069] The principle diagram of the gesture recognition method is as attached in the manual. Figure 2 As shown. The solution of this embodiment is to perform gesture recognition based on the electromyographic signal of the human arm muscle. Compared with the traditional gesture recognition solution based on visual algorithm, this solution is not affected by problems such as light and occlusion, and the recognition effect is stable. Each gesture will generate a corresponding electromyographic signal, so the gesture recognition based on electromyographic signal has a high accuracy rate.

[0070] In addition, since different people may have different electromyographic data when making the same gesture, the gesture recognition method of this embodiment needs to be trained accordingly for each user when using it to obtain a recognition model for each user. The data of the recognition model may be different for different users, and targeted training and learning can make the prediction more accurate.

[0071] In this embodiment, a flexible multi-channel bracelet is used to collect electromyographic signals. A flexible multi-channel bracelet is placed at the user's arm position to collect N-channel electromyographic data sets of relevant gestures. For the wearer, the flexible multi-channel bracelet is very convenient to collect and wear. Among them, the electrodes in the multi-channel bracelet are arranged according to a specific array. Preferably, the electrode array arrangement form in the multi-channel bracelet includes a 4×8 array, a 4×16 array or other array forms. Flexible electrodes require more electrode arrangements to obtain more electromyographic data. The more electrodes, the more data density information. Too few electrodes obtain too little data and cannot accurately identify gestures. Too many electrodes not only increase the acquisition cost, but also increase the amount of data for subsequent data processing. The 4×8 array and 4×16 array selected in this embodiment can adapt to most acquisition situations.

[0072] Specifically, the collected multi-channel electromyography data set is filtered, including high-pass and low-pass filtering and 50Hz power frequency notch filtering, to remove other noises and obtain the first data set. When training the algorithm, the aforementioned scheme is also used for filtering. The image of the electromyography signal collected by the flexible multi-channel bracelet after filtering is shown in the attached manual. Figure 3 shown.

[0073] The specific segmentation processing includes: before collecting the multi-channel electromyographic data set of the user's gesture, pre-collecting the electromyographic signal of the user in a resting state to obtain the resting state threshold; dividing the gesture activity segment according to the resting state threshold. Figure 3 The waveform after segmentation processing is shown in the attached manual. Figure 4 The myoelectric signals of a person in a resting state and a non-resting state are obviously different. This embodiment obtains a resting state threshold to exclude the data of the user in a resting state and only retains the myoelectric signals in a non-resting state.

[0074] The process of acquiring gesture features includes: acquiring a valid gesture activity segment to obtain a first gesture activity segment; and then calculating gesture features of each channel in the first gesture activity segment. The gesture features can be reflected by the average value of each channel data.

[0075] The first data set is first segmented and then gesture features are extracted. In a specific embodiment, the first data set may be marked with corresponding gesture labels and compared with the predicted gestures to verify whether the prediction is correct. The order of the steps of extracting gesture features and marking gesture labels can be set arbitrarily.

[0076] In this embodiment, a pre-trained recognition model is used for gesture prediction. In particular, the recognition mode is an algorithm model built based on hyperdimensional computing (HDC). The recognition model of this embodiment is significantly different from the traditional AI algorithm based on neural networks. The HDC algorithm only requires less data to achieve the ideal recognition result, and the model is lightweight, which is suitable for model learning and prediction on local hardware such as FPGA and single-chip microcomputers.

[0077] During the prediction process, the recognition model will encode the test data into a hyper-vector, and then calculate the similarity between it and the hyper-vector in the associative memory to find similar gesture categories and obtain the final gesture result.

[0078] Since each user's electromyographic signal is different, the corresponding recognition model needs to be customized according to the user. Itemmemory and associative memory are preset in the recognition model. Itemmemory is used as the first memory and associative memory is used as the second memory. The recognition model has memory and reaction, and can achieve true perception ability instead of relying on image learning to improve recognition accuracy. The associative memory stores relevant sample hyperdimensional vectors. Each sample hyperdimensional vector corresponds to a corresponding sample gesture. As long as the matching sample hyperdimensional vector is found, the corresponding gesture can be predicted. The stored sample hyperdimensional vector is equivalent to memory, and predicting the corresponding gesture based on the matching sample hyperdimensional vector is equivalent to perception.

[0079] When using the trained recognition model to predict gestures, the item memory will obtain the data of the corresponding channel and gesture data in the data set for encoding operations, including addition operations and matrix multiplication operations, to obtain a hyperdimensional vector. Then, the hyperdimensional vector and the sample hyperdimensional vector of the corresponding gesture in the associative memory are similarly calculated to find the sample hyperdimensional vector with the highest similarity to the hyperdimensional vector.

[0080] If the user's hyperdimensional vector has the highest similarity to a certain sample hyperdimensional vector, the sample gesture corresponding to the sample hyperdimensional vector is used as the predicted gesture of the user.

[0081] Among them, the training process of the recognition model includes:

[0082] 201. Collect multi-channel electromyographic signals of a user at the arm to obtain a sample data set of multiple sample gestures of the user. Place a flexible multi-channel bracelet at the user's arm position to collect an N-channel electromyographic data set of a preset sample gesture.

[0083] 202. Perform filtering processing on the sample data set to obtain a first training data set. The filtering processing includes high-pass and low-pass filtering and 50 Hz power frequency notch filtering.

[0084] 203. Segment processing of the first training data set to extract valid gesture activity segments, extract gesture features from the gesture activity segments, and mark corresponding gesture categories to obtain training set data. Segment processing, gesture feature extraction and gesture category marking are the same as the above parts.

[0085] 204. Construct the initial algorithm model based on hyperdimensional computing, and set the data dimension of the model to D. Build the recognition model, and set the dimension of the hypervector of the recognition model to D. D is a set parameter, and the vector with a large dimension is called a hyperdimensional vector. The data type of the recognition model is a hyperdimensional vector, which stores itemmemory and associativememory.

[0086] 205. Initialize the first memory and the second memory in the initial algorithm model respectively. After building the model, initialize the itemmemory and associativememory of the recognition model respectively. After initializing the associativememory, an initial hyperdimensional vector will be obtained. During training, both itemmemory and associativememory encode the data.

[0087] 206. Input the training data set into the initial algorithm model for training, obtain a sample hyperdimensional vector carrying the sample gesture, and store it in the second memory.

[0088] Specifically, the first memory encodes the gesture features of each channel and each gesture category in the training data set to obtain a hyperdimensional vector; the second memory is preset with the initial hyperdimensional vector of each gesture category after initialization; the hyperdimensional vector and the initial hyperdimensional vector of the corresponding gesture category are encoded to obtain a sample hyperdimensional vector carrying the sample gesture. In this embodiment, the encoding operation includes operations commonly used in the encoding process such as addition operations and matrix multiplication operations.

[0089] 207. After the training is completed, the second memory is subjected to polarization processing to simplify the computational cost.

[0090] 208. Output the processed algorithm model to obtain a recognition model for the user.

[0091] Learning methods based on neural networks cannot remember, resulting in large models and slow data processing. The so-called memory ability of current artificial intelligence models is to use past information for processing, but this is not true memory ability. For example, when a baseball player is preparing to catch a baseball flying from a distance, he does not catch the ball through precise calculations, but uses muscle reactions after years of training. At present, AI does not have such natural reaction ability similar to human muscle memory.

[0092] The gesture recognition method of this embodiment is based on the recognition model, which can generate memory, greatly reduce the computing requirements, and enable tasks to be completed faster and more efficiently, requiring only a small amount of data to achieve the desired results. More importantly, the HDC algorithm is very lightweight and suitable for direct learning and prediction on the local hardware, which can save a lot of costs in data transmission and model storage.

[0093] This embodiment provides a gesture recognition method based on brain-like computing. By performing gesture recognition on the surface electromyographic signals of human arms, poor recognition results caused by light and occlusion problems can be avoided, and the gesture recognition accuracy based on electromyographic signals is very high. The solution only needs to be learned on a small amount of data to achieve an ideal recognition accuracy; it is easy to calculate, does not require multiple iterations, and has a small training model, which is suitable for direct learning and prediction on the local hardware.

[0094] Example 2

[0095] Embodiment 2 of the present invention discloses a gesture recognition system based on brain-like computing. Embodiment 1 is a systematized gesture recognition method. The specific structure of the system is as shown in the attached specification. Figure 5 The specific plan is as follows:

[0096] A gesture recognition system based on brain-like computing, comprising:

[0097] The acquisition unit 1 is used to collect multi-channel electromyographic signals of the user at the arm to obtain a multi-channel electromyographic data set about the gesture; the electromyographic signals are collected using a flexible multi-channel bracelet. A flexible multi-channel bracelet is placed at the user's arm position to collect N-channel electromyographic data sets of related gestures. For the wearer, the collection and wearing of the flexible multi-channel bracelet are very convenient. Among them, the electrodes in the multi-channel bracelet are arranged in a specific array. Preferably, the electrode array arrangement form in the multi-channel bracelet includes a 4×8 array, a 4×16 array or other array forms.

[0098] Filtering unit 2, used for filtering the multi-channel electromyography data set to obtain a first data set;

[0099] The preprocessing unit 3 is used to process the first data set in segments to extract valid gesture activity segments, and obtain gesture features from the extracted gesture activity segments to obtain a test data set; the segmentation processing specifically includes: before collecting the multi-channel electromyographic data set of the user's gesture, pre-collecting the electromyographic signal of the user in a resting state to obtain the resting state threshold; dividing the gesture activity segments according to the resting state threshold. Figure 3 The waveform after segmentation processing is shown in the attached manual. Figure 4 The myoelectric signals of a person in a resting state and a non-resting state are obviously different. This embodiment obtains a resting state threshold to exclude the data of the user in a resting state and only retains the myoelectric signals in a non-resting state.

[0100] A model acquisition unit 4 is used to acquire a recognition model that has been trained for the user, and input the test data set into the recognition model for gesture prediction. The recognition model is preset with a first memory and a second memory, and the second memory includes a sample hyperdimensional vector carrying a sample gesture;

[0101] Prediction unit 5, in the recognition model, the first memory maps the test data set encoding into a hyperdimensional vector, and calculates the similarity with the sample hyperdimensional vector, finds the sample hyperdimensional vector with the highest similarity, and outputs the sample gesture corresponding to the sample hyperdimensional vector as the predicted gesture.

[0102] It also includes a model training unit 6, which specifically includes: collecting multi-channel electromyographic signals of a user at the arm to obtain a sample data set of multiple sample gestures of the user; filtering the sample data set to obtain a first training data set; segmenting the first training data set to extract effective gesture activity segments, extracting gesture features from the gesture activity segments, and marking corresponding gesture categories to obtain training set data; constructing an initial algorithm model based on hyperdimensional computing, setting the data dimension of the model to a hyperdimensional vector; respectively initializing the first memory and the second memory in the initial algorithm model; inputting the training data set into the initial algorithm model for training, encoding and mapping the training data set by the first memory and the second memory to obtain a sample hyperdimensional vector carrying the sample gesture, and storing it in the second memory; after the training is completed, outputting the algorithm model to obtain a recognition model for the user.

[0103] In this embodiment, a pre-trained recognition model is used for gesture prediction. In particular, the recognition mode is an algorithm model built based on hyperdimensional computing (HDC). The recognition model of this embodiment is significantly different from the traditional AI algorithm based on neural networks. The HDC algorithm only requires less data to achieve the ideal recognition result, and the model is lightweight, which is suitable for model learning and prediction on local hardware such as FPGA and single-chip microcomputers.

[0104] This embodiment discloses a gesture recognition system based on brain-like computing, which systematizes the gesture recognition method of Embodiment 1 to make it more practical.

[0105] Example 3

[0106] This embodiment discloses a gesture recognition device, which combines a gesture recognition method based on brain-like computing in Embodiment 1 with FPGA. The specific scheme is as follows:

[0107] A gesture recognition device comprises a collection module and a main control module.

[0108] The acquisition module is used to collect multi-channel electromyographic signals at the user's arm. The acquisition module uses a flexible multi-channel bracelet to collect electromyographic signals. A flexible multi-channel bracelet is placed at the user's arm to collect N-channel electromyographic data sets of related gestures. For the wearer, the flexible multi-channel bracelet is very convenient for collection and wearing. Among them, the electrodes in the multi-channel bracelet are arranged in a specific array. Preferably, the electrode array arrangement form in the multi-channel bracelet includes a 4×8 array or a 4×16 array or other array forms.

[0109] The main control module is integrated on the FPGA and electrically connected to the acquisition module, and is used to control the acquisition module to collect signals and execute a gesture recognition method based on brain-like computing as in Example 1.

[0110] A gesture recognition method based on brain-like computing includes the following steps:

[0111] 101. Collect electromyographic signals of the user at the arm to obtain a multi-channel electromyographic data set about the gesture;

[0112] 102. Filter the multi-channel electromyography data set to obtain a first data set;

[0113] 103. Process the first data set in segments to extract valid gesture activity segments, and obtain gesture features from the extracted gesture activity segments to obtain a test data set;

[0114] 104. Obtain a recognition model that has been trained for the user, and input the test data set into the recognition model for gesture prediction, wherein the recognition model is preset with a first memory and a second memory, and the second memory includes a sample hyperdimensional vector carrying the sample gesture;

[0115] 105. In the recognition model, the first memory encodes and maps the test data set into a hyperdimensional vector, and performs similarity calculation with the sample hyperdimensional vector to find the sample hyperdimensional vector with the highest similarity, and outputs the sample gesture corresponding to the sample hyperdimensional vector as the predicted gesture.

[0116] This embodiment provides a gesture recognition device, which combines a gesture recognition method based on brain-like computing in Embodiment 1 with FPGA, so that it can be directly learned and predicted on the local hardware.

[0117] The present invention proposes a gesture recognition method, system and gesture recognition device based on brain-like computing. By performing gesture recognition on the surface electromyographic signals of human arms, poor recognition effects caused by light and occlusion problems can be avoided, and the gesture recognition accuracy based on electromyographic signals is very high. The scheme only needs to be learned on a small amount of data to achieve an ideal recognition accuracy; and the calculation is convenient, no multiple iterations are required, the training model is small, and it is suitable for direct learning and prediction on the local hardware. The gesture recognition method is integrated into the local hardware such as FPGA, which is convenient for direct learning and prediction on the local hardware.

[0118] Those skilled in the art will appreciate that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily necessary for the implementation of the present invention. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the description of the implementation scenario, or can be changed accordingly and located in one or more devices different from the present implementation scenario. The modules of the above-mentioned implementation scenarios can be combined into one module, or can be further divided into multiple sub-modules. The above-mentioned serial numbers of the present invention are only for description and do not represent the advantages and disadvantages of the implementation scenarios. Disclosed above are only a few specific implementation scenarios of the present invention, but the present invention is not limited thereto, and any changes that can be thought of by those skilled in the art should fall within the scope of protection of the present invention.

Claims

1. A gesture recognition method based on brain-like computing, It is characterized in that These include: The multi-channel bracelet with an electrode array arrangement collects multi-channel electromyographic signals of the user at the arm, the multi-channel bracelet is placed at the user's arm position, and the multi-channel bracelet is a flexible multi-channel bracelet, and a multi-channel electromyographic data set about gestures is obtained; the electrode array arrangement form in the multi-channel bracelet includes Array or Array; Performing filtering processing on the multi-channel electromyography data set to obtain a first data set; Processing the first data set in segments to extract valid gesture activity segments, and acquiring gesture features from the extracted gesture activity segments to obtain a test data set; The segmentation processing specifically includes: before collecting the multi-channel electromyographic data set of the user's gesture, pre-collecting the electromyographic signal of the user in a resting state to obtain the resting state threshold, and dividing the gesture activity segment according to the resting state threshold; Obtaining a recognition model that has been trained for the user, the recognition model is an algorithm model built based on hyperdimensional computing, inputting the test data set into the recognition model for gesture prediction, the recognition model being preset with a first memory and a second memory, the second memory including a sample hyperdimensional vector carrying a sample gesture; In the recognition model, the first memory encodes and maps the test data set into a hyperdimensional vector through addition operations and matrix multiplication operations, and performs similarity calculation with the sample hyperdimensional vector to find the sample hyperdimensional vector with the highest similarity, and outputs the sample gesture corresponding to the sample hyperdimensional vector as the predicted gesture; The training process of the recognition model includes: Collect multi-channel electromyographic signals of a user at the arm to obtain a sample data set of multiple sample gestures of the user; Performing filtering on the sample data set to obtain a first training data set; Processing the first training data set in segments to extract valid gesture activity segments, extracting gesture features from the extracted gesture activity segments, and marking corresponding gesture categories to obtain training set data; Build an initial algorithm model based on hyperdimensional computing, and set the data dimension of the model to D; Initializing the first memory and the second memory in the initial algorithm model respectively; The training data set is input into the initial algorithm model for training, the first memory encodes the gesture features of each channel and each gesture category in the training data set to obtain a hyperdimensional vector; the second memory is preset with the initial hyperdimensional vector of each gesture category after initialization; the hyperdimensional vector and the initial hyperdimensional vector of the corresponding gesture category are encoded to obtain a sample hyperdimensional vector carrying the sample gesture; After the training is completed, the second memory is polarized and the algorithm model is output to obtain a recognition model for the user.

2. A gesture recognition system based on brain-like computing, It is characterized in that These include: A collection unit is used to collect multi-channel electromyographic signals of a user at the arm through a multi-channel bracelet arranged with an electrode array, the multi-channel bracelet is placed at the user's arm position, and the multi-channel bracelet is a flexible multi-channel bracelet, to obtain a multi-channel electromyographic data set about gestures; the electrode array arrangement form in the multi-channel bracelet includes a 4×8 array or a 4×16 array; A filtering unit, used for filtering the multi-channel electromyography data set to obtain a first data set; A preprocessing unit is used to process the first data set in segments to extract valid gesture activity segments, and obtain gesture features from the extracted gesture activity segments to obtain a test data set, wherein the segmentation processing specifically includes: before collecting a multi-channel electromyographic data set of user gestures, pre-collecting the electromyographic signal of the user in a resting state to obtain a resting state threshold, and dividing the gesture activity segments according to the resting state threshold; A model acquisition unit, used to acquire a recognition model that has been trained for the user, wherein the recognition model is an algorithm model constructed based on hyperdimensional computing, and the test data set is input into the recognition model for gesture prediction. The recognition model is preset with a first memory and a second memory, and the second memory includes a sample hyperdimensional vector carrying a sample gesture; A prediction unit, in the recognition model, wherein the first memory encodes and maps the test data set into a hyperdimensional vector through addition and matrix multiplication operations, and performs similarity calculation with the sample hyperdimensional vector, finds the sample hyperdimensional vector with the highest similarity, and outputs the sample gesture corresponding to the sample hyperdimensional vector as the predicted gesture; Model training unit: Collect multi-channel electromyographic signals of a user at the arm to obtain a sample data set of multiple sample gestures of the user; Performing filtering on the sample data set to obtain a first training data set; Processing the first training data set in segments to extract valid gesture activity segments, extracting gesture features from the gesture activity segments, and marking corresponding gesture categories to obtain training set data; Construct an initial algorithm model based on hyperdimensional computing, and set the data dimension of the model to be a hyperdimensional vector; respectively initialize the first memory and the second memory in the initial algorithm model; The training data set is input into the initial algorithm model for training, the first memory encodes the gesture features of each channel and each gesture category in the training data set to obtain a hyperdimensional vector; the second memory is preset with the initial hyperdimensional vector of each gesture category after initialization; the hyperdimensional vector and the initial hyperdimensional vector of the corresponding gesture category are encoded to obtain a sample hyperdimensional vector carrying the sample gesture; After the training is completed, the second memory is polarized and the algorithm model is output to obtain a recognition model for the user.

3. A gesture recognition device, It is characterized in that include: An acquisition module, used to acquire multi-channel electromyographic signals from the user's arm; The main control module is integrated on the FPGA and electrically connected to the acquisition module, and is used to control the acquisition module and execute the gesture recognition method based on brain-like computing as claimed in claim 1.

Images