Gesture recognition system based on human-computer interaction

Abstract

The invention belongs to the field of gesture recognition, relates to a data analysis technology, and is used for solving the problem that a gesture recognition system in the prior art generally can only adopt a single recognition processing mode to carry out gesture recognition, in particular to a gesture recognition system based on human-computer interaction. Comprising a gesture information acquisition module, a recognition processing module and an instruction execution module, the gesture information acquisition module, the recognition processing module and the instruction execution module are sequentially in communication connection, and the recognition processing module is further in communication connection with a database; according to the method, recognition processing can be carried out on the recognition video of the user gesture, the processing modes comprise the standard splitting mode and the recognition splitting mode, the two splitting modes have the characteristic of reference deviation in the processing means, and when one splitting mode cannot complete recognition, the other splitting mode is directly adopted for recognition processing, so that the recognition processing efficiency is greatly improved. The probability of system identification failure is reduced, and the user experience is prevented from being affected by frequent gesture performance of the user.

Description

Technical Field

[0001] The present invention belongs to the field of gesture recognition, relates to data analysis technology, and specifically is a gesture recognition system based on human-computer interaction. Background Art

[0002] In computer science, gesture recognition is a topic that uses mathematical algorithms to identify human gestures. Gesture recognition can come from the movement of various parts of the human body, but generally refers to the movement of the face and hands. Users can use simple gestures to control or interact with devices, allowing computers to understand human behavior. Its core technologies are gesture segmentation, gesture analysis, and gesture recognition.

[0003] The gesture recognition system in the prior art can usually only use a single recognition processing method for gesture recognition. When recognition fails, a failure signal is directly generated and gesture information is re-collected for re-recognition. This recognition method has low recognition efficiency, frequent manual operations when recognition fails, and poor user experience.

[0004] In view of the above technical problems, this application proposes a solution. Summary of the invention

[0005] The purpose of the present invention is to provide a gesture recognition system based on human-computer interaction, which is used to solve the problem that the gesture recognition system in the prior art can usually only use a single recognition processing method to perform gesture recognition;

[0006] The technical problem to be solved by the present invention is: how to provide a gesture recognition system based on human-computer interaction that can adopt multiple recognition processing methods to perform gesture recognition.

[0007] The purpose of the present invention can be achieved through the following technical solutions:

[0008] A gesture recognition system based on human-computer interaction, comprising a gesture information acquisition module, a recognition processing module and an instruction execution module, wherein the gesture information acquisition module, the recognition processing module and the instruction execution module are sequentially communicatively connected, and the recognition processing module is also communicatively connected to a database;

[0009] The gesture information acquisition module is used to collect the recognition video of the user's gesture and send it to the recognition processing module;

[0010] The recognition processing module is used to perform recognition processing on the recognition video of the user's gesture: retrieve a standard data group through a database, the standard data group includes a number of standard data, each standard data includes a standard video and an execution instruction; randomly select a split mode from the standard split mode and the recognition split mode for recognition analysis; after the recognition split mode fails to be recognized, the standard split mode is used for recognition analysis; after the standard split mode fails to be recognized, the recognition split mode is used for recognition analysis; when both the recognition split mode and the standard split mode fail to be recognized, a re-collection signal is generated and the re-collection signal is sent to the gesture information collection module;

[0011] When the recognition is successful, the approximate recognition object is marked, and the execution instruction in the approximate recognition object is sent to the instruction execution module;

[0012] After receiving the execution instruction, the instruction execution module performs task execution control by executing the instruction.

[0013] Furthermore, the specific process of using the standard splitting mode to perform recognition analysis on the recognition video includes: splitting the standard video into standard images frame by frame i , i = 1, 2, ..., n, n is a positive integer, for the standard image i The analysis set is obtained by elimination processing; then the recognition video is split into recognition images frame by frame, and all recognition images corresponding to the same recognition video constitute a recognition set. The analysis set is compared and analyzed with the recognition set to obtain the effective coefficient of the standard data. After all the analysis sets of the standard data are compared with the recognition set, the standard data with the largest effective coefficient value is marked as the approximate recognition object, and whether the recognition is successful is determined by the effective coefficient of the approximate recognition object.

[0014] Furthermore, for the standard image i The specific process of culling includes: i Mark as analysis image, calculate standard image i+1 The degree of coincidence with the analyzed image is marked as the standard image i+1 The coincidence value i+1 , the coincidence value i+1 Compare with the preset overlap threshold: if the overlap value i+1 If the value is less than the overlap threshold, the standard image is retained. i+1 And the standard image i+1 Mark as a new analysis image; if the coincidence value i+1 If the overlap threshold is greater than or equal to the standard image i+1 Eliminate from the standard data; then calculate the standard image i+2 The degree of coincidence with the analysis image, and so on, until all standard images nThe images are discarded or marked as analysis images; the analysis images retained in the standard data constitute the analysis set of the standard data.

[0015] Furthermore, the process of obtaining the effective coefficient of the standard data includes: randomly selecting an analysis image from the analysis set and marking it as a comparison image, comparing all the recognition images in the recognition set with the comparison image to obtain the comparison value of the comparison image relative to all the recognition images, marking the maximum value of the comparison value of the comparison image relative to all the recognition images as the completion value of the comparison image, and comparing the completion value with a preset completion threshold: if the completion value is less than the completion threshold, the corresponding comparison image is marked as an invalid image; if the completion value is greater than or equal to the completion threshold, the corresponding comparison image is marked as a valid image; and the ratio of the number of valid images to the total number of elements in the analysis set is marked as the effective coefficient of the standard data.

[0016] Furthermore, the specific process of determining whether the recognition is successful includes: comparing the effective coefficient of the approximate recognition object with a preset effective threshold: if the effective coefficient is less than the effective threshold, the recognition is determined to have failed; if the effective coefficient is greater than or equal to the effective threshold, the execution instruction in the approximate recognition object is sent to the instruction execution module.

[0017] Furthermore, the specific process of using the recognition splitting mode to perform recognition analysis on the recognition video includes: splitting the recognition video into recognition images frame by frame e , i = 1, 2, ..., m, m is a positive integer, for the recognition image e The analysis set of the recognized video is obtained by elimination processing, and then the standard video is split into standard images frame by frame. All standard images constitute a standard set. The analysis set of the recognized video is compared with the standard set to obtain an approximate recognized object, and the execution instructions in the approximate recognized object are sent to the instruction execution module.

[0018] Furthermore, the recognition processing module is also communicatively connected to a mode management module, and the mode management module is used to perform priority management analysis on the splitting mode of user gesture recognition: generate a management cycle, obtain failure data SS and secondary data ES of the standard splitting mode within the management cycle; obtain the standard priority coefficient BX of the standard splitting mode through the formula YX=k1*SS+k2*ES, wherein k1 and k2 are both proportional coefficients, and k1>k2>1; obtain the recognition priority coefficient SY of the recognition splitting mode in the same manner as the standard priority coefficient BX of the standard splitting mode, mark the absolute value of the difference between the standard priority coefficient BY and the recognition priority coefficient SY as the priority necessary value, and mark the splitting mode to be preferentially adopted in the next management cycle through the priority necessary value.

[0019] Furthermore, the process of obtaining the failure data SS includes: marking the number of times the standard splitting mode is used for recognition and analysis within the management cycle as the execution value, and marking the ratio of the number of times the standard splitting mode is used for recognition but the recognition fails to the execution value as the failure data SS; the process of obtaining the secondary data ES includes: marking the number of times the user wakes up the gesture information collection module again for recognition video collection within L1 seconds after the recognition is completed and the ratio of the execution value as the secondary data ES.

[0020] Furthermore, the specific process of marking the splitting mode to be preferentially adopted in the next management cycle includes: comparing the priority necessary value with the preset priority necessary threshold: if the priority necessary value is less than the priority necessary threshold, then continuing to use a random method to select the splitting mode in the next management cycle; if the priority necessary value is greater than or equal to the priority necessary threshold, then comparing the identification priority coefficient SY with the standard priority coefficient BY: if the identification priority coefficient SY is greater than the standard priority coefficient BY, then the standard splitting mode is preferentially adopted for identification analysis in the next management cycle; if the identification priority coefficient SY is less than the standard priority coefficient BY, then the identification splitting mode is preferentially adopted for identification analysis in the next management cycle.

[0021] The present invention has the following beneficial effects:

[0022] 1. The recognition processing module can process the recognition video of the user's gesture. The processing modes include standard split mode and recognition split mode. The two split modes have the characteristics of baseline deviation in processing methods. When one split mode cannot complete the recognition, the other split mode is directly used for recognition processing to reduce the probability of system recognition failure and avoid users frequently performing gestures and affecting the user experience;

[0023] 2. The mode management module can be used to perform priority management analysis on the split mode of user gesture recognition, calculate the priority coefficients of the standard split mode and the recognition split mode in the management cycle respectively, and then mark the split mode with higher priority through numerical comparison results, thereby improving the system's one-time recognition success rate based on dual-mode split recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 is a system block diagram of Embodiment 1 of the present invention;

[0026] Figure 2 This is a system block diagram of Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0027] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] Embodiment 1

[0029] like Figure 1 As shown, a gesture recognition system based on human-computer interaction includes a gesture information acquisition module, a recognition processing module and an instruction execution module. The gesture information acquisition module, the recognition processing module and the instruction execution module are communicatively connected in sequence, and the recognition processing module is also communicatively connected to a database.

[0030] The gesture information acquisition module is used to collect the recognition video of the user's gesture and send it to the recognition processing module.

[0031] The recognition processing module is used to perform recognition processing on the recognition video of the user's gesture: retrieve a standard data group through a database, the standard data group includes a number of standard data, each standard data includes a standard video and an execution instruction; randomly select a split mode from the standard split mode and the recognition split mode for recognition analysis; the specific process of using the standard split mode to perform recognition analysis on the recognition video includes: splitting the standard video into standard images frame by frame i , i = 1, 2, ..., n, n is a positive integer, for the standard image i Perform elimination processing: standard image i Mark as analysis image, calculate standard image i+1 The degree of coincidence with the analyzed image is marked as the standard image i+1 The coincidence value i+1 , the coincidence value i+1 Compare with the preset overlap threshold: if the overlap value i+1 If the value is less than the overlap threshold, the standard image is retained. i+1 And the standard image i+1 Mark as a new analysis image; if the coincidence value i+1 If the overlap threshold is greater than or equal to the standard image i+1 Eliminate from the standard data; then calculate the standard image i+2 The degree of coincidence with the analysis image, and so on, until all standard images nThe images are removed or marked as analysis images; the analysis images retained in the standard data constitute the analysis set of the standard data; then the recognition video is split into recognition images frame by frame, and all recognition images corresponding to the same recognition video constitute the recognition set, and the analysis set is compared and analyzed with the recognition set: an analysis image is randomly selected from the analysis set and marked as the comparison image, and all recognition images in the recognition set are overlapped and compared with the comparison image to obtain the comparison value of the comparison image relative to all recognition images, and the maximum comparison value of the comparison image relative to all recognition images is marked as the completion value of the comparison image, and the completion value is compared with the preset completion threshold: if the completion value is less than the completion threshold, then The corresponding comparison image is marked as an invalid image; if the completion value is greater than or equal to the completion threshold, the corresponding comparison image is marked as a valid image; the ratio of the number of valid images to the total number of elements in the analysis set is marked as the effective coefficient of the standard data, and after all the analysis sets of all standard data are compared with the recognition set, the standard data with the largest effective coefficient value is marked as the approximate recognition object, and the effective coefficient of the approximate recognition object is compared with the preset effective threshold: if the effective coefficient is less than the effective threshold, it is determined that the recognition has failed, and the recognition split mode is used for recognition analysis; if the effective coefficient is greater than or equal to the effective threshold, the execution instruction in the approximate recognition object is sent to the instruction execution module.

[0032] The specific process of using the recognition split mode to perform recognition analysis on the recognition video includes: splitting the recognition video into recognition images frame by frame e , i = 1, 2, ..., m, m is a positive integer, for the recognition image e A elimination process is performed to obtain an analysis set of the recognition video, and then the standard video is split into standard images frame by frame. All standard images constitute a standard set, and the analysis set of the recognition video is compared and analyzed with the standard set to obtain an approximate recognition object, and the execution instruction in the approximate recognition object is sent to the instruction execution module; after the recognition split mode fails to be recognized, the standard split mode is adopted for recognition analysis; when both the recognition split mode and the standard split mode fail to be recognized, a re-collection signal is generated and sent to the gesture information collection module; the recognition processing module can perform recognition processing on the recognition video of the user's gesture, and the processing modes include the standard split mode and the recognition split mode. The two split modes have the characteristic of baseline deviation in terms of processing methods. When one split mode cannot complete the recognition, another split mode is directly adopted for recognition processing, thereby reducing the probability of system recognition failure and avoiding users from frequently performing gestures and affecting the user experience.

[0033] Embodiment 2

[0034] like Figure 2 As shown, the identification processing module is also communicatively connected to the mode management module.

[0035] The mode management module is used to perform priority management and analysis on the split mode of user gesture recognition: generate a management cycle, obtain the failure data SS and secondary data ES of the standard split mode within the management cycle; the acquisition process of the failure data SS includes: marking the number of times the standard split mode is used for recognition analysis within the management cycle as the execution value, and marking the ratio of the number of times the standard split mode is used for recognition but the recognition fails to the execution value as the failure data SS; the acquisition process of the secondary data ES includes: marking the ratio of the number of times the user wakes up the gesture information collection module again to collect recognition video within L1 seconds after the completion of the recognition to the execution value as the secondary data ES; L1 is a numerical constant, and the specific value of L1 is set by the management personnel; the standard priority coefficient BX of the standard split mode is obtained by the formula YB=k1*SS+k2*ES, wherein k1 and k2 are both proportional coefficients, and k1>k2>1; the recognition priority coefficient BX of the recognition split mode is obtained in the same way as the standard priority coefficient BX of the standard split mode. The coefficient SY marks the absolute value of the difference between the standard priority coefficient BY and the recognition priority coefficient SY as the priority necessary value, and compares the priority necessary value with the preset priority necessary threshold: if the priority necessary value is less than the priority necessary threshold, the random method will continue to be used to select the split mode in the next management cycle; if the priority necessary value is greater than or equal to the priority necessary threshold, the recognition priority coefficient SY will be compared with the standard priority coefficient BY: if the recognition priority coefficient SY is greater than the standard priority coefficient BY, the standard split mode will be given priority for recognition analysis in the next management cycle; if the recognition priority coefficient SY is less than the standard priority coefficient BY, the recognition split mode will be given priority for recognition analysis in the next management cycle; the split mode of user gesture recognition is subject to priority management analysis, and the corresponding priority coefficients of the standard split mode and the recognition split mode in the management cycle are calculated respectively, and then the split mode with higher priority is marked through the numerical comparison result, so as to improve the one-time recognition success rate of the system on the basis of dual-mode split recognition.

[0036] A gesture recognition system based on human-computer interaction collects recognition videos of user gestures and sends them to a recognition processing module. A splitting mode is randomly selected from a standard splitting mode and a recognition splitting mode for recognition analysis. When recognition is successful, an execution instruction of an approximate recognition object is output. When recognition fails, another splitting mode is used to continue recognition analysis.

[0037] The above contents are merely examples and explanations of the structure of the present invention. The technicians in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

[0038] The above formulas are obtained by collecting a large amount of data for software simulation and selecting a formula close to the real value. The coefficients in the formula are set by technicians in this field according to the actual situation; for example: formula YB = k1*SS+k2*ES; technicians in this field collect multiple groups of sample data and set corresponding standard priority coefficients for each group of sample data; substitute the set standard priority coefficients and the collected sample data into the formula, any two formulas constitute a set of two-variable linear equations, screen the calculated coefficients and take the average, and obtain the values ​​of k1 and k2 as 3.05 and 2.73 respectively;

[0039] The size of the coefficient is to quantify each parameter to obtain a specific value for subsequent comparison. The size of the coefficient depends on the amount of sample data and the technical personnel in this field preliminarily set the corresponding standard priority coefficient for each set of sample data; as long as it does not affect the proportional relationship between the parameter and the quantized value, such as the standard priority coefficient is proportional to the value of the failed data.

[0040] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0041] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A gesture recognition system based on human-computer interaction, characterized in that: It includes a gesture information acquisition module, a recognition processing module and an instruction execution module, wherein the gesture information acquisition module, the recognition processing module and the instruction execution module are sequentially connected in communication, and the recognition processing module is also connected in communication with a database; The gesture information acquisition module is used to collect the recognition video of the user's gesture and send it to the recognition processing module; The recognition processing module is used to perform recognition processing on the recognition video of the user's gesture: retrieve a standard data group through a database, the standard data group includes a number of standard data, each standard data includes a standard video and an execution instruction; randomly select a split mode from the standard split mode and the recognition split mode for recognition analysis; after the recognition split mode fails to be recognized, the standard split mode is used for recognition analysis; after the standard split mode fails to be recognized, the recognition split mode is used for recognition analysis; when both the recognition split mode and the standard split mode fail to be recognized, a re-collection signal is generated and the re-collection signal is sent to the gesture information collection module; When the recognition is successful, the approximate recognition object is marked, and the execution instruction in the approximate recognition object is sent to the instruction execution module; After receiving the execution instruction, the instruction execution module performs task execution control by executing the instruction.

2. The gesture recognition system based on human-computer interaction according to claim 1, characterized in that: The specific process of using the standard splitting mode to identify and analyze the recognition video includes: splitting the standard video into standard images frame by frame i , i = 1, 2, ..., n, n is a positive integer, for the standard image i The analysis set is obtained by elimination processing; then the recognition video is split into recognition images frame by frame, and all recognition images corresponding to the same recognition video constitute a recognition set. The analysis set is compared and analyzed with the recognition set to obtain the effective coefficient of the standard data. After all the analysis sets of the standard data are compared with the recognition set, the standard data with the largest effective coefficient value is marked as the approximate recognition object, and whether the recognition is successful is determined by the effective coefficient of the approximate recognition object.

3. The gesture recognition system based on human-computer interaction according to claim 2, characterized in that: For standard images i The specific process of culling includes: i Mark as analysis image, calculate standard image i+1 The degree of coincidence with the analyzed image is marked as the standard image i+1 The coincidence value i+1 , the coincidence value i+1 Compare with the preset overlap threshold: if the overlap value i+1 If the value is less than the overlap threshold, the standard image is retained. i+1 And the standard image i+1 Mark as a new analysis image; if the coincidence value i+1 If the overlap threshold is greater than or equal to the standard image i+1 Eliminate from the standard data; then calculate the standard image i+2 The degree of coincidence with the analysis image, and so on, until all standard images n The images are discarded or marked as analysis images; the analysis images retained in the standard data constitute the analysis set of the standard data.

4. The gesture recognition system based on human-computer interaction according to claim 3, characterized in that: The process of obtaining the effective coefficient of standard data includes: randomly selecting an analysis image from the analysis set and marking it as a comparison image, comparing all recognition images in the recognition set with the comparison image to obtain the comparison value of the comparison image relative to all recognition images, marking the maximum value of the comparison value of the comparison image relative to all recognition images as the completion value of the comparison image, and comparing the completion value with a preset completion threshold: if the completion value is less than the completion threshold, the corresponding comparison image is marked as an invalid image; if the completion value is greater than or equal to the completion threshold, the corresponding comparison image is marked as a valid image; and the ratio of the number of valid images to the total number of elements in the analysis set is marked as the effective coefficient of the standard data.

5. The gesture recognition system based on human-computer interaction according to claim 4, characterized in that: The specific process of determining whether the recognition is successful includes: comparing the effective coefficient of the approximate recognition object with a preset effective threshold: if the effective coefficient is less than the effective threshold, the recognition is determined to have failed; if the effective coefficient is greater than or equal to the effective threshold, the execution instruction in the approximate recognition object is sent to the instruction execution module.

6. The gesture recognition system based on human-computer interaction according to claim 5, characterized in that: The specific process of using the recognition split mode to perform recognition analysis on the recognition video includes: splitting the recognition video into recognition images frame by frame e , i = 1, 2, ..., m, m is a positive integer, for the recognition image e The analysis set of the recognized video is obtained by elimination processing, and then the standard video is split into standard images frame by frame. All standard images constitute a standard set. The analysis set of the recognized video is compared with the standard set to obtain an approximate recognized object, and the execution instructions in the approximate recognized object are sent to the instruction execution module.

7. The gesture recognition system based on human-computer interaction according to claim 6, characterized in that: The recognition processing module is also communicatively connected to a mode management module, and the mode management module is used to perform priority management analysis on the splitting mode of user gesture recognition: generate a management cycle, and obtain failure data SS and secondary data ES of the standard splitting mode within the management cycle; The standard priority coefficient BX of the standard splitting mode is obtained by the formula YX=k1*SS+k2*ES, wherein k1 and k2 are both proportional coefficients, and k1>k2>1; the identification priority coefficient SY of the identification splitting mode is obtained in the same way as the standard priority coefficient BX of the standard splitting mode, and the absolute value of the difference between the standard priority coefficient BY and the identification priority coefficient SY is marked as the priority necessary value, and the splitting mode to be adopted preferentially in the next management cycle is marked by the priority necessary value.

8. The gesture recognition system based on human-computer interaction according to claim 7, characterized in that: The process of obtaining the failure data SS includes: marking the number of times the standard splitting mode is used for recognition and analysis within the management cycle as the execution value, and marking the ratio of the number of times the standard splitting mode is used but the recognition fails to be performed to the execution value as the failure data SS; the process of obtaining the secondary data ES includes: marking the number of times the user wakes up the gesture information collection module again for recognition video collection within L1 seconds after the completion of the recognition to the ratio of the execution value as the secondary data ES.

9. The gesture recognition system based on human-computer interaction according to claim 8, characterized in that: The specific process of marking the splitting mode to be preferentially adopted in the next management cycle includes: comparing the priority necessary value with the preset priority necessary threshold: if the priority necessary value is less than the priority necessary threshold, then the splitting mode selection will continue to be performed randomly in the next management cycle; if the priority necessary value is greater than or equal to the priority necessary threshold, then the identification priority coefficient SY will be compared with the standard priority coefficient BY: if the identification priority coefficient SY is greater than the standard priority coefficient BY, then the standard splitting mode will be preferentially adopted for identification analysis in the next management cycle; if the identification priority coefficient SY is less than the standard priority coefficient BY, then the identification splitting mode will be preferentially adopted for identification analysis in the next management cycle.

Images