Gesture sensing system based on intelligent interaction

Abstract

The invention discloses a gesture sensing system based on intelligent interaction, which comprises a data acquisition module, an interactive gesture analysis module and an interactive sensing learning module, and is characterized in that the data acquisition module is used for acquiring man-machine interaction process data; the interaction gesture analysis module is used for analyzing gesture interaction actions in a man-machine interaction process, and the interaction perception learning module is used for carrying out learning control management on the intelligent interaction system according to a user gesture analysis and judgment result. The data acquisition module, the interactive gesture analysis module and the interactive perception learning module are electrically connected with one another, and the data acquisition module comprises a perception logic acquisition module, a gesture capture module, a timing unit and a withdrawal instruction recognition module. The sensing logic acquisition module is used for acquiring a preset gesture sensing logic program of the man-machine interaction system, and the system has the advantages of being high in precision and good in convenience.

Description

Technical Field

[0001] The present invention relates to the field of intelligent interaction technology, and in particular to a gesture perception system based on intelligent interaction. Background Art

[0002] Existing virtual reality (VR) or augmented reality (AR) human-computer interaction technologies are divided into two categories: gesture recognition and proprietary devices. Proprietary devices have advantages in operation feedback, input accuracy, and user experience, but proprietary devices must be configured for interaction. Different interactive scenarios require different devices, which increases costs. Gesture recognition brings the advantages of convenience and efficiency. Input operations based on hand movements bring the advantages of convenience and efficiency, but the user's hands lack feedback, which affects the accuracy and experience of input, and there are difficulties in judging complex input intentions. It cannot be effectively improved according to user habits, and long-term standardized operations are prone to fatigue and poor experience. Therefore, it is necessary to design a gesture perception system based on intelligent interaction with high accuracy and good convenience. Summary of the invention

[0003] The purpose of the present invention is to provide a gesture perception system based on intelligent interaction to solve the problems raised in the above background technology.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions: a gesture perception system based on intelligent interaction, comprising a data acquisition module, an interactive gesture analysis module and an interactive perception learning module, wherein the data acquisition module is used to collect human-computer interaction process data, the interactive gesture analysis module is used to analyze gesture interaction actions in the human-computer interaction process, and the interactive perception learning module is used to perform learning, control and management of the intelligent interaction system according to the user gesture analysis and judgment results, and the data acquisition module, the interactive gesture analysis module and the interactive perception learning module are electrically connected to each other.

[0005] According to the above technical solution, the data acquisition module includes a perception logic acquisition module, a gesture capture module, a timing unit and a withdrawal instruction recognition module. The perception logic acquisition module is used to obtain a preset gesture perception logic program of the human-computer interaction system, the gesture capture module is used to capture and collect user gesture images during the human-computer interaction process, the timing unit is used to perform timing during the human-computer interaction process, and the withdrawal instruction recognition module is used to collect withdrawal instructions during the human-computer interaction process.

[0006] According to the above technical solution, the interactive gesture analysis module includes a calibration module, a fast fitting line module, a sorting and analysis module, a perception matching module and a similarity judgment module. The calibration module is used to mark the feature points of the user gesture screen, the fast fitting line module is used to fit the marked activity trajectory in space, the sorting and analysis module is used to sort the marked points and their activity trajectory data, the perception matching module is used to match the sorted and analyzed gesture activity trajectory data with the gesture perception instructions of the human-computer interaction system one by one, and the similarity judgment module is used to make a similarity judgment on the sorted and analyzed gesture activity trajectory data.

[0007] According to the above technical solution, the interactive perception learning module includes a perception logic calibration module and a perception logic callback module. The perception logic calibration module is used to calibrate and debug the gesture perception logic used by the user in the human-computer interaction process according to the interactive gesture analysis results, and the perception logic callback module is used to track and call back the calibrated perception logic after learning the human-computer interaction process.

[0008] According to the above technical solution, the operation method of the gesture perception system based on intelligent interaction includes the following steps:

[0009] Step S1: When a user uses the human-computer interaction system, the system collects the perception logic of the connected human-computer interaction system through the perception logic collection module;

[0010] Step S2: The gesture capture module is started to capture the gestures of the user during the human-computer interaction process;

[0011] Step S3: When the user uses the human-computer interaction system to operate, the user's operation time and the perception instruction are identified by the timing unit and the withdrawal instruction recognition module respectively;

[0012] Step S4: the system retrieves the data collected in step S1 and step S2, and further analyzes the gesture interaction actions of the user during the human-computer interaction process;

[0013] Step S5: Based on the analysis results, the system learns the user's gesture interaction perception and debugs and controls the human-computer interaction system.

[0014] According to the above technical solution, step S4 further includes the following steps:

[0015] Step S41: During the human-computer interaction process, the system identifies the captured gesture action screen, identifies the features of the user's finger endpoints, knuckles, and wrists in the screen, and marks each feature point;

[0016] Step S42: Establish a three-dimensional space coordinate system, record the position of each marking point in real time, and obtain the coordinates of each marking point;

[0017] Step S43: When the user's gesture action changes, the fast fitting line module records and real-time fits the spatial activity trajectory of the marked points in the gesture according to the position change.

[0018] Step S44: Obtain the timing unit and the spatial activity trajectory, and organize and record the total moving distance l of all the marked point trajectories at each time point.

[0019] Step S45: Obtain the gesture instruction of the human-computer interaction system, and use the timing unit to match the corresponding fitting spatial activity trajectory at the time point of the gesture instruction.

[0020] Step S46: Centered on the time point where the gesture instruction is matched, query the effective gesture trajectory within 5 seconds before and after the central time point in the fitting activity trajectory. Among them, the effective gesture trajectory is the spatial activity trajectory where the total moving distance l of all the marked point trajectories at the unit time point is greater than the system preset value j and has not been matched by the human-computer interaction system.

[0021] Step S47: Determine the similarity between the effective gesture trajectory and the corresponding fitting spatial activity trajectory at the gesture instruction time point.

[0022] Step S48: According to the judgment result, the interactive gesture analysis module outputs a judgment signal to the interactive perception learning module.

[0023] According to the above technical solution, step S47 further includes the following steps:

[0024] Step S471: For the effective gesture trajectory and the corresponding fitting spatial activity trajectory at the gesture instruction time point, respectively obtain the coordinate positions of each marked point before and after the unit time point.

[0025] Step S472: Then, calculate the ratio value of the distance from each marked point after the unit time point to the wrist compared to the distance from each marked point before the unit time point in sequence, and then calculate the variance f1 of each ratio value corresponding to the effective gesture trajectory and the variance f2 of each ratio value corresponding to the corresponding fitting spatial activity trajectory at the gesture instruction time point.

[0026] Step S473: When 90%f2 < f1 < 110%f2, it is judged that there is similarity, otherwise it is judged that there is no similar gesture.

[0027] According to the above technical solution, step S5 further includes the following steps:

[0028] Step S51: When the obtained signal is a similarity judgment signal, the perception logic calibration module calibrates the perception logic of the human-computer interaction system to the gesture perception logic of the effective gesture trajectory in real time.

[0029] Step S52: Repeat step S51 to continuously learn and adapt the interactive gesture perception logic of the current user. At the same time, when the system collects the withdrawal command that appears twice during the user gesture interaction, the gesture perception logic is called back to the gesture perception logic of the previous learning stage through the perception logic callback module.

[0030] Compared with the prior art, the beneficial effect achieved by the present invention is as follows: the present invention can analyze interactive gestures during the operation of a user using a human-computer interaction system, firstly match and screen out gesture trajectories made in the human-computer interaction system for issuing gesture interaction control instructions to avoid repeated recognition operations, and then screen out trajectories in which the user's gestures have effective movement or displacement in the time period near the issuance of the gesture interaction instruction but are not input into the human-computer interaction system as instructions, and further analyze and judge them as effective gesture trajectories, and after fully identifying and analyzing the effective gesture trajectory, perceive and learn it as a preparatory gesture trajectory in the subsequent human-computer interaction of the current user, so that it can be effectively improved according to user habits, and at the same time effectively avoid the situation where the user's gestures are difficult to recognize when the standard is lowered due to fatigue after long-term standardized operation, and gesture perception during human-computer interaction can be performed with a relatively small gesture amplitude, achieving the effects of strong practicality, high accuracy and good convenience. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0032] Figure 1 It is a schematic diagram of the system module composition of the present invention. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] See also Figure 1 The present invention provides a technical solution: a gesture perception system based on intelligent interaction, including a data acquisition module, an interactive gesture analysis module and an interactive perception learning module. The data acquisition module is used to collect human-computer interaction process data, the interactive gesture analysis module is used to analyze gesture interaction actions in the human-computer interaction process, and the interactive perception learning module is used to learn, control and manage the intelligent interaction system according to the user gesture analysis and judgment results. The data acquisition module, the interactive gesture analysis module and the interactive perception learning module are electrically connected to each other.

[0035] The data acquisition module includes a perception logic acquisition module, a gesture capture module, a timing unit and a withdrawal instruction recognition module. The perception logic acquisition module is used to obtain the preset gesture perception logic program of the human-computer interaction system. The gesture capture module is used to capture and collect user gesture images during the human-computer interaction process. The timing unit is used to time the human-computer interaction process. The withdrawal instruction recognition module is used to collect withdrawal instructions during the human-computer interaction process.

[0036] The interactive gesture analysis module includes a calibration module, a fast fitting line module, a sorting and analysis module, a perception matching module and a similarity judgment module. The calibration module is used to mark the feature points of the user gesture screen, the fast fitting line module is used to fit the marked activity trajectory in space, the sorting and analysis module is used to sort the marked points and their activity trajectory data, the perception matching module is used to match the sorted and analyzed gesture activity trajectory data with the gesture perception instructions of the human-computer interaction system one by one, and the similarity judgment module is used to make similarity judgment on the sorted and analyzed gesture activity trajectory data.

[0037] The interactive perception learning module includes a perception logic calibration module and a perception logic callback module. The perception logic calibration module is used to calibrate and debug the gesture perception logic of the user during the human-computer interaction process according to the interactive gesture analysis results. The perception logic callback module is used to track and call back the calibrated perception logic after learning the human-computer interaction process.

[0038] The operation method of the gesture perception system based on intelligent interaction includes the following steps:

[0039] Step S1: When a user uses the human-computer interaction system, the system collects the perception logic of the connected human-computer interaction system through the perception logic collection module;

[0040] Step S2: The gesture capture module is started to capture the gestures of the user during the human-computer interaction process;

[0041] Step S3: When the user uses the human-computer interaction system to operate, the user's operation time and the perception instruction are identified by the timing unit and the withdrawal instruction recognition module respectively;

[0042] Step S4: the system retrieves the data collected in step S1 and step S2, and further analyzes the gesture interaction actions of the user during the human-computer interaction process;

[0043] Step S5: Based on the analysis results, the system learns the user's gesture interaction perception and debugs and controls the human-computer interaction system.

[0044] Step S4 further comprises the following steps:

[0045] Step S41: During the human-computer interaction process, the system identifies the captured gesture action screen, identifies the features of the user's finger endpoints, knuckles, and wrists in the screen, and marks each feature point;

[0046] Step S42: Establish a three-dimensional space coordinate system, record the position of each marking point in real time, and obtain the coordinates of each marking point;

[0047] Step S43: when the user's gesture changes, the fast fitting line module fits the spatial activity trajectory of the marker points in the gesture in real time according to the position change record;

[0048] Step S44: obtaining the timing unit and the spatial activity trajectory, and arranging and recording the total moving distance l of all the marked point trajectories at each time point;

[0049] Step S45: obtaining a gesture command of the human-computer interaction system, and matching the gesture command time point with the fitted spatial activity trajectory using a timing unit;

[0050] Step S46: Taking the time point matched to the gesture command as the center, query the valid gesture trajectory within 5 seconds before and after the center time point in the fitting activity trajectory, where the valid gesture trajectory is the spatial activity trajectory in which the sum of the moving distances l of all the marked points in a unit time point is greater than the system preset value j and is not matched by the human-computer interaction system;

[0051] Step S47: performing similarity judgment between the effective gesture trajectory and the corresponding fitting space activity trajectory of the gesture command time point;

[0052] Step S48: According to the judgment result, the interactive gesture analysis module outputs a judgment signal to the interactive perception learning module.

[0053] Step S47 further includes the following steps:

[0054] Step S471: fitting the spatial activity trajectory corresponding to the valid gesture trajectory and the gesture command time point, and obtaining the coordinates of each marked point before and after the unit time point respectively;

[0055] Step S472: Calculate the ratio of the distance from each mark point to the wrist after the unit time point to the distance from each mark point to the wrist before the unit time point, and then calculate the variance f1 of each ratio value corresponding to the effective gesture trajectory and the variance f2 of each ratio value corresponding to the fitting space activity trajectory corresponding to the gesture command time point;

[0056] Step S473: When 90%f2 < f1 < 110%f2, it is determined that there is similarity; otherwise, it is determined that there is no similar gesture. Through the above steps, during the operation of the human-computer interaction system by the user, the interaction gestures can be analyzed. First, the gesture trajectories made for issuing gesture interaction control instructions in the human-computer interaction system are matched and filtered out to avoid repeated recognition operations. Then, after screening the trajectories where the user has effective movement or displacement of the gesture within a period of time near the issued gesture interaction instruction but has not been input into the human-computer interaction system as an instruction, they are used as effective gesture trajectories for further analysis and judgment. By ignoring the gesture trajectory distance and deeply analyzing the change ratio of each detailed trajectory, when the change similarity between the effective gesture trajectory and the gesture trajectory of the issued gesture interaction instruction is high, it reflects that the selected effective gesture trajectory not only meets the adjacent time requirement but also meets the gesture trajectory change requirement. Then the system determines that this gesture is the behavior of the user attempting to issue a gesture interaction control instruction but not being perceived due to the gesture standard not conforming to the logic of the human-computer interaction system. After fully identifying and analyzing the effective gesture trajectory through the above steps, it is used as the preparatory gesture trajectory in the subsequent human-computer interaction of the current user for perception learning, so that it can be effectively improved according to the user's habits. At the same time, it can effectively avoid the situation where it is difficult to recognize when the standard of the user's gesture decreases after long-term standardized operation, and can perform gesture perception during human-computer interaction with a relatively small gesture amplitude, achieving the effects of strong practicability, high accuracy, and good convenience.

[0057] Step S5 further includes the following steps:

[0058] Step S51: When the obtained signal is a similarity judgment signal, the perception logic calibration module calibrates the perception logic of the human-computer interaction system to the gesture perception logic of the effective gesture trajectory in real time;

[0059] Step S52: Repeat Step S51, continuously learning and adapting to the interaction gesture perception logic of the current user. At the same time, when the system collects two consecutive withdrawal instructions during the user's gesture interaction, the gesture perception logic is called back to the gesture perception logic of the previous learning stage through the perception logic callback module; thus avoiding misjudgment caused by the increase in gesture perception sensitivity due to overlearning and improving the learning calibration accuracy.

[0060] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device.

[0061] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A gesture perception system based on intelligent interaction, comprising a data acquisition module, an interactive gesture analysis module and an interactive perception learning module, characterized in that: The data acquisition module is used to collect data of the human-computer interaction process, the interactive gesture analysis module is used to analyze the gesture interaction actions in the human-computer interaction process, and the interactive perception learning module is used to perform learning, control and management on the intelligent interactive system according to the user gesture analysis and judgment results. The data acquisition module, the interactive gesture analysis module and the interactive perception learning module are electrically connected to each other; The data acquisition module includes a perception logic acquisition module, a gesture capture module, a timing unit and a withdrawal instruction recognition module. The perception logic acquisition module is used to obtain a preset gesture perception logic program of the human-computer interaction system. The gesture capture module is used to capture and collect user gesture images during the human-computer interaction process. The timing unit is used to perform timing during the human-computer interaction process. The withdrawal instruction recognition module is used to collect withdrawal instructions during the human-computer interaction process. The interactive gesture analysis module includes a calibration module, a fast fitting line module, a sorting and analysis module, a perception matching module and a similarity judgment module; The interactive perception learning module includes a perception logic calibration module and a perception logic callback module. The perception logic calibration module is used to calibrate and debug the gesture perception logic of the user in the human-computer interaction process according to the interactive gesture analysis result. The perception logic callback module is used to call back the calibrated perception logic after tracking the human-computer interaction process. The operation method of the gesture perception system based on intelligent interaction includes the following steps: Step S1: When a user uses the human-computer interaction system, the system collects the perception logic of the connected human-computer interaction system through the perception logic collection module; Step S2: The gesture capture module is started to capture the gestures of the user during the human-computer interaction process; Step S3: When the user uses the human-computer interaction system to operate, the user's operation time and the perception instruction are identified by the timing unit and the withdrawal instruction recognition module respectively; Step S4: the system retrieves the data collected in step S1 and step S2, and further analyzes the gesture interaction actions of the user during the human-computer interaction process; Step S5: Based on the analysis results, the system learns the user's gesture interaction perception and debugs and controls the human-computer interaction system; The step S4 further comprises the following steps: Step S41: During the human-computer interaction process, the system identifies the captured gesture action screen, identifies the features of the user's finger endpoints, knuckles, and wrists in the screen, and marks each feature point; Step S42: Establish a three-dimensional space coordinate system, record the position of each marking point in real time, and obtain the coordinates of each marking point; Step S43: when the user's gesture changes, the fast fitting line module fits the spatial activity trajectory of the marker points in the gesture in real time according to the position change record; Step S44: obtaining the timing unit and the spatial activity trajectory, and arranging and recording the total moving distance l of all the marked point trajectories at each time point; Step S45: obtaining a gesture command of the human-computer interaction system, and matching the gesture command time point with the fitted spatial activity trajectory using a timing unit; Step S46: Centering on the time point when the gesture instruction is matched, query the effective gesture trajectories within 5 seconds before and after the central time point in the fitted activity trajectory. Herein, the effective gesture trajectory is a spatial activity trajectory where the total moving distance l of all marker point trajectories at a unit time point is greater than the system preset value j and has not been matched by the human-computer interaction system; Step S47: Determine the similarity between the effective gesture trajectory and the corresponding fitted spatial activity trajectory at the gesture instruction time point; Step S48: According to the judgment result, the interactive gesture analysis module outputs a judgment signal to the interactive perception learning module; The said Step S47 further includes the following steps: Step S471: For the effective gesture trajectory and the corresponding fitted spatial activity trajectory at the gesture instruction time point, respectively obtain the coordinate positions of each marker point before and after a unit time point; Step S472: Then, calculate the ratio of the distance from each marker point after a unit time point to the wrist to the distance from each marker point before a unit time point to the wrist in sequence, and then calculate the variance f1 of each ratio value corresponding to the effective gesture trajectory and the variance f2 of each ratio value corresponding to the fitted spatial activity trajectory at the gesture instruction time point; Step S473: When 90%f2 < f1 < 110%f2, it is determined that there is similarity, otherwise it is determined that there is no similar gesture.

Images