Action recognition method, interactive device, head-mounted display system and readable storage medium

Abstract

The embodiment of the present disclosure discloses a motion recognition method, an interactive device, a head-mounted display system and a readable storage medium. The interactive device comprises a pressure sensor array composed of a plurality of pressure sensors. The method comprises the following steps: acquiring a plurality of pressure signals output by the plurality of pressure sensors; determining a first matrix according to the pressure value of each pressure signal in the plurality of pressure signals; in the case that there is no reference matrix same as the first matrix in a reference matrix set, determining a target reference matrix matching the first matrix from the reference matrix set according to a preset fuzzy matching condition, wherein the reference matrix set comprises a plurality of reference matrices, and one reference matrix corresponds to one set action; determining a target action corresponding to the target reference matrix according to the association relationship between the matrix and the set action; and executing an operation instruction corresponding to the target action.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of head-mounted display devices, and more particularly, to a motion recognition method, an interactive device, a head-mounted display system, and a readable storage medium. BACKGROUND

[0002] A head-mounted display device (HMD) can realize different effects such as virtual reality (VR), augmented reality (AR), and mixed reality (MR). In order to provide a better game experience for users, higher requirements are put forward for the performance of HMD products and handles used in conjunction therewith.

[0003] In related technologies, handles used in conjunction with HMD products usually adopt mechanical keys or touch keys. However, for mechanical keys, there are problems such as poor control experience, loud noise, and limited service life. SUMMARY

[0004] Embodiments of the present disclosure aim to provide a motion recognition method, an interactive device, a head-mounted display system, and a readable storage medium, so as to recognize a set motion of a user through a pressure sensor array, and to improve the accuracy and success rate of motion recognition.

[0005] According to a first aspect of embodiments of the present disclosure, a motion recognition method is provided, applied to an interactive device, the interactive device including a pressure sensor array composed of a plurality of pressure sensors, and the method includes:

[0006] obtaining a plurality of pressure signals output by the plurality of pressure sensors;

[0007] determining a first matrix according to a pressure value of each pressure signal in the plurality of pressure signals;

[0008] in a case where there is no reference matrix identical to the first matrix in a reference matrix set, determining a target reference matrix matching the first matrix from the reference matrix set according to a preset fuzzy matching condition, wherein the reference matrix set includes a plurality of reference matrices, and one reference matrix corresponds to one set motion;

[0009] determining a target motion corresponding to the target reference matrix according to an association relationship between matrices and set motions, and executing an operation instruction corresponding to the target motion.

[0010] Optionally, the determining of the target reference matrix matching the first matrix from the reference matrix set according to the preset fuzzy matching condition includes:

[0011] obtaining a difference degree of each reference matrix in the reference matrix set from the first matrix;

[0012] determining a reference matrix in the reference matrix set as a target reference matrix matched with the first matrix, if the difference degree of the reference matrix meets the fuzzy matching condition.

[0013] Optionally, the difference degree comprises at least one of a number of difference points and an offset amount.

[0014] The difference point is a detection position where a pressure value in the reference matrix is different from a pressure value in the first matrix.

[0015] Optionally, the difference degree is the number of difference points; and the determining the reference matrix in the reference matrix set as the target reference matrix matched with the first matrix, if the difference degree of the reference matrix meets the fuzzy matching condition, comprises:

[0016] determining a reference matrix in the reference matrix set as the target reference matrix matched with the first matrix, if a number of difference points of the reference matrix is less than a predetermined number.

[0017] The difference point is a detection position where a pressure value in the reference matrix is different from a pressure value in the first matrix.

[0018] Optionally, the difference degree is the offset amount; and the determining the reference matrix in the reference matrix set as the target reference matrix matched with the first matrix, if the difference degree of the reference matrix meets the fuzzy matching condition, comprises:

[0019] determining a reference matrix in the reference matrix set as the target reference matrix matched with the first matrix, if an offset amount of the reference matrix is less than a predetermined offset amount.

[0020] Optionally, the predetermined offset amount is one row or one column.

[0021] In a case where the predetermined offset amount is one row, a pressure value in an i-th row of the first matrix corresponds to a pressure value in an i+1-th row or an i-1-th row of the target reference matrix.

[0022] In a case where the predetermined offset amount is one column, a pressure value in an i-th column of the first matrix corresponds to a pressure value in an i+1-th column or an i-1-th column of the target reference matrix.

[0023] Optionally, before the determining the target action corresponding to the target reference matrix according to the association relationship between the matrix and the set action, the method further comprises:

[0024] In a case where there is a same reference matrix as the first matrix in the reference matrix set, the same reference matrix as the first matrix is determined as a target reference matrix.

[0025] Optionally, the step of acquiring each reference matrix in the reference matrix set comprises:

[0026] issuing prompt information corresponding to the setting action;

[0027] In a case where a setting action performed by a user in response to the prompt information, a plurality of reference pressure signals output by a plurality of sensors are acquired;

[0028] According to the plurality of reference pressure signals, a reference matrix corresponding to the setting action is determined.

[0029] According to a second aspect of the embodiments of the present disclosure, an interactive device is provided, comprising a pressure sensor array composed of a plurality of pressure sensors, and the interactive device further comprises:

[0030] a memory for storing executable computer instructions;

[0031] a processor for executing the action recognition method according to the first aspect of the embodiments of the present disclosure under the control of the executable computer instructions;

[0032] The plurality of pressure sensors are connected to the processor to output the plurality of pressure signals collected to the processor.

[0033] According to a third aspect of the embodiments of the present disclosure, a head-mounted display system is provided, comprising:

[0034] a head-mounted display device;

[0035] The interactive device according to the second aspect of the embodiments of the present disclosure is in communication connection with the head-mounted display device.

[0036] According to a fourth aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, which stores computer instructions readable and executable by a computer, and when the computer instructions are run by a processor, the action recognition method according to the first aspect of the embodiments of the present disclosure is executed,

[0037] According to the embodiment of the present disclosure, in the process of identifying the target action performed by the user through the interaction device, the first matrix can be determined according to the plurality of pressure signals output by the plurality of pressure sensors in the pressure sensor array, and the first matrix is matched with the reference matrix set established in advance. In the case where there is no reference matrix same as the first matrix in the reference matrix set, the target reference matrix matching the first matrix can be determined from the reference matrix set according to the preset fuzzy matching condition, and then the target action corresponding to the target reference matrix can be determined, and the operation instruction corresponding to the target action is executed. In this way, the embodiment can identify the action performed by the user based on the pressure sensor array, and the comfort of operation is better than that of the traditional mechanical key. Moreover, the action recognition method provided by the embodiment supports the fuzzy matching mode, which can improve the success rate of recognition while ensuring the accuracy of recognition, accurately identify the target action of the user when the holding position or holding force of the user changes slightly, and thus provide better user experience.

[0038] Other features and advantages of the embodiments of the present disclosure will become more apparent from the following detailed description of exemplary embodiments of the embodiments of the present disclosure, with reference to the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings needed in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some of the embodiments of the present disclosure, and therefore should not be considered as limiting the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0040] Figure 1 is a flowchart of an action recognition method according to an embodiment;

[0041] Figure 2 is a hardware structure schematic diagram of an action recognition device according to an embodiment;

[0042] Figure 3 is a hardware structure schematic diagram of an interaction device according to an embodiment;

[0043] Figure 4 is a hardware structure schematic diagram of a head-mounted display system according to an embodiment. DETAILED DESCRIPTION

[0044] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. Note that the relative arrangement, numerical expressions, and numerical values of the components and steps set forth in these embodiments are not limiting the scope of the embodiments of the present disclosure unless otherwise specifically stated.

[0045] The following description of at least one exemplary embodiment is merely exemplary in nature and is in no way intended to limit the disclosure, its application, or uses.

[0046] Techniques, methods, and devices known to those of ordinary skill in the relevant art can not be discussed in detail herein. However, where appropriate, such techniques, methods, and devices can be considered part of the description of the present disclosure.

[0047] In all of the examples shown and discussed herein, any specific values should be interpreted as merely exemplary, and not as a limitation. Thus, other examples of exemplary embodiments can have different values.

[0048] It should be noted that like reference numerals and letters refer to like items throughout the attached drawings, and thus, once an item is defined in one drawing, it is not necessary that it be further discussed in the remaining drawings.

[0049] Below, various embodiments and examples according to the present disclosure are described with reference to the accompanying drawings.

[0050] Embodiments of the present disclosure provide a motion recognition method, which is implemented by an interactive device. Below, first, an interactive device for implementing the motion recognition method is introduced.

[0051] The interactive device includes a device body, and a pressure sensor array is disposed on the device body, the pressure sensor array being composed of a plurality of pressure sensors. Exemplarily, the plurality of pressure sensors are distributed in a matrix manner in a holding region of the interactive device.

[0052] In the present embodiment, the pressure sensor array is composed of a plurality of pressure sensors. The pressure sensor array can be a rectangular array or a circular array. For example, the pressure sensor array includes 9 pressure sensors arranged in a 3x3 matrix. For another example, the pressure sensor array includes 25 pressure sensors arranged in a 5x5 matrix. For yet another example, the pressure sensor array includes 18 pressure sensors arranged in a 3x6 matrix. It can be understood herein that the more the number of pressure sensors included in the pressure sensor array, the higher the detection accuracy. Those skilled in the art can set the type of the pressure sensor array and the number of pressure sensors according to actual needs, and the present embodiment does not limit this.

[0053] In the embodiment, when the user performs a setting action using the interactive device, the user's hand forms pressure on the plurality of pressure sensors of the holding area of the interactive device. Further, when the user performs different setting actions, the user's hand forms pressure on different positions of the pressure sensors of the pressure sensor array, and the pressure sensors at different positions sense different pressure values. For example, when the user performs a shooting action, the user's index finger forms greater pressure on the corresponding positions of the plurality of pressure sensors. Based on this, the plurality of pressure sensors in the pressure sensor array can sense the pressure applied by different parts of the user's hand on the interactive device, and convert the pressure signals into electrical signals and output to the processor. The processor can obtain a pressure value matrix corresponding to the pressure sensor array according to the electrical signals output by the pressure sensor array, so as to identify the setting action performed by the user according to the pressure value matrix.

[0054] In the embodiment, the interactive device can further include a power module connected with the processor and the pressure sensor array respectively, for supplying power to the processor and the pressure sensor array.

[0055] In the embodiment, the interactive device can be a handle used in cooperation with a head-mounted display device.

[0056] The action recognition method provided by the embodiment of the disclosure will be described below based on the above-mentioned interactive device.

[0057] Figure 1 A flowchart of an action recognition method of one embodiment of the disclosure is shown, which is applied to the interactive device described in the above-mentioned embodiment. The interactive device includes a pressure sensor array composed of a plurality of pressure sensors, and the plurality of pressure sensors are distributed on the holding area of the interactive device in a matrix arrangement.

[0058] As shown in Figure 1 The action recognition method provided by the embodiment can include the following steps S1100-S1400.

[0059] In step S1100, a plurality of pressure signals output by the plurality of pressure sensors are obtained.

[0060] In the embodiment, the plurality of pressure sensors constituting the pressure sensor array are distributed on the interactive device in a matrix arrangement. For example, the interactive device is provided with a pressure sensor array composed of 18 pressure sensors arranged in a 3x6 matrix.

[0061] The plurality of pressure sensors are used to sense the pressure values formed by different positions of the user's hand on the interactive device.

[0062] In step S1200, a first matrix is determined according to the pressure values of each pressure signal in the plurality of pressure signals.

[0063] The first matrix is a pressure value matrix composed of pressure values of pressure signals collected by the plurality of pressure sensors. The first matrix can reflect the magnitude of the pressure formed by the user's hand on the interactive device sensed by the pressure sensors at different positions in the pressure sensor array.

[0064] Specifically, the pressure values of the pressure signals collected by each pressure sensor are combined according to the position of each pressure sensor in the pressure sensor array to obtain the first matrix.

[0065] Taking a 3x6 matrix as an example, the first matrix is:

[0066] In step S1300, in the case that there is no reference matrix identical to the first matrix in the reference matrix set, a target reference matrix matching the first matrix is determined from the reference matrix set according to a preset fuzzy matching condition, wherein the reference matrix set includes a plurality of reference matrices, and one reference matrix corresponds to one set action.

[0067] In this embodiment, the reference matrix set can be a pressure value matrix set established by the user in advance for identifying set actions. The reference matrix set can be composed of a plurality of reference matrices corresponding to a plurality of set actions, wherein one set action corresponds to one reference matrix. The set actions are, for example, shooting actions, sword swinging actions, punching actions, etc.

[0068] In this embodiment, when the user uses the interactive device to perform a set action, the user's hand will form pressure on the plurality of pressure sensors of the holding area of the interactive device. Further, when the user performs different set actions, the user's hand will form pressure on the pressure sensors at different positions in the pressure sensor array, and the pressure sensors at different positions will sense different magnitudes of pressure. For example, when the user performs a shooting action, the index finger of the user will exert greater pressure on several pressure sensors at the corresponding positions. Based on this, before determining the target reference matrix matching the first matrix using the reference matrix set, the step of establishing the reference matrix set is further included.

[0069] In one embodiment, the step of obtaining each reference matrix in the reference matrix set can further include: issuing prompt information corresponding to a set action, and in response to the user performing the set action for the prompt information, obtaining a plurality of reference pressure signals output by a plurality of sensors, and determining a reference matrix corresponding to the set action according to the plurality of reference pressure signals.

[0070] In this embodiment, the prompt information is used to prompt the user to perform a set action. The prompt information can be "please make a shooting action". Exemplarily, the prompt information can be issued by the interactive device, or can be issued by the head-mounted display device in communication connection with the interactive device.

[0071] The reference matrix corresponding to the set action can reflect the size of the pressure signal that the pressure sensor at different positions in the pressure sensor array should output when the user performs the set action. Matching the first matrix with the reference matrix can identify the set action corresponding to the first matrix.

[0072] Specifically, the reference matrix corresponding to the set action can be, in a mode of setting the reference matrix, when the user performs the set action, acquiring the reference pressure value of the reference pressure signal output by each pressure sensor in the pressure sensor array, combining the reference pressure value of the reference pressure signal collected by each pressure sensor according to the position of each pressure sensor in the pressure sensor array, to obtain the reference matrix corresponding to the set action. Then, each reference matrix is stored in association with the set action corresponding thereto.

[0073] For example, the reference matrix corresponding to the shooting action is:

[0074] It should be noted that when the user first uses the interactive device, the setting mode of the reference matrix can be entered through the head-mounted display device in communication connection with the interactive device, and the corresponding reference matrix for each set action can be set according to the prompt issued by the head-mounted display device and the above process, to obtain the reference matrix set. In addition, when other users use the interactive device, the reference matrix set matched with the user can also be set.

[0075] According to this embodiment, the user can establish the reference matrix set according to the prompt information, so as to realize the identification of the target action performed by the user based on the reference matrix set in the process of using the interactive device, which can improve the success rate of action recognition. In addition, different users can establish the reference matrix set according to their own use habits, which can further improve the accuracy of action recognition, thereby improving the user experience.

[0076] In the embodiment, on one hand, when the user performs the setting action through the interactive device, there is a certain offset between the actual holding position of the user and the holding position when the reference matrix corresponding to the setting action is constructed. On the other hand, when the user performs the setting action through the interactive device, the pressure value of the pressure signal output by one or more pressure sensors in the pressure sensor array is slightly different from the pressure value of the reference pressure signal output when the reference matrix corresponding to the setting action is constructed. Based on this, in order to improve the success rate of recognition, a fuzzy matching condition can be set, so that in the case where there is no reference matrix identical to the first matrix in the reference matrix set, the reference matrix closest to the first matrix is determined from the reference matrix set as the target reference matrix matched with the first matrix based on the fuzzy matching condition. The process of determining the target reference matrix according to the fuzzy matching condition is described below with specific embodiments.

[0077] In one embodiment, the determining, according to a preset fuzzy matching condition, of the target reference matrix matched with the first matrix from the reference matrix set can further include: obtaining a difference degree of each reference matrix in the reference matrix set from the first matrix; and determining, as the target reference matrix matched with the first matrix, the reference matrix in the reference matrix set that has the difference degree meeting the fuzzy matching condition.

[0078] In the embodiment, in the process of recognizing the action performed by the user through the interactive device, in the case where there is no reference matrix identical to the first matrix in the reference matrix set, the target reference matrix matched with the first matrix can be determined according to the difference degree of the first matrix from each reference matrix in the reference matrix set, so as to recognize the target action performed by the user, which can improve the sensitivity and accuracy of recognition, so that the action recognition process of the interactive device has a certain recognition tolerance, and the user experience is better.

[0079] Optionally, the difference degree of the reference matrix from the first matrix can be the number of difference points in the reference matrix, where the difference point is a detection position where the pressure value in the reference matrix is different from that in the first matrix. More specifically, the difference point can be a detection position where the pressure value in the reference matrix is different from that in the first matrix and the difference is less than a first threshold. The first threshold can be set according to simulation test results and actual requirements, which is not limited in the embodiments of the present disclosure.

[0080] In the embodiment, when the user performs a setting action through the interactive device, the pressure value of the pressure signal output by one or more pressure sensors in the pressure sensor array has a small difference with the pressure value of the reference pressure signal output when the reference matrix corresponding to the setting action is constructed. In this case, the fuzzy matching condition can be set according to the number of the number of allowed difference points, so as to determine the target reference matrix matched with the first matrix according to the fuzzy matching condition.

[0081] In one embodiment, in the case where the difference degree of the reference matrix and the first matrix is the number of difference points, the reference matrix set in which the difference degree meets the fuzzy matching condition is determined as the target reference matrix matched with the first matrix, which can further include: determining the reference matrix in which the number of difference points is less than a predetermined number in the reference matrix set as the target reference matrix matched with the first matrix; wherein the difference point is a detection position where the pressure values in the reference matrix and the first matrix are different.

[0082] The predetermined number can be the number of allowed differences between the pressure values at the corresponding positions in the first matrix and the reference matrix. For example, the predetermined number is 3.

[0083] In a specific implementation, in the case where there is no reference matrix identical to the first matrix in the reference matrix set, the number of abnormal points where the pressure values in each reference matrix in the reference matrix set and the first matrix are different is determined, and the reference matrix in which the number of difference points is less than a predetermined number is determined as the target reference matrix matched with the first matrix.

[0084] For example, when the user performs a shooting action, the first matrix obtained is: The reference matrix corresponding to the shooting action is: The number of abnormal points where the pressure values in the first matrix and the reference matrix corresponding to the shooting action are different is 1, and the reference matrix corresponding to the shooting action can be determined as the target reference matrix matched with the first matrix.

[0085] In the embodiment, in the case where there is no reference matrix identical to the first matrix in the reference matrix set, the number of abnormal points where the pressure values in each reference matrix in the reference matrix set and the first matrix are different is determined, and the reference matrix in which the number of difference points is less than a predetermined number is determined as the target reference matrix matched with the first matrix. In this way, when the holding force changes slightly when the user uses the interactive device, the success rate of action recognition can be improved through fuzzy matching, thereby improving the game experience of the user.

[0086] Alternatively, the difference degree of the reference matrix and the first matrix can also be the offset of the first matrix relative to the reference matrix.

[0087] In the embodiment, when the user performs the setting action through the interactive device, there is a certain deviation between the actual holding position of the user and the holding position when the user holds the reference matrix corresponding to the setting action. In this case, the fuzzy matching condition can be set according to the allowed deviation, so as to determine the target reference matrix matched with the first matrix according to the fuzzy matching condition.

[0088] In one embodiment, the difference between the reference matrix and the first matrix is the deviation; and the reference matrix in the reference matrix set, the difference of which conforms to the fuzzy matching condition, can be further determined as the target reference matrix matched with the first matrix.

[0089] The predetermined deviation can be the minimum deviation of the first matrix relative to the reference matrix. For example, the predetermined deviation is one row or one column.

[0090] In a specific implementation, in the case where there is no reference matrix identical to the first matrix in the reference matrix set, the deviation of each reference matrix in the reference matrix set relative to the first matrix can be obtained, and the reference matrix in the reference matrix set, the deviation of which is less than the predetermined deviation, can be determined as the target reference matrix matched with the first matrix.

[0091] In an optional embodiment, in the case where the predetermined deviation is one row, the pressure value of the i-th row in the first matrix corresponds to the pressure value of the i+1-th row or the i-1-th row in the target reference matrix.

[0092] For example, it is assumed that the reference matrix corresponding to the shooting action is If the first matrix is determined as according to the plurality of pressure signals output by the plurality of pressure sensors, the pressure value of the 2nd row in the first matrix corresponds to the pressure value of the 1st row of the reference matrix corresponding to the shooting action, and the pressure value of the 3rd row in the first matrix corresponds to the pressure value of the 2nd row of the reference matrix corresponding to the shooting action, that is, the first matrix is moved down by one row relative to the reference matrix corresponding to the shooting action, and the reference matrix corresponding to the shooting action is determined as the target reference matrix matched with the first matrix according to the fuzzy matching condition.

[0093] For example, it is assumed that the reference matrix corresponding to the shooting action is The pressure value in the first matrix corresponding to the first row is the pressure value in the reference matrix corresponding to the second row of the shooting action, the pressure value in the first matrix corresponding to the second row is the pressure value in the reference matrix corresponding to the third row of the shooting action, that is, the first matrix is moved one row up relative to the reference matrix corresponding to the shooting action, and the reference matrix corresponding to the shooting action is determined as the target reference matrix matched with the first matrix according to the fuzzy matching condition.

[0094] In another optional embodiment, when the predetermined offset is one column, the pressure value in the i-th column of the first matrix corresponds to the pressure value in the i+1-th column or the i-1-th column of the target reference matrix.

[0095] For example, assuming that the reference matrix corresponding to the shooting action is If the first matrix is determined according to the plurality of pressure signals output by the plurality of pressure sensors The pressure value in the i+1-th column of the first matrix corresponds to the pressure value in the i-th row of the reference matrix corresponding to the shooting action, that is, the first matrix is moved one column to the right relative to the reference matrix corresponding to the shooting action, and the reference matrix corresponding to the shooting action is determined as the target reference matrix matched with the first matrix according to the fuzzy matching condition.

[0096] For example, continuing to take the shooting action as an example, if the first matrix is determined according to the plurality of pressure signals output by the plurality of pressure sensors The pressure value in the i-th column of the first matrix corresponds to the pressure value in the i+1-th column of the reference matrix corresponding to the shooting action, that is, the first matrix is moved one column to the left relative to the reference matrix corresponding to the shooting action, and the reference matrix corresponding to the shooting action is determined as the target reference matrix matched with the first matrix according to the fuzzy matching condition.

[0097] In this embodiment, when there is no reference matrix identical to the first matrix in the reference matrix set, the offset of each reference matrix in the reference matrix set relative to the pressure value in the first matrix is obtained, so that the target reference matrix matched with the first matrix is determined from the reference matrix set according to the offset of each reference matrix relative to the pressure value in the first matrix, so that when the holding position of the user using the interactive device is slightly offset, the target action made by the user can still be recognized through fuzzy matching, the success rate of action recognition can be improved, and the game experience of the user can be improved.

[0098] In one embodiment, before the target action corresponding to the target reference matrix is determined according to the association between the matrix and the set action, the method further includes: when there is a reference matrix identical to the first matrix in the reference matrix set, the reference matrix identical to the first matrix is determined as the target reference matrix.

[0099] In the embodiment, when there is a same reference matrix as the first matrix in the reference matrix set, the reference matrix is directly taken as the target reference matrix matched with the first matrix when identifying the action performed by the user through the interaction device. When there is no same reference matrix as the first matrix in the reference matrix set, the target reference matrix matched with the first matrix is determined from the reference matrix set through the fuzzy matching condition. The embodiment supports multiple matching modes, and can improve the accuracy and success rate of identification.

[0100] After step S1300, step S1400 is performed, the target action corresponding to the target reference matrix is determined according to the association relationship between the matrix and the set action, and the operation instruction corresponding to the target action is performed.

[0101] In specific implementation, the association relationship between the matrix and the set action can be established in advance, the target action corresponding to the target reference matrix is determined based on the association relationship between the matrix and the set action, and then the identification information corresponding to the target action is sent to the head-mounted display device in communication connection with the interaction device, so that the head-mounted display device performs the operation instruction corresponding to the target action.

[0102] According to the embodiments of the present disclosure, in the process of identifying the target action performed by the user through the interaction device, the first matrix can be determined according to the multiple pressure signals output by the multiple pressure sensors in the pressure sensor array, and the first matrix is matched with the reference matrix set established in advance. In the case that there is no same reference matrix as the first matrix in the reference matrix set, the target reference matrix matched with the first matrix can be determined from the reference matrix set according to the preset fuzzy matching condition, and then the target action corresponding to the target reference matrix can be determined, and the operation instruction corresponding to the target action is performed. In this way, the embodiment can identify the action performed by the user based on the pressure sensor array, and the comfort degree of operation is better than that of the traditional mechanical key. Moreover, the action recognition method provided by the embodiment supports the fuzzy matching mode, can improve the success rate of identification in the case of ensuring the accuracy of identification, can accurately identify the target action of the user when the holding position or the holding force of the user changes slightly, and thus provides better use experience for the user.

[0103] The embodiments of the present disclosure provide an action recognition device applied to the interaction device described in the above embodiments. The interaction device includes a pressure sensor array composed of multiple pressure sensors, and the multiple pressure sensors are distributed in a holding area of the interaction device in a matrix arrangement manner. As shown in the figure, the action recognition device 200 can include an acquisition module 210, a determination module 220, a matching module 230 and an execution module 240. Figure 2 ​

[0104] The acquisition module 210 is configured to acquire a plurality of pressure signals output by the plurality of pressure sensors.

[0105] The determination module 220 is configured to determine a first matrix according to a pressure value of each of the plurality of pressure signals.

[0106] The matching module 230 is configured to, in a case where there is no reference matrix identical to the first matrix in a reference matrix set, determine a target reference matrix matching the first matrix from the reference matrix set according to a preset fuzzy matching condition, wherein the reference matrix set includes a plurality of reference matrices, and one reference matrix corresponds to one set action.

[0107] The execution module 240 is configured to determine a target action corresponding to the target reference matrix according to an association between the matrix and the set action, and execute an operation instruction corresponding to the target action.

[0108] In an embodiment, the matching module 230 can further include:

[0109] An acquisition unit is configured to acquire a difference degree of each reference matrix in the reference matrix set from the first matrix.

[0110] A matching unit is configured to determine, as a target reference matrix matching the first matrix, a reference matrix in the reference matrix set that has a difference degree conforming to the fuzzy matching condition.

[0111] In an embodiment, the difference degree includes at least one of a number of difference points and an offset.

[0112] The difference point is a detection position where the pressure value in the reference matrix is different from that in the first matrix.

[0113] In an embodiment, the difference degree is the number of difference points, and the matching unit is specifically configured to determine, as the target reference matrix matching the first matrix, a reference matrix in the reference matrix set that has a number of difference points less than a predetermined number. The difference point is a detection position where the pressure value in the reference matrix is different from that in the first matrix.

[0114] In an embodiment, the difference degree is the offset, and the matching unit is specifically configured to determine, as the target reference matrix matching the first matrix, a reference matrix in the reference matrix set that has an offset less than a predetermined offset.

[0115] In an embodiment, the predetermined offset is one row or one column.

[0116] In a case where the predetermined offset is one row, the pressure value of the ith row in the first matrix corresponds to the pressure value of the i+1th row or the i-1th row in the target reference matrix.

[0117] In a case where the predetermined offset is one column, the pressure value of the ith column in the first matrix corresponds to the pressure value of the i+1th column or the i-1th column in the target reference matrix.

[0118] In an embodiment, before determining the target action corresponding to the target reference matrix according to the association between the matrix and the set action, the matching module 230 is further configured to, in a case where there is a reference matrix identical to the first matrix in the reference matrix set, determine the reference matrix identical to the first matrix as the target reference matrix.

[0119] In an embodiment, the step of obtaining each reference matrix in the reference matrix set comprises: issuing prompt information corresponding to a set action; in a case where a set action performed by a user in response to the prompt information, obtaining a plurality of reference pressure signals output by a plurality of sensors; and determining a reference matrix corresponding to the set action according to the plurality of reference pressure signals.

[0120] According to the embodiments of the present disclosure, the embodiments can identify an action performed by a user based on a pressure sensor array, and the comfort of operation is better than that of a traditional mechanical key. Moreover, the action identification method provided by the embodiments supports a fuzzy matching mode, can improve the success rate of identification while ensuring the accuracy of identification, can accurately identify a target action of a user when a holding position or a holding force of the user changes slightly, and thus provides a better user experience.

[0121] The embodiments of the present disclosure further provide an interactive device, as shown in the accompanying drawings, the interactive device 300 can comprise a memory 310, a processor 320 and a pressure sensor array 330. Figure 3

[0122] The memory 310 can be used to store executable computer instructions.

[0123] The processor 320 can be used to control the execution of the action identification method according to the embodiments of the method of the present disclosure according to the executable computer instructions.

[0124] The pressure sensor array 330 is composed of a plurality of pressure sensors, and the plurality of pressure sensors are distributed in a matrix manner in a holding area of the interactive device. The plurality of pressure sensors are connected to the processor to output a plurality of pressure signals collected by the plurality of pressure sensors to the processor.

[0125] ​In an embodiment, each module of the above action recognition apparatus 200 can be implemented by the processor 320 running computer instructions stored in the memory 310.

[0126] The embodiments of the present disclosure further provide a head-mounted display system, which can include a head-mounted display device 410 and an interaction device 420, the head-mounted display device 410 being communicatively connected with the interaction device 420. The interaction device 420 can include the action recognition apparatus 200 as shown in Figure 4 Figure 2 The interaction device 420 can include the action recognition apparatus 200 as shown in Figure 3

[0127] The embodiments of the present disclosure further provide a computer readable storage medium, which has stored thereon computer instructions, and the computer instructions are run by a processor to perform the action recognition method provided by the embodiments of the present disclosure.

[0128] The embodiments of the present disclosure can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions embodied therewith, and the computer readable program instructions are used to cause a processor to implement various aspects of the embodiments of the present disclosure.

[0129] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch cards or punched tape, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0130] ​​Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0131] Computer readable program instructions for carrying out operations of embodiments of the present disclosure can be assembly-level instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object-oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, for example, through the Internet using an Internet Service Provider. In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of embodiments of the present disclosure.

[0132] Various aspects of embodiments of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present disclosure. It will be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer readable program instructions.

[0133] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data, programs, program modules, e.g., instructions for operation, or digital content stored thereon or therein for a short time or not at all. The computer readable storage medium can also have instructions stored thereon or therein which may

[0134] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0135] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0136] Having described above several embodiments of the disclosure, any modifications and variations that fall within the scope of the described embodiments are also intended to be within the scope of the disclosure. As such, the above description is not exhaustive of all possible embodiments and, in light thereof, many modifications and variations of the described embodiments will be apparent to those skilled in the art. The selection of specific terms to be used in the description is not intended to limit the scope of the embodiments described herein or the scope of the claims that follow. Any and all modifications and variations are intended to be included within the scope of the present disclosure.

Claims

1. An action recognition method applied to an interactive device, characterized in that, The interactive device includes a pressure sensor array consisting of multiple pressure sensors, and the method includes: Acquire multiple pressure signals output by the multiple pressure sensors; The first matrix is ​​determined based on the pressure value of each of the plurality of pressure signals; If there is no reference matrix in the reference matrix set that is the same as the first matrix, a target reference matrix that matches the first matrix is ​​determined from the reference matrix set according to a preset fuzzy matching condition. The reference matrix set includes multiple reference matrices, and each reference matrix corresponds to a set action. Based on the relationship between the matrix and the set action, determine the target action corresponding to the target reference matrix, and execute the operation instruction corresponding to the target action; The step of determining the target reference matrix matching the first matrix from the reference matrix set according to preset fuzzy matching conditions includes: Obtain the degree of difference between each reference matrix in the reference matrix set and the first matrix; The reference matrices in the set of reference matrices whose degree of difference meets the fuzzy matching conditions are determined as the target reference matrices that match the first matrix.

2. The method according to claim 1, characterized in that, The degree of difference includes at least one of the number of difference points and the offset; The difference point is the detection location where the pressure value differs between the reference matrix and the first matrix.

3. The method according to claim 1, characterized in that, The degree of difference refers to the number of difference points; determining the reference matrix in the reference matrix set whose degree of difference meets the fuzzy matching condition as the target reference matrix matching the first matrix includes: Reference matrices in the reference matrix set whose number of differences is less than a predetermined number are identified as target reference matrices that match the first matrix. The difference point is the detection location where the pressure value differs between the reference matrix and the first matrix.

4. The method according to claim 1, characterized in that, The degree of difference is an offset; determining the reference matrix in the reference matrix set whose degree of difference meets the fuzzy matching condition as the target reference matrix matching the first matrix includes: Reference matrices in the reference matrix set whose offsets are less than a predetermined offset are identified as target reference matrices that match the first matrix.

5. The method according to claim 4, characterized in that, The predetermined offset can be a row or a column; Wherein, when the predetermined offset is one row, the pressure value of the i-th row in the first matrix corresponds one-to-one with the pressure value of the (i+1)-th or (i-1)-th row in the target reference matrix; When the predetermined offset is a column, the pressure value in the i-th column of the first matrix corresponds one-to-one with the pressure value in the (i+1)-th or (i-1)-th column of the target reference matrix.

6. The method according to claim 1, characterized in that, Before determining the target action corresponding to the target reference matrix based on the association between the matrix and the set action, the method further includes: If there exists a reference matrix in the set of reference matrices that is identical to the first matrix, the reference matrix that is identical to the first matrix shall be determined as the target reference matrix.

7. The method according to claim 1, characterized in that, The steps for obtaining each reference matrix in the set of reference matrices include: Issue a prompt message corresponding to the set action; In response to a user's pre-defined action in response to the prompt information, multiple reference pressure signals output by multiple sensors are acquired; Based on the plurality of reference pressure signals, a reference matrix corresponding to the set action is determined.

8. An interactive device, characterized in that, The interactive device includes a pressure sensor array consisting of multiple pressure sensors, and further includes: Memory is used to store executable computer instructions; A processor, configured to execute the action recognition method according to any one of claims 1-7, under the control of the executable computer instructions; The plurality of pressure sensors are all connected to the processor to output the collected pressure signals to the processor.

9. A head-mounted display system, characterized in that, include: Head-mounted display devices; The interactive device as described in claim 8 is communicatively connected to the head-mounted display device.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that can be read and executed by a computer. When the computer instructions are executed by a processor, they perform the action recognition method according to any one of claims 1-7.

Images