A gesture interaction method, system, device and medium

Abstract

The application discloses a gesture interaction method, system, device and medium, image collection is performed on a user to obtain a motion image of the user; skeleton point information of the user is generated according to the motion image, and a coordinate system is established according to the skeleton point information; a plurality of interaction regions are established according to the coordinate system; interaction gesture information is obtained according to the skeleton point information; in the coordinate system, whether the interaction gesture is located in the interaction region is judged according to the interaction gesture information, and a judgment result is output. The application has high interaction efficiency, can better meet high-frequency operation, and can realize instant interaction in the interaction process, reduce the dwell time of the interaction gesture, effectively improve the interaction accuracy, and realize no significant waiting in the interaction process. Meanwhile, the application has low interaction method cost, can be effectively popularized and used, is stable, and has good universality.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of interaction, in particular to a gesture interaction method, system, device and medium. BACKGROUND

[0002] Gesture recognition technology is a key technology of a new generation of natural human-computer interaction. Compared with traditional mouse, keyboard and other contact type operation methods, gestures have the advantages of natural intuition, easy understanding, simple operation, good experience and so on, and are more in line with human daily communication habits. Gesture recognition has become a research hotspot of human-computer interaction scheme. Gesture, as a natural and convenient language, is very suitable for human-computer interaction in terms of emotion and practicality. The significance of gesture recognition technology research lies in applying this natural and intuitive communication method to the interface technology of human-computer interaction, making the human-computer interface closer to human use habits, and thus making human-computer interaction more natural and convenient.

[0003] The existing interaction method needs to fix the action and stay for several seconds after the user makes an action or gesture, and then the action can be recognized. However, this interaction cannot be used in high-frequency interaction scenarios. The existing interaction method also uses a 3D camera to accurately determine the user's posture and project the human body contour onto the screen, thereby allowing the user to operate elements at any position on the screen. Although this interaction method has high flexibility, it has poor stability and complex interaction method. SUMMARY

[0004] The purpose of the present application is to ensure instant interaction, support high-frequency operation during interaction, and improve the recognition accuracy of interaction, use efficiency and flexibility. The present application provides a gesture interaction method, system, device and medium.

[0005] To achieve the above-mentioned purpose of the application, the present application provides a gesture interaction method, comprising:

[0006] Image acquisition is performed on the user to obtain an action image of the user;

[0007] Skeletal point information of the user is generated according to the action image, and a coordinate system is established according to the skeletal point information;

[0008] A plurality of interaction regions are established according to the coordinate system;

[0009] Interaction gesture information is obtained according to the skeletal point information;

[0010] In the coordinate system, it is judged whether the interaction gesture is located in the interaction region according to the interaction gesture information, and a judgment result is output.

[0011] In use, the application establishes a coordinate system by acquiring bone point information of a user, and establishes a plurality of interaction regions in the coordinate system, each of which corresponds to a preset control instruction. When interaction gesture information is acquired according to the bone point information, the interaction gesture information includes an interaction gesture, after the interaction gesture is recognized according to the bone point information, it is judged whether the interaction gesture is located in an interaction region, if the interaction gesture is located in an interaction region, the control instruction corresponding to the interaction region is output. The interaction gesture of the application is preset, which is a specific action, when the user makes the action and appears in a certain interaction region, the control instruction corresponding to the interaction region is output. In the application, the interaction gesture is preferably a hand-clapping action with two hands close to each other. For a normal person, a single hand is a default state, so based on the operation of a single hand, a specific anti-misjudgment logic needs to be added. The inventor believes that hand-clapping is a very good instant interaction action. From the user side, hand-clapping is not a default gesture, anyone can do it, and it is very easy to do. It can be considered that the hand-clapping action can well express the operation intention of the user and will not be confused with other natural actions. From the technical side, hand-clapping is an action that is easy to determine, we only need to determine whether the positions of the left and right hand bone points are close together. In this way, in use, not only the interaction accuracy is higher, but also there is no significant waiting time when recognizing, not only can support high-frequency operation, but also the interaction recognition accuracy is high, effectively improving the use efficiency and flexibility.

[0012] In the application, the coordinate system is established according to the bone point information, specifically including:

[0013] The bone point information of the user is generated according to the action image, and the bone point information includes a head-top bone point and a pelvic bone point;

[0014] The straight-line distance between the head-top bone point and the pelvic bone point is acquired, and the unit length of the coordinate system is acquired according to the straight-line distance;

[0015] The straight line between the head-top bone point and the pelvic bone point is taken as the longitudinal coordinate, the pelvic bone point is taken as the origin, and the right-angle coordinate system is constructed with the acquired unit length.

[0016] The distance between the top-of-the-head bone point and the pelvic bone point is selected as the key reference point, firstly because for users of different sizes, the distance between the top-of-the-head bone point and the pelvic bone point is generally stable in length and is suitable for determining a unit length, and for the unit length of the present application, the length of each unit length 1 is determined after the coordinate system is established, and the unit length is related to the subsequent determination of the distance between the two hands, the determination of whether it is in the interactive region, and the size of the interactive region. In the present application, preferably, the unit length is one-sixth of the straight-line distance between the top-of-the-head bone point and the pelvic bone point, and based on this unit length, it can be applied to almost all people. After the coordinate system is determined, the interactive gesture information is obtained according to the bone point information, specifically including:

[0017] Obtaining bone point information, the bone point information including left hand bone point and right hand bone point;

[0018] Obtaining the distance between the left hand bone point and the right hand bone point;

[0019] Comparing the distance between the left hand bone point and the right hand bone point with a threshold value T, if the distance between the left hand bone point and the right hand bone point is less than or equal to the threshold value T, then obtaining the interactive gesture information, the interactive gesture information including the middle point coordinate information between the left hand bone point and the right hand bone point.

[0020] For the present application, the threshold value T is a preset length, which is used to determine whether the two hands are close. In actual use, when the user makes a hand-clapping action, the two hands may not necessarily be completely close together, so the present application determines whether the two hands are close by using the threshold value T, which has higher recognition accuracy. When the distance between the left hand bone point and the right hand bone point is less than or equal to the threshold value T, it is proved that the two hands are close, and a hand-clapping action is made, that is, an interactive gesture is made. At this time, the middle point coordinate information between the left hand bone point and the right hand bone point is obtained, and the interactive gesture is determined to be located in a certain interactive region by using the middle point coordinate information between the left hand bone point and the right hand bone point. When determining, each interactive region is set in the coordinate system as needed. When the middle point coordinate information is located in a certain interactive region in the coordinate system, the result is output, and the output result is the control instruction corresponding to the interactive region.

[0021] In the present application, the interactive gesture can also be other preset actions, for example, the left hand skeleton point and the right foot sole skeleton point contact each other, that is, the left hand and the right foot contact each other, for the interactive gesture, whether the left hand and the right foot contact each other, the judgment method is the same as the judgment method of the above-mentioned hand-clapping interactive gesture, that is, by comparing the distance between the left hand skeleton point and the right foot sole skeleton point with the threshold value T, if the distance between the left hand skeleton point and the right foot sole skeleton point is less than or equal to the threshold value T, the interactive gesture in the interactive gesture information is obtained, through the interactive gesture, that is, the middle point of the straight line connecting line between the left hand skeleton point and the right foot sole skeleton point is taken as the coordinate information of the interactive gesture, and whether the interactive gesture is located in the interactive region is judged through the coordinate information.

[0022] In the present application, the interactive gesture can also be that the right hand and the left foot contact each other, can also be that the user jumps up the left foot and the right foot contact each other, can also be that the left hand and the right thigh contact each other, etc.

[0023] In the present application, the interactive gesture includes at least two skeleton points, the selection of the two skeleton points generally will not be easily located in the same interactive region, and even if the two skeleton points are located in an interactive region during the interaction, when judging whether it is an interactive gesture, it is further necessary to judge whether the two skeleton points contact each other, for the present application, whether the two skeleton points contact each other is judged by the threshold value T. When the interactive gesture is set and selected, the two skeleton points of the interactive gesture generally will not occur the interactive gesture or will not be in the interactive region at the same time when the user does not interact, so that the misjudgment can be further avoided.

[0024] In the present application, the position and area of each interactive region in the coordinate system can be set according to actual needs, can be set in four quadrants in the coordinate system, or can be in an area overlapping several quadrants, which can be set according to needs, in the present application, the specific position and area of the interactive region in the coordinate system are not limited, when the specific position and area are set, the coordinate point can be set.

[0025] On this basis, in order to further avoid misjudgment, or when the user's interaction gesture appears in a certain interaction area, if the user's interaction gesture appears in the adjacent interaction area of the interaction area in the process of continuous movement, another control instruction of the interaction area is triggered, so that misjudgment is prone to occur in the process of continuous movement, and the experience and fluency of the user are reduced, therefore, the interaction method of the application, when in use, if the interaction gesture is located in an interaction area, the interaction area is in an activated state, and the rest of the interaction areas are in an inactivated state; in the coordinate system, the area of the interaction area in the activated state is increased, and the area of the interaction area in the inactivated state is reduced. When the interaction gesture is located in an interaction area, the control instruction corresponding to the interaction area is output, at this time, the area of the interaction area in the coordinate system is increased, and the areas of the other interaction areas are reduced, so that misjudgment does not occur when the interaction gesture is located in an interaction area in the inactivated state in the process of continuous movement, for example, when the interaction gesture is a handclap, and there are four interaction areas, the four interaction areas are located in the four quadrants of the coordinate system, when the user makes an interaction gesture in one of the interaction areas, when the hand is released, the adjacent interaction area of the area is prone to be triggered, so that misjudgment is prone to occur, therefore, when an interaction area is output, the area of the interaction area is increased, and the areas of the other interaction areas are reduced, so that the problem can be better avoided.

[0026] Corresponding to the method in the application, the application also provides a gesture interaction system, comprising:

[0027] An acquisition module is configured to acquire a user image to obtain a motion image of a user.

[0028] A first identification module is configured to acquire skeleton point information of the user according to the motion image.

[0029] An interaction module is configured to establish a coordinate system according to the skeleton point information, and establish a plurality of interaction areas in the coordinate system.

[0030] A second identification module is configured to acquire interaction gesture information according to the skeleton point information.

[0031] A judgment module is configured to judge whether the interaction gesture is located in the interaction area according to the interaction gesture information, and output a judgment result.

[0032] An execution module is configured to execute a control instruction corresponding to the interaction area according to the judgment result.

[0033] The system further comprises an adjustment module, which is configured to adjust the area of the interaction area according to the judgment result.

[0034] Corresponding to the method in the present application, the present application also provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned gesture interaction method when executing the computer program.

[0035] Corresponding to the method in the present application, the present application also provides a storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the above-mentioned gesture interaction method.

[0036] The one or more technical solutions provided by the present application have at least the following technical effects or advantages:

[0037] The interaction efficiency of the present application is high, which can better meet the high-frequency operation, and the present application can realize instant interaction in the process of interaction, reduce the dwell time of the interaction gesture, effectively improve the accuracy of the interaction, and realize the interaction process without significant waiting. At the same time, the interaction method of the present application has low cost and can be effectively popularized and used, which is not only stable but also has good universality. BRIEF DESCRIPTION OF DRAWINGS

[0038] The accompanying drawings, which are included to provide a further understanding of the embodiments of the present application and constitute a part of the application, illustrate embodiments of the present application and do not limit the present application. In the drawings:

[0039] Figure 1 It is a flowchart of a gesture interaction method;

[0040] Figure 2 It is a composition diagram of a gesture interaction system;

[0041] Figure 3 It is a schematic diagram of three interaction regions in the coordinate system in embodiment 1;

[0042] Figure 4 It is a schematic diagram of five interaction regions in the coordinate system in embodiment 2. DETAILED DESCRIPTION

[0043] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application will be further described in detail below in combination with the drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0044] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, however, the present application can also be implemented in other ways different from the scope described herein, therefore, the protection scope of the present application is not limited by the specific embodiments disclosed below.

[0045] Those skilled in the art should understand that in the disclosure of the present application, the orientation or positional relationship indicated by the terms "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like are based on the orientation or positional relationship shown in the drawings, which is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, therefore the above terms cannot be understood as a limitation of the present application.

[0046] It can be understood that the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of one element can be one, and in another embodiment, the number of the element can be multiple, and the term "one" cannot be understood as a limitation on the number.

[0047] Embodiment one

[0048] Please refer to Figure 1 , Figure 1 A flowchart of a gesture interaction method is provided, and the present application provides a gesture interaction method, which comprises:

[0049] Image acquisition is performed on the user to obtain the action image of the user;

[0050] Skeletal point information of the user is generated according to the action image, wherein the skeletal point information comprises a head-top skeletal point and a pelvic skeletal point;

[0051] A straight-line distance between the head-top skeletal point and the pelvic skeletal point is obtained, and a unit length of a coordinate system is obtained according to the straight-line distance;

[0052] A straight-line between the head-top skeletal point and the pelvic skeletal point is taken as a longitudinal coordinate, the pelvic skeletal point is taken as an origin, and a rectangular coordinate system is constructed with the obtained unit length.

[0053] A plurality of interaction regions are established according to the coordinate system;

[0054] Skeletal point information is obtained, which comprises a left-hand skeletal point and a right-hand skeletal point;

[0055] A distance between the left-hand skeletal point and the right-hand skeletal point is obtained;

[0056] The distance between the left-hand skeletal point and the right-hand skeletal point is compared with a threshold value T, if the distance between the left-hand skeletal point and the right-hand skeletal point is less than or equal to the threshold value T, then interaction gesture information is obtained, which comprises the coordinate information of the middle point between the left-hand skeletal point and the right-hand skeletal point;

[0057] In the coordinate system, it is judged according to the interaction posture information whether the interaction posture is located in the interaction region, whether the intermediate point coordinate information is located in the interaction region, and a judgment result is output. If the intermediate point coordinate information is located in the interaction region, a control instruction corresponding to the interaction region is output.

[0058] A gesture interaction method according to an embodiment of the present application will be described below with reference to specific examples:

[0059] Step 1: image acquisition of a user to obtain a motion image of the user;

[0060] Step 2: generating skeleton point information of the user according to the motion image, and establishing a coordinate system according to the skeleton point information;

[0061] Step 2.1: the skeleton point information includes a head top skeleton point, a pelvic skeleton point, a left hand skeleton point, and a right hand skeleton point;

[0062] Step 2.2: obtaining a straight line distance between the head top skeleton point and the pelvic skeleton point, and obtaining a unit length of the coordinate system according to the straight line distance;

[0063] Step 2.21: obtaining a straight line distance between the head top skeleton point and the pelvic skeleton point, and taking one sixth of the straight line distance as the unit length of the coordinate system, i.e., the size of the unit length 1 of the coordinate system is one sixth of the straight line distance;

[0064] Step 2.3: taking the straight line between the head top skeleton point and the pelvic skeleton point as a longitudinal coordinate, taking the pelvic skeleton point as an origin, and constructing a rectangular coordinate system with the obtained unit length;

[0065] Step 3: establishing a plurality of interaction regions according to the coordinate system;

[0066] Figure 3 In this embodiment, there are three interaction regions, which are a first interaction region, a second interaction region, and a third interaction region, as shown in Figure 3 , the first interaction region is an A region in Figure 3 , the second interaction region is a B region in Figure 3 , and the third interaction region is a C region in , wherein the first interaction region is centered on the head top skeleton point, has a length of 12 unit lengths and a height of 10 unit lengths in a vertical direction of 5 unit lengths and a horizontal direction of 6 unit lengths, respectively. The first interaction region has an upper edge of 5 unit lengths upward, a lower edge of 5 unit lengths downward, a left edge of 6 unit lengths to the left, and a right edge of 6 unit lengths to the right, all with the head top skeleton point as the center. The first interaction region is preset with a control instruction of determination, i.e., when the interaction posture is located in the first interaction region, a control instruction corresponding to the first interaction region is output, i.e., a control instruction of determination is output.

[0067]

[0067] The control instruction of the second interaction area is left, and the control instruction of the third interaction area is right. The second interaction area is a region with a length of 13 unit lengths and a height of 12 unit lengths, which is formed by taking the pelvis skeleton point as a starting point, taking 2 unit lengths vertically upward as an upper edge, taking 10 unit lengths vertically downward as a lower edge, taking the head top skeleton point as a starting point, taking 3 unit lengths horizontally to the left as a right edge, and taking 16 unit lengths horizontally to the left as a left edge. The third interaction area is a region with a length of 13 unit lengths and a height of 12 unit lengths, which is formed by taking the pelvis skeleton point as a starting point, taking 2 unit lengths vertically upward as an upper edge, taking 10 unit lengths vertically downward as a lower edge, taking the head top skeleton point as a starting point, taking 3 unit lengths horizontally to the right as a left edge, and taking 16 unit lengths horizontally to the right as a right edge.

[0068] Step 4: obtaining interaction gesture information according to the skeleton point information;

[0069] Step 4.1: obtaining a distance between the left hand skeleton point and the right hand skeleton point;

[0070] Step 4.2: comparing the distance between the left hand skeleton point and the right hand skeleton point with a threshold T, in this embodiment, the threshold T is 15 cm, if the distance between the left hand skeleton point and the right hand skeleton point is less than or equal to the threshold T, it is judged that the user makes an interaction gesture, in this embodiment, the interaction gesture is a handslap, that is, the user makes a handslap action, if the distance between the left hand skeleton point and the right hand skeleton point is greater than the threshold T, the user does not make a handslap action;

[0071] Step 4.3: obtaining the coordinate information of the middle point between the left hand skeleton point and the right hand skeleton point when the user makes the interaction gesture;

[0072] Step 5: judging whether the interaction gesture is located in the interaction area according to the interaction gesture information in the coordinate system;

[0073] Step 5.1: judging whether the coordinates of the middle point coordinate information are located in the interaction area;

[0074] Step 5.11: if the coordinates of the middle point coordinate information are located in one of the interaction areas, outputting the control instruction corresponding to the interaction area;

[0075] Step 5.12: outputting the control instruction corresponding to the interaction area, at the same time, the interaction area is in an activated state, and the rest of the interaction areas are in an unactivated state; the area of the interaction area in the activated state is increased, and the area of the interaction area in the unactivated state is decreased.

[0076] In this embodiment, the length of the interaction area in the activated state is increased by 1.5 unit lengths, and the height is increased by 0.5 unit lengths; the length of the interaction area in the unactivated state is decreased by 2 unit lengths, and the height is decreased by 1 unit length.

[0077] When the control instruction corresponding to the interaction area is not output, the area of the interaction area does not change, after the control instruction is output, the area of the interaction area is adjusted, and after the control instruction is executed, the area of the interaction area returns to the original state.

[0078] Embodiment Two

[0079] In this embodiment, the fitness device is a smart fitness mirror with a display screen. When in use, the user stands in the collection area of the smart fitness mirror, and the display screen plays a fitness video. The device includes the following steps in the interaction process:

[0080] Step 1: The smart fitness mirror collects the image of the user in the collection area to obtain the motion image of the user.

[0081] Step 2: Generate the skeletal point information of the user according to the motion image, and establish a coordinate system according to the skeletal point information.

[0082] Step 2.1: The skeletal point information includes a head skeletal point, a pelvic skeletal point, a left hand skeletal point, and a right hand skeletal point.

[0083] Step 2.2: Obtain the straight-line distance between the head skeletal point and the pelvic skeletal point, and obtain the unit length of the coordinate system according to the straight-line distance.

[0084] Step 2.21: Obtain the straight-line distance between the head skeletal point and the pelvic skeletal point, and take one-sixth of the straight-line distance as the unit length of the coordinate system, i.e., the size of the unit length 1 of the coordinate system is one-sixth of the straight-line distance.

[0085] Step 2.3: Take the straight line between the head skeletal point and the pelvic skeletal point as the longitudinal coordinate, take the pelvic skeletal point as the origin, and construct a rectangular coordinate system with the obtained unit length.

[0086] Step 3: Establish a plurality of interaction areas according to the coordinate system.

[0087] In this embodiment, there are five interaction areas, which are a first interaction area, a second interaction area, a third interaction area, a fourth interaction area, and a fifth interaction area, as shown in Figure 4 The first interaction area is the A area in Figure 4 , the second interaction area is the B area in Figure 4 , the third interaction area is the C area in Figure 4 , the fourth interaction area is the D area in Figure 4 , and the fifth interaction area is the E area in Figure 4 .

[0088] The first interaction area is 5 unit lengths in the longitudinal direction and 6 unit lengths in the transverse direction as the first interaction area, that is, the length of the first interaction area is 12 unit lengths and the height is 10 unit lengths. The first interaction area is 5 unit lengths upward in the longitudinal direction as the upper edge and 5 unit lengths downward in the longitudinal direction as the lower edge; 6 unit lengths to the left in the transverse direction as the left edge and 6 unit lengths to the right in the transverse direction as the right edge with the head top skeleton point as the center. The preset control instruction of the first interaction area is determination A, that is, when the interaction posture is located in the first interaction area, the control instruction corresponding to the first interaction area is output, that is, the control instruction of determination A is output.

[0089] The control instruction of the second interaction area is left, and the control instruction of the third interaction area is right. The second interaction area is a region with a length of 13 unit lengths and a height of 12 unit lengths, which is composed of the pelvis skeleton point as the starting point, 2 unit lengths upward in the longitudinal direction as the upper edge, 10 unit lengths downward in the longitudinal direction as the lower edge, the head top skeleton point as the starting point, 3 unit lengths to the left in the transverse direction as the right edge, and 16 unit lengths to the left in the transverse direction as the left edge. The third interaction area is a region with a length of 13 unit lengths and a height of 12 unit lengths, which is composed of the pelvis skeleton point as the starting point, 2 unit lengths upward in the longitudinal direction as the upper edge, 10 unit lengths downward in the longitudinal direction as the lower edge, the head top skeleton point as the starting point, 3 unit lengths to the right in the transverse direction as the left edge, and 16 unit lengths to the right in the transverse direction as the right edge.

[0090] The control instruction of the fourth interaction area is determination D, and the control instruction of the fifth interaction area is determination E. The fourth interaction area is a region with a length of 11 unit lengths and a height of 9 unit lengths, which is composed of the head top skeleton point as the starting point, 4 unit lengths upward in the longitudinal direction as the upper edge, 5 unit lengths downward in the longitudinal direction as the lower edge, the head top skeleton point as the starting point, 7 unit lengths to the left in the transverse direction as the right edge, and 16 unit lengths to the left in the transverse direction as the left edge.

[0091] The fifth interaction area is a region with a length of 11 unit lengths and a height of 9 unit lengths, which is composed of the head top skeleton point as the starting point, 4 unit lengths upward in the longitudinal direction as the upper edge, 5 unit lengths downward in the longitudinal direction as the lower edge, the head top skeleton point as the starting point, 7 unit lengths to the right in the transverse direction as the left edge, and 16 unit lengths to the right in the transverse direction as the right edge.

[0092] Step 4: identifying the interaction posture according to the skeleton point information;

[0093] Step 4.1: obtaining the distance between the left hand skeleton point and the right hand skeleton point;

[0094] Step 4.2, the distance between the left hand skeletal point and the right hand skeletal point is compared with a threshold value T, if the distance between the left hand skeletal point and the right hand skeletal point is less than or equal to the threshold value T, the interactive gesture is recognized, in the embodiment, the threshold value T is 15 cm, if the distance between the left hand skeletal point and the right hand skeletal point is less than or equal to the threshold value T, it is judged that the user makes the interactive gesture, the interactive gesture is recognized, in the embodiment, the interactive gesture is clapping, that is, the user makes the clapping action; if the distance between the left hand skeletal point and the right hand skeletal point is greater than the threshold value T, the user does not make the clapping action;

[0095] Step 4.3, if the user makes the interactive gesture, the coordinates of the intermediate point between the left hand skeletal point and the right hand skeletal point are obtained;

[0096] Step 5, it is judged whether the coordinates of the intermediate point are located in the interactive region or not;

[0097] Step 5.1, it is judged whether the coordinates of the intermediate point are located in the interactive region or not;

[0098] Step 5.11, if the coordinates of the intermediate point are located in one of the interactive regions, the interactive region is in the active state, and the control instruction corresponding to the interactive region is output; the other interactive regions are in the inactive state;

[0099] Step 5.12, while the control instruction corresponding to the interactive region is output, the interactive region is in the active state, and the other interactive regions are in the inactive state; the area of the interactive region in the active state is increased, and the area of the interactive region in the inactive state is decreased.

[0100] In the embodiment, the length of the interactive region in the active state is increased by 1.5 unit lengths, and the height is increased by 0.5 unit lengths, specifically, in the embodiment, the length of the interactive region in the active state is sequentially increased to the left and right sides, that is, it is expanded by 0.75 unit lengths, and the height is sequentially increased to the upper and lower ends, that is, it is expanded by 0.25 unit lengths. The length of the interactive region in the inactive state is reduced by 2 unit lengths, and the height is reduced by 1 unit length. Specifically, the length of the interactive region in the inactive state is sequentially reduced to the left and right sides, that is, it is reduced by 1 unit length, and the height is reduced to the upper and lower ends, that is, it is reduced by 0.5 unit length.

[0101] The area of the interactive region does not change when the control instruction corresponding to the interactive region is not output, the area of the interactive region is adjusted after the control instruction is output, and the area of the interactive region returns to the original state after the control instruction is executed.

[0102] When the user makes an interaction gesture in the first interaction area, the fitness mirror recognizes the interaction gesture, outputs the control instruction corresponding to the first interaction area, and the area of the first interaction area is expanded, that is, the upper edge is increased by 0.25 unit lengths vertically upward, the lower edge is increased by 0.25 unit lengths vertically downward, the left edge is increased by 0.75 unit lengths horizontally to the left, and the right edge is increased by 0.75 unit lengths horizontally to the right.

[0103] Embodiment Three

[0104] In this embodiment, on the basis of Embodiment 2, the skeleton point information is acquired, and the interaction gesture is acquired according to the skeleton point information.

[0105] It is judged whether the interaction gesture is located in the interaction area.

[0106] If the interaction gesture is located in the interaction area, the control instruction corresponding to the interaction area is output.

[0107] The interaction gesture is that the left hand skeleton point and the right foot palm skeleton point contact each other, that is, the left hand and the right foot contact each other. For this interaction gesture, whether the left hand and the right foot contact each other is judged in the same way as the judgment method of the interaction gesture of the handslapping, that is, by comparing the distance between the left hand skeleton point and the right foot palm skeleton point with the threshold value T. If the distance between the left hand skeleton point and the right foot palm skeleton point is less than or equal to the threshold value T, the interaction gesture in the interaction gesture information is acquired. The middle point of the straight line connecting the left hand skeleton point and the right foot palm skeleton point is taken as the coordinate information of the interaction gesture, and whether the interaction gesture is located in the interaction area is judged by the coordinate information.

[0108] Embodiment Four

[0109] In this embodiment, on the basis of Embodiment 2, the skeleton point information is acquired, and the interaction gesture is acquired according to the skeleton point information.

[0110] It is judged whether the interaction gesture is located in the interaction area.

[0111] If the interaction gesture is located in the interaction area, the control instruction corresponding to the interaction area is output.

[0112] The interaction gesture is that the right hand skeleton point and the left foot palm skeleton point contact each other, that is, the right hand and the left foot contact each other. For this interaction gesture, whether the right hand and the left foot contact each other is judged in the same way as the judgment method of the interaction gesture of the handslapping, that is, by comparing the distance between the right hand skeleton point and the left foot palm skeleton point with the threshold value T. If the distance between the right hand skeleton point and the left foot palm skeleton point is less than or equal to the threshold value T, the interaction gesture in the interaction gesture information is acquired. The middle point of the straight line connecting the right hand skeleton point and the left foot palm skeleton point is taken as the coordinate information of the interaction gesture, and whether the interaction gesture is located in the interaction area is judged by the coordinate information.

[0113] Embodiment five

[0114] In this embodiment, the skeleton point information is acquired, and the interactive gesture is acquired according to the skeleton point information on the basis of Embodiment 2.

[0115] It is judged whether the interactive gesture is located in the interactive region.

[0116] If the interactive gesture is located in the interactive region, the control instruction corresponding to the interactive region is output.

[0117] The interactive gesture is that the right hand skeleton point and the left knee skeleton point contact each other, that is, the right hand and the left knee contact each other. For the interactive gesture, whether the right hand and the left knee contact each other is judged by the same method as that of the above-mentioned interactive gesture of the handslapping, that is, by comparing the distance between the right hand skeleton point and the left knee skeleton point with the threshold value T. If the distance between the right hand skeleton point and the left knee skeleton point is less than or equal to the threshold value T, the interactive gesture in the interactive gesture information is acquired. The coordinate information of the interactive gesture is obtained by the middle point of the straight line connecting the right hand skeleton point and the left knee skeleton point, and it is judged whether the interactive gesture is located in the interactive region by the coordinate information.

[0118] Embodiment six

[0119] Please refer to Figure 2 , Figure 2 A gesture interactive system is provided in Embodiment two of the present application, and a composition schematic diagram of the gesture interactive system is shown in Fig. 2. The system comprises:

[0120] An acquisition module is configured to acquire a user image to obtain a motion image of a user.

[0121] A first recognition module is configured to acquire skeleton point information of the user according to the motion image.

[0122] An interactive module is configured to establish a coordinate system according to the skeleton point information, and establish a plurality of interactive regions in the coordinate system.

[0123] A second recognition module is configured to acquire interactive gesture information according to the skeleton point information.

[0124] A judgment module is configured to judge whether an interactive gesture is located in an interactive region according to the interactive gesture information, and output a judgment result.

[0125] An execution module is configured to execute a control instruction corresponding to the interactive region according to the judgment result.

[0126] Embodiment seven

[0127] The embodiment seven of the present application provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the gesture interaction method when executing the computer program.

[0128] The processor can be a central processing unit, and can also be other general-purpose processors, digital signal processors, application-specific integrated circuits, ready programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any other conventional processor.

[0129] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the gesture interaction device by running or executing the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required for a function (such as a sound playing function, an image playing function, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, an intelligent memory card, a secure digital card, a flash memory card, at least one disk storage device, a flash memory device, or other volatile solid-state memory devices.

[0130] Embodiment eight

[0131] The embodiment eight of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the gesture interaction method.

[0132] The computer storage medium of the embodiments of the present application can adopt any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device.

[0133] Although preferred embodiments of the application have been described, those skilled in the art will be able to make additional modifications and variations to the described embodiments without departing from the spirit and scope of the application. Accordingly, the appended claims are intended to encompass within their scope all such modifications and variations as fall within the scope of the present application.

[0134] Obviously, various modifications and changes can be made to the present application by those skilled in the art without departing from the spirit and scope of the present application. Accordingly, the present application intends to include all such modifications and changes as fall within the scope of the present application claims and their equivalents.

Claims

1. A gesture interaction method, characterized by, The method comprises the following steps: image acquisition of a user to obtain action images of the user; skeleton point information of the user is generated according to the action images, and a coordinate system is established according to the skeleton point information; a plurality of interaction regions are established according to the coordinate system; interaction gesture information is obtained according to the skeleton point information, and a distance between a left hand skeleton point and a right hand skeleton point is obtained; if the distance between the left hand skeleton point and the right hand skeleton point is less than or equal to a threshold T, it is determined that the user makes an interaction gesture; if the distance between the left hand skeleton point and the right hand skeleton point is greater than the threshold T, it is determined that the user does not make an interaction gesture; in the coordinate system, it is determined whether the interaction gesture is located in the interaction region according to the interaction gesture information, and a determination result is output; skeleton point information of the user is generated according to the action images, and a coordinate system is established according to the skeleton point information, specifically comprising: skeleton point information of the user is generated according to the action images, and the skeleton point information comprises a head top skeleton point and a pelvic bone skeleton point; a straight line distance between the head top skeleton point and the pelvic bone skeleton point is obtained, and a unit length of the coordinate system is obtained according to the straight line distance; a straight line between the head top skeleton point and the pelvic bone skeleton point is taken as a longitudinal coordinate, the pelvic bone skeleton point is taken as an origin, and a rectangular coordinate system is constructed with the obtained unit length; interaction gesture information is obtained according to the skeleton point information, specifically comprising: skeleton point information is obtained, and an interaction gesture is obtained according to the skeleton point information; it is determined whether the interaction gesture is located in the interaction region; if the interaction gesture is located in the interaction region, a control instruction corresponding to the interaction region is output; coordinate information of the interaction gesture in the coordinate system is obtained, and it is determined whether the interaction gesture is located in the interaction region through the coordinate information; the interaction gesture comprises at least two skeleton point information; if the interaction gesture is located in one interaction region, the interaction region is in an activated state, and the remaining interaction regions are in an inactivated state; in the coordinate system, the area of the interaction region in the activated state increases, and the area of the interaction region in the inactivated state decreases.

2. A gesture interaction system implementing a gesture interaction method as claimed in claim 1, characterized in that The method comprises the following steps: an acquisition module is configured to obtain action images of a user from a user image; a first recognition module is configured to obtain skeleton point information of the user according to the action images; an interaction module is configured to establish a coordinate system according to the skeleton point information, and establish a plurality of interaction regions in the coordinate system; a second recognition module is configured to obtain interaction gesture information according to the skeleton point information; a judgment module is configured to determine whether the interaction gesture is located in the interaction region according to the interaction gesture information, and output a determination result; an execution module is configured to execute a control instruction corresponding to the interaction region according to the determination result.

3. A posture interaction system according to claim 2, wherein The method further comprises an adjustment module configured to adjust the area of the interaction region according to the determination result. 4.An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, The processor executes the computer program to implement the steps of the gesture interaction method of claim 1.

5. A storage medium storing a computer program, characterized by The computer program is executed by the processor to implement the steps of the gesture interaction method of claim 1.

Images