Gesture recognition method, gesture control method and electronic equipment

Abstract

The invention provides a gesture recognition method, a gesture control method and electronic equipment, and the method comprises the steps: obtaining the face, hand and human body of a user in a target image comprising a plurality of users, determining the user corresponding to the face and human body based on the region where the face and human body are located and the corresponding relation between the region and the user, according to the human body corresponding to the human hand, the left-right position of the human hand relative to the human body, the human face and the user corresponding to the human body, determining the user corresponding to the human hand and the left-right position of the human hand relative to the user according to the human body corresponding to the human hand, the left-right position of the human hand relative to the human body, the human face and the user corresponding to the human body; and determining the gesture type. Therefore, the user to which the hand belongs and the left-right position of the hand relative to the user can be determined in a multi-user scene, namely whether the hand is the left hand or the right hand of the corresponding user can be determined, the accuracy of gesture recognition can be improved, and the accuracy of gesture control is improved.

Description

Technical Field

[0001] The present application relates to the field of gesture control technology, and in particular to a gesture recognition method, a gesture control method and an electronic device. Background Art

[0002] With the continuous improvement of the degree of digitalization of electronic devices, there are more and more scenarios in which human-computer interaction is achieved by interacting with electronic devices through gestures. In the prior art, user gestures are usually recognized based on the captured scene images, and human-computer interaction processing such as electronic device control is performed according to the gesture type of the user gesture. Based on this approach, on the one hand, in a multi-user scenario, there is a problem of not being able to accurately determine the user to whom the gesture belongs, and on the other hand, there is usually a problem of not being able to accurately identify whether the gesture is a left-hand gesture or a right-hand gesture. Therefore, the existing gesture recognition method has the problem of inaccurate gesture recognition, which affects the accuracy of human-computer interaction processing such as gesture control. Summary of the invention

[0003] The purpose of the present application is to solve the problem of inaccurate gesture recognition in the prior art gesture recognition method, which affects the accuracy of human-machine interaction processing such as gesture control.

[0004] Among them, based on the recognition of the human hand corresponding to the user and the left and right position of the human hand relative to the user, it is possible to more accurately determine the user to which the gesture belongs and whether the gesture is a left-hand gesture or a right-hand gesture, thereby obtaining a more accurate gesture type, improving the accuracy of gesture recognition, and thereby improving the accuracy of human-computer interaction processing such as gesture control.

[0005] To solve the above technical problems, in a first aspect, the implementation method of the present application discloses a gesture recognition method, which includes: acquiring a target image, the target image includes multiple users; identifying the user's face, hand, and human body other than the face and hand in the target image; determining the user corresponding to the face and human body based on the area where the face and human body are located, and the correspondence between the preset area and the user; obtaining the arm edge contour and the clothing edge contour according to the human body, and determining the human body corresponding to the hand and the left and right position of the hand relative to the human body according to the arm edge contour and the clothing edge contour, as well as the human hand; determining the user corresponding to the hand and the left and right position of the hand relative to the user according to the correspondence between the hand and the human body, the left and right position of the hand relative to the human body, and the correspondence between the face and the human body and the user; determining the gesture type according to the user corresponding to the hand and the left and right position of the hand relative to the user.

[0006] By adopting the above technical solution, in a multi-user scenario, the acquired target image includes multiple users. In order to accurately identify gestures, the faces, hands and bodies of the users in the target image are first identified, and then the users corresponding to the faces and bodies are determined according to the areas where the faces and bodies are located, and the correspondence between the preset areas and the users. The human body corresponding to the hands and the left and right positions of the hands relative to the human body are determined according to the arm edge contour and clothing edge contour obtained from the human body, as well as the human hand. Then, according to the correspondence between the hands and the human body, the left and right positions of the hands relative to the human body, and the correspondence between the faces and the human body and the users, the users corresponding to the hands and the left and right positions of the hands relative to the human body are determined, and the gesture type is determined based on the users corresponding to the hands and the left and right positions of the hands relative to the users. In this way, the users corresponding to the hands and the left and right positions of the hands relative to the users can be more accurately identified, and the users to whom the gesture belongs and whether the gesture is a left-hand gesture or a right-hand gesture can be more accurately determined, that is, whether the hands are left-hand gestures or right-hand gestures of a certain user can be more accurately determined, so that a more accurate gesture type can be obtained, the accuracy of gesture recognition can be improved, and the accuracy of human-computer interaction processing such as gesture control can be improved.

[0007] According to another specific implementation of the present application, the gesture recognition method disclosed in the implementation of the present application determines the human body corresponding to the human hand and the left and right position of the human hand relative to the human body based on the edge contour of the arm and the edge contour of the clothing, as well as the human hand, including: determining the inflection point of the arm edge contour and the inflection point of the clothing edge contour; obtaining multiple segment curves of the arm edge contour and multiple segment curves of the clothing edge contour based on the inflection point of the arm edge contour and the inflection point of the clothing edge contour; calculating the similarity and distance between each segment curve of the arm edge contour and each segment curve of the clothing edge contour to obtain the corresponding curve similarity and curve distance; determining the human body corresponding to the human hand and the left and right position of the human hand relative to the human body based on the curve similarity, curve distance, human hand, and the preset correspondence between the human hand edge contour and the arm edge contour, and the correspondence between the human body edge contour and the clothing edge contour.

[0008] By adopting the above technical solution, based on the curve similarity and curve distance corresponding to the arm edge contour and the clothing edge contour, as well as the human hand and the preset correspondence, the human body corresponding to the hand and the left and right positions of the hand relative to the human body can be determined. This can more accurately identify the user corresponding to the hand and the left and right positions of the hand relative to the user, and can more accurately determine the user to whom the gesture belongs and whether the gesture is a left-hand gesture or a right-hand gesture.

[0009] According to another specific implementation of the present application, the gesture recognition method disclosed in the implementation of the present application determines the inflection point of the arm edge contour and the inflection point of the clothing edge contour, including: determining discrete points of the arm edge contour and the discrete points of the clothing edge contour; performing a first curvature calculation on the discrete points of the arm edge contour to obtain a first curvature calculation value; based on the first curvature calculation value, obtaining the inflection point of the arm edge contour; and performing a second curvature calculation on the discrete points of the clothing edge contour to obtain a second curvature calculation value; based on the second curvature calculation value, obtaining the inflection point of the clothing edge contour.

[0010] By adopting the above technical scheme, the corresponding inflection point is determined based on the curvature calculation value obtained based on the discrete points corresponding to the arm edge contour and the clothing edge contour. Thus, the human body corresponding to the hand and the left and right position of the hand relative to the human body are determined based on the segment curve of the arm edge contour and the clothing edge contour determined by the inflection point. This can more accurately identify the user corresponding to the hand and the left and right position of the hand relative to the user.

[0011] According to another specific implementation of the present application, the gesture recognition method disclosed in the implementation of the present application determines discrete points of the edge contour of the arm and discrete points of the edge contour of the clothing, including: determining the edge points of the edge contour of the arm and the edge points of the edge contour of the clothing; discretizing the edge points of the edge contour of the arm and the edge points of the edge contour of the clothing to obtain discrete points of the edge contour of the arm and the discrete points of the edge contour of the clothing.

[0012] By adopting the above technical solution, the edge points corresponding to the edge contour of the arm and the edge contour of the clothing are discretized, which facilitates the subsequent calculation of the inflection point.

[0013] According to another specific implementation of the present application, the gesture recognition method disclosed in the implementation of the present application calculates the similarity and distance between each segment curve of the arm edge contour and each segment curve of the clothing edge contour to obtain the corresponding curve similarity and curve distance, including: calculating the similarity between each segment curve of the arm edge contour and each segment curve of the clothing edge contour based on the curve similarity algorithm to obtain the corresponding curve similarity; and calculating the distance between each segment curve of the arm edge contour and each segment curve of the clothing edge contour based on the curve distance algorithm to obtain the corresponding curve distance.

[0014] By adopting the above technical solution, based on the curve similarity and curve distance calculated by the algorithm, and based on the human hand and the preset correspondence, the human body corresponding to the hand and the left and right position of the hand relative to the human body are determined, and then the user corresponding to the hand and the left and right position of the hand relative to the user are determined. This can more accurately determine the user corresponding to the gesture and whether the gesture is a left-hand gesture or a right-hand gesture.

[0015] According to another specific implementation of the present application, in the gesture recognition method disclosed in the implementation of the present application, the curve similarity algorithm is a dynamic time warping algorithm, and the curve distance algorithm is a Hausdorff algorithm.

[0016] By adopting the above technical solution, the curve similarity and curve distance between each segment curve of the arm edge contour and each segment curve of the clothing edge contour are calculated through the dynamic time warping algorithm and the Hausdorff algorithm, so that the calculation of curve similarity and curve distance is more accurate.

[0017] According to another specific implementation of the present application, the gesture recognition method disclosed in the implementation of the present application determines the human body corresponding to the human hand, and the left and right position of the human hand relative to the human body according to curve similarity, curve distance, human hand, and the preset correspondence between the human hand edge contour and the arm edge contour, and the correspondence between the human body edge contour and the clothing edge contour, including: when the curve similarity is greater than the preset curve similarity threshold, and the curve distance is less than the preset curve distance threshold, obtaining the clothing edge contour corresponding to the arm edge contour, and the left and right position of the arm edge contour relative to the clothing edge contour; determining the clothing edge contour corresponding to the human hand edge contour, and the left and right position of the human hand edge contour relative to the clothing edge contour according to the correspondence between the arm edge contour and the clothing edge contour, the left and right position of the arm edge contour relative to the clothing edge contour, and the preset correspondence between the human hand edge contour and the arm edge contour; determining the human body corresponding to the human hand, and the left and right position of the human hand edge contour relative to the human body according to the correspondence between the human hand edge contour and the clothing edge contour, the left and right position of the human hand edge contour relative to the clothing edge contour, the human hand, and the preset correspondence between the human body edge contour and the clothing edge contour.

[0018] By adopting the above technical scheme, based on the correspondence between the edge contour of the arm and the edge contour of the clothing, the left and right position of the edge contour of the arm relative to the edge contour of the clothing, the human hand and the preset correspondence, the human body corresponding to the human hand and the left and right position of the human hand relative to the human body are determined, and then the user corresponding to the human hand and the left and right position of the human hand relative to the user are determined. The user to whom the gesture belongs and whether the gesture is a left-hand gesture or a right-hand gesture can be accurately determined, thereby obtaining a more accurate gesture type and improving the accuracy of gesture recognition.

[0019] According to another specific implementation of the present application, the gesture recognition method disclosed in the implementation of the present application, the correspondence between the edge contour of a human hand and the edge contour of an arm, and the correspondence between the edge contour of a human body and the edge contour of clothing are obtained in the following manner: obtaining the edge contour of a human hand according to the human hand, and determining the correspondence between the edge contour of the human hand and the edge contour of the arm according to the edge contour of the human hand and the edge contour of the arm; obtaining the edge contour of a human body according to the human body, and determining the correspondence between the edge contour of the human body and the edge contour of clothing according to the edge contour of the human body and the edge contour of clothing.

[0020] By adopting the above technical scheme, the edge contour of the hand and the edge contour of the body are obtained respectively based on the hand and the body of the user in the target image, and the correspondence between the edge contour of the hand and the edge contour of the arm, and the correspondence between the edge contour of the body and the edge contour of the clothing are determined, and then the body corresponding to the hand and the left and right position of the hand relative to the body are determined, so that the user corresponding to the hand and the left and right position of the hand relative to the user can be determined more accurately.

[0021] In a second aspect, the implementation of the present application further discloses a gesture control method, which includes: determining a user's gesture type, the gesture type being determined based on a gesture recognition method provided by any implementation of the first aspect; and performing corresponding gesture control processing based on the gesture type.

[0022] By adopting the above technical solution, gesture control processing is performed based on the determined user's gesture type, and a more accurate gesture type can be obtained while accurately identifying the user to whom the gesture belongs and whether the gesture is a left-hand gesture or a right-hand gesture, thereby improving the accuracy of human-computer interaction processing such as gesture control.

[0023] In a third aspect, an implementation of the present application further discloses an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores a computer program, and the processor executes the computer program stored in the memory, so that the electronic device implements a gesture recognition method as provided in any implementation of the first aspect above, and / or implements a gesture control method as provided in any implementation of the second aspect.

[0024] In a fourth aspect, the implementation method of the present application further discloses a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, it is used to implement the gesture recognition method provided in any one of the implementation methods of the first aspect above, and / or to implement the gesture control method provided in any one of the implementation methods of the second aspect.

[0025] In a fifth aspect, the implementation method of the present application also discloses a computer program product, including a computer program, which, when executed by a processor, implements the gesture recognition method provided in any one of the implementation methods of the first aspect above, and / or implements the gesture control method provided in any one of the implementation methods of the second aspect.

[0026] It can be understood that the beneficial effects of the third to fifth aspects mentioned above can also be found in the relevant descriptions in the first and / or second aspects mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a flowchart of a gesture recognition method provided in an embodiment of the present application;

[0028] Figure 2 It is a flowchart of a method for determining a human body corresponding to a human hand and the left and right positions of the human hand relative to the human body provided in an embodiment of the present application;

[0029] Figure 3 It is a schematic flow chart of a method for determining the inflection point of an arm edge contour and the inflection point of a clothing edge contour provided in an embodiment of the present application;

[0030] Figure 4 It is a schematic flow chart of a method for determining discrete points of an arm edge contour and discrete points of a clothing edge contour provided in an embodiment of the present application;

[0031] Figure 5 It is a flowchart of another method for determining the human body corresponding to a human hand and the left and right positions of the human hand relative to the human body provided in an embodiment of the present application;

[0032] Figure 6 is a flowchart of a gesture control method provided in an embodiment of the present application;

[0033] Figure 7 is a flowchart of another gesture control method provided in an embodiment of the present application;

[0034] Figure 8 This is an effect diagram of a successful matching of the segment curves of the arm edge contour and the clothing edge contour provided by the embodiment of the present application;

[0035] Fig. 9 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] As mentioned above, the existing method of performing human-computer interaction processing such as controlling electronic devices according to the gesture type after recognizing the user gesture according to the captured scene image has the following two problems. On the one hand, in a multi-user scenario, there is a problem of not being able to accurately determine the user to whom the gesture belongs, and on the other hand, there is usually a problem of not being able to accurately identify whether the gesture is a left-hand gesture or a right-hand gesture.

[0037] Therefore, the existing gesture recognition method has the problem of inaccurate gesture recognition, which affects the accuracy of human-computer interaction processing such as gesture control.

[0038] Based on this, the implementation method of the present application provides a gesture recognition method and a gesture control method. Based on the identification of the human hand corresponding to the user and the left and right positions of the human hand relative to the user, it is possible to more accurately determine the user to which the gesture belongs and whether the gesture is a left-hand gesture or a right-hand gesture, thereby obtaining a more accurate gesture type, improving the accuracy of gesture recognition, and thereby improving the accuracy of human-computer interaction processing such as gesture control.

[0039] Next, with reference to the accompanying drawings, the gesture recognition method provided by the implementation of the present application is described in detail.

[0040] See also Figure 1 The gesture recognition method provided by the implementation of this application specifically includes the following steps.

[0041] S100, acquiring a target image, where the target image includes multiple users.

[0042] S200, identifying the face, hands, and human body other than the face and hands of the user in the target image.

[0043] S300, determining the user corresponding to the face and the body based on the area where the face and the body are located, and the correspondence between the preset area and the user.

[0044] S400, obtaining an arm edge contour and a clothing edge contour according to the human body, and determining the human body corresponding to the hand and the left and right position of the hand relative to the human body according to the arm edge contour, the clothing edge contour, and the human hand.

[0045] S500, determining the user corresponding to the hand and the left and right position of the hand relative to the user according to the correspondence between the hand and the human body, the left and right position of the hand relative to the human body, and the correspondence between the face, the human body and the user.

[0046] S600: Determine a gesture type according to the user corresponding to the human hand and the left and right position of the human hand relative to the user.

[0047] The gesture recognition method provided by the implementation of the present application, in a multi-user scenario, the acquired target image includes multiple users. In order to accurately recognize the gesture, the face, hand and body of the user in the target image are first recognized, and then the user corresponding to the face and body is determined according to the area where the face and body are located, and the corresponding relationship between the preset area and the user, and the arm edge contour and clothing edge contour obtained from the human body, and the human hand, the human body corresponding to the hand, and the left and right positions of the hand relative to the human body are determined, and then according to the corresponding relationship between the hand and the human body, the left and right positions of the hand relative to the human body, and the corresponding relationship between the face and the human body and the user, the user corresponding to the hand and the left and right positions of the hand relative to the human body are determined, and the gesture type is determined based on the user corresponding to the hand and the left and right positions of the hand relative to the user. In this way, the user corresponding to the hand and the left and right positions of the hand relative to the user can be more accurately identified, and the user to which the gesture belongs and whether the gesture is a left-hand gesture or a right-hand gesture can be more accurately determined, that is, it can be more accurately determined whether the hand is a left-hand gesture or a right-hand gesture of a certain user, so that a more accurate gesture type can be obtained, the accuracy of gesture recognition is improved, and then the accuracy of human-computer interaction processing such as gesture control is improved.

[0048] Exemplarily, regarding the above step S100, a target image is acquired, and the target image includes a plurality of users. The target image including the plurality of users may be acquired by a pre-set camera.

[0049] Further, with respect to the above-mentioned step S200, the user's face, hands, and human bodies other than the face and hands in the target image are identified by detecting the face, hands, and human bodies in the target image through a pre-deployed target detection model, thereby obtaining the face, hands, and human bodies other than the face and hands in the target image.

[0050] Exemplarily, the target detection model can be a face detection model, a hand detection model, or a human body detection model based on a neural network model. Of course, the target detection model can also be other models.

[0051] Further, regarding the above-mentioned step S300, based on the areas where the face and body are located, and the correspondence between the preset areas and the users, determining the users corresponding to the faces and body can be based on the areas where the faces and body are located in the acquired target image, and taking the users corresponding to the areas in the target image as the correspondence between the areas and the users, thereby determining the users corresponding to the faces and body based on the areas where the face and body are located and the correspondence between the preset areas and the users.

[0052] In this way, the captured target image is detected, and after obtaining the face, hand and body in the target image, the detected face and body are matched to the corresponding user according to the area where the face and body are located and the correspondence between the area and the user, so that the face, body and user in the target image correspond one to one.

[0053] Furthermore, regarding the above step S400, the arm edge contour and the clothing edge contour are obtained based on the human body, and the human body image in the target image can be segmented by a pre-deployed image segmentation model to obtain the arm edge contour and the clothing edge contour.

[0054] Also, see Figure 2 In one implementation of the present application, regarding the above step S400, the human body corresponding to the hand and the left and right positions of the hand relative to the human body are determined according to the edge contour of the arm and the edge contour of the clothing, as well as the human hand, including the following steps.

[0055] S410, determining the inflection point of the arm edge contour and the inflection point of the clothing edge contour.

[0056] For example, see Figure 3 In one implementation of the present application, determining the inflection point of the arm edge contour and the inflection point of the clothing edge contour includes:

[0057] S411, determining discrete points of the arm edge contour and discrete points of the clothing edge contour.

[0058] For example, see Figure 4 In one implementation of the present application, determining discrete points of the edge contour of the arm and discrete points of the edge contour of the clothing includes:

[0059] S101, determining edge points of the arm edge contour and edge points of the clothing edge contour;

[0060] S102, discretizing the edge points of the arm edge contour and the edge points of the clothing edge contour to obtain discrete points of the arm edge contour and the discrete points of the clothing edge contour.

[0061] Exemplarily, in the implementation method of the present application, the edge contour of the arm and the edge contour of the clothing are first identified to obtain the edge points of the arm edge contour and the edge points of the clothing edge contour, and then the edge points of the arm edge contour and the edge points of the clothing edge contour are discretized to obtain discrete points of the arm edge contour and the discrete points of the clothing edge contour, so as to facilitate the subsequent calculation of the inflection point.

[0062] S412, performing a first curvature calculation on discrete points of the arm edge contour to obtain a first curvature calculation value.

[0063] Exemplarily, the first curvature calculation is performed on the discrete points of the arm edge contour, and the first curvature calculation can be performed by a difference method, a tangent method, a curve fitting method, etc. to obtain a corresponding first curvature calculation value. Of course, the first curvature calculation value can also be calculated by other methods.

[0064] For example, the first curvature calculation value is obtained by the difference method, which can be based on the discrete points of the determined arm edge contour. The first-order difference of each discrete point is first calculated, that is, the difference between the selected discrete point and the adjacent discrete point is calculated to obtain the first-order difference of the discrete point, and then the rate of change of the first-order difference is calculated to obtain the second-order difference of each discrete point, that is, the difference between the first-order difference of the selected discrete point and the first-order difference of the adjacent discrete point is calculated to obtain the second-order difference of the discrete point, so as to use the first-order difference and the second-order difference to obtain the curvature value (that is, the first curvature calculation value) through the curvature calculation formula.

[0065] S413, obtaining the inflection point of the arm edge contour based on the first curvature calculation value.

[0066] Exemplarily, by performing a first curvature calculation on discrete points of the arm edge contour, a first curvature calculation value of each point (ie, discrete point) is obtained, and then the point where the curvature value (ie, the first curvature calculation value) changes drastically is taken as an inflection point, thereby obtaining the inflection point of the arm edge contour.

[0067] S414, performing a second curvature calculation on discrete points of the edge contour of the clothing to obtain a second curvature calculation value.

[0068] For example, the second curvature calculation for discrete points of the edge contour of the clothing may be performed by a difference method, a tangent method, a curve fitting method, etc. to obtain a corresponding second curvature calculation value. Of course, the second curvature calculation value may also be calculated by other methods.

[0069] For example, the second curvature calculation value is obtained by the difference method, which can be based on the discrete points of the determined clothing edge contour. The first-order difference of each discrete point is first calculated, that is, the difference between the selected discrete point and its adjacent discrete point is calculated to obtain the first-order difference of the discrete point, and then the rate of change of the first-order difference is calculated to obtain the second-order difference of each discrete point, that is, the difference between the first-order difference of the selected discrete point and the first-order difference of the adjacent discrete point is calculated to obtain the second-order difference of the discrete point, so as to use the first-order difference and the second-order difference to obtain the curvature value (that is, the second curvature calculation value) through the curvature calculation formula.

[0070] S415: Obtain an inflection point of the edge contour of the clothing based on the second curvature calculation value.

[0071] Exemplarily, by performing a second curvature calculation on discrete points of the edge contour of clothing, a second curvature calculation value of each point (ie, discrete point) is obtained, and then the point where the curvature value (ie, the second curvature calculation value) changes dramatically is taken as an inflection point, thereby obtaining the inflection point of the edge contour of clothing.

[0072] In this way, the above calculation method is used to calculate the corresponding curvatures of the discrete points of the arm edge contour and the discrete points of the clothing edge contour to obtain the inflection points of the arm edge contour and the inflection points of the clothing edge contour.

[0073] S420, obtaining a plurality of segment curves of the arm edge contour and a plurality of segment curves of the clothing edge contour according to the inflection points of the arm edge contour and the inflection points of the clothing edge contour.

[0074] Exemplarily, in the implementation method of the present application, after obtaining the inflection point of the arm edge contour and the inflection point of the clothing edge contour by curvature calculation, the arm edge contour and the clothing edge contour are divided according to the inflection point to obtain multiple segment curves of the arm edge contour and multiple segment curves of the clothing edge contour, wherein the segment curve of the arm edge contour does not exceed 6 segments, and the segment curve of the clothing edge contour does not exceed 12 segments.

[0075] For example, based on the calculated inflection points, the small curve segments (ie, segment curves) 1, 2, 3 of the right arm edge contour and the small curve segments 1', 2', 3' of the right clothing edge contour are obtained.

[0076] S430, calculating the similarity and distance between each segment curve of the arm edge contour and each segment curve of the clothing edge contour, and obtaining corresponding curve similarity and curve distance.

[0077] Exemplarily, in one implementation of the present application, the similarity and distance between each segment curve of the arm edge contour and each segment curve of the clothing edge contour are calculated to obtain the corresponding curve similarity and curve distance, including: calculating the similarity between each segment curve of the arm edge contour and each segment curve of the clothing edge contour based on a curve similarity algorithm to obtain the corresponding curve similarity; and calculating the distance between each segment curve of the arm edge contour and each segment curve of the clothing edge contour based on a curve distance algorithm to obtain the corresponding curve distance.

[0078] Furthermore, in one implementation of the present application, the curve similarity algorithm is a dynamic time warping (DTW) algorithm, and the curve distance algorithm is a Hausdorff algorithm. Of course, the curve similarity algorithm and the curve distance algorithm may also be other algorithms for calculating curve similarity and curve distance.

[0079] Exemplarily, by using the DTW algorithm and the Hausdorff algorithm, similarities and distances are calculated for each segment curve of the arm edge contour and each segment curve of the clothing edge contour one by one, and corresponding curve similarities and curve distances are obtained.

[0080] For example, the segment curves 1, 2, 3 of the right arm edge contour and the segment curves 1', 2', 3' of the right clothing edge contour are obtained, and then the similarities and distances between 1, 2, 3 and 1', 2', 3' are calculated one by one to obtain the corresponding curve similarities and curve distances. That is, a distance matrix is ​​constructed based on 1 and 1', and then the optimal alignment path between 1 and 1' is obtained based on the distance matrix, and the cumulative distance of the optimal alignment path is used as the curve similarity between 1 and 1', and the same processing is performed on 1 and 2', 1 and 3', 2 and 1', 2 and 2', 2 and 3', 3 and 1', 3 and 2', and 3 and 3' to obtain the corresponding curve similarities of 1 and 2', 1 and 3', 2 and 1', 2 and 2', 2 and 3', 3 and 1', 3 and 2', and 3 and 3'; calculate 1 Calculate the distance between each point in 1` and the nearest point in 1`, and similarly calculate the distance between each point in 1` and the nearest point in 1, and then find the largest distance among the distances of these nearest points as the curve distance between 1 and 1`, and perform the same process on 1 and 2`, 1 and 3`, 2 and 1`, 2 and 2`, 2 and 3`, 3 and 1`, 3 and 2`, and 3 and 3` to obtain the corresponding curve distances of 1 and 2`, 1 and 3`, 2 and 1`, 2 and 2`, 2 and 3`, 3 and 1`, 3 and 2`, and 3 and 3`.

[0081] S440, determining the human body to which the hand corresponds, and the left and right position of the hand relative to the human body, based on the curve similarity, the curve distance, the human hand, and the preset correspondence between the edge contour of the hand and the edge contour of the arm, and the correspondence between the edge contour of the human body and the edge contour of the clothing.

[0082] For example, see Figure 5 In another implementation of the present application, according to the curve similarity, the curve distance, the human hand, and the preset correspondence between the edge contour of the human hand and the edge contour of the arm, and the correspondence between the edge contour of the human body and the edge contour of the clothing, the human body corresponding to the human hand and the left and right position of the human hand relative to the human body are determined, including:

[0083] S441, when the curve similarity is greater than the preset curve similarity threshold and the curve distance is less than the preset curve distance threshold, the clothing edge contour corresponding to the arm edge contour and the left and right positions of the arm edge contour relative to the clothing edge contour are obtained.

[0084] Exemplarily, when the curve similarity is greater than a preset curve similarity threshold and the curve distance is less than a preset curve distance threshold, the clothing edge contour corresponding to the arm edge contour and the left and right position of the arm edge contour relative to the clothing edge contour can be determined.

[0085] For example, the curve similarities and curve distances corresponding to 1 and 1`, 1 and 2`, 1 and 3`, 2 and 1`, 2 and 2`, 2 and 3`, 3 and 1`, 3 and 2`, and 3 and 3` are obtained, and then the corresponding curve similarities and curve distances are matched one by one, and it is obtained that the curve similarity between 2 and 2` is greater than the preset threshold s (that is, the curve similarity threshold) and the curve distance is less than the preset threshold d (that is, the curve distance threshold), and the curve similarity between 3 and 2` is greater than the preset threshold s (that is, the curve similarity threshold) and the curve distance is less than the preset threshold d (that is, the curve distance threshold), so that the edge contour of the right arm corresponds to the edge contour of the right clothing.

[0086] Among them, the curve similarity threshold s and the curve distance threshold d can be set based on actual needs.

[0087] S442, determining the clothing edge contour corresponding to the hand edge contour, and the left and right position of the hand edge contour relative to the clothing edge contour based on the correspondence between the arm edge contour and the clothing edge contour, the left and right position of the arm edge contour relative to the clothing edge contour, and the preset correspondence between the hand edge contour and the arm edge contour.

[0088] S443, determining the human body to which the human hand corresponds, and the left and right position of the human hand relative to the human body, based on the correspondence between the edge contour of the human hand and the edge contour of the clothing, the left and right position of the edge contour of the human hand relative to the edge contour of the clothing, the human hand, and the preset correspondence between the edge contour of the human body and the edge contour of the clothing.

[0089] Exemplarily, in one implementation of the present application, the correspondence between the edge contour of a human hand and the edge contour of an arm, and the correspondence between the edge contour of a human body and the edge contour of clothing are obtained in the following manner: obtaining the edge contour of a human hand based on the human hand, and determining the correspondence between the edge contour of the human hand and the edge contour of the arm based on the edge contour of the human hand and the edge contour of the arm; obtaining the edge contour of a human body based on the human body, and determining the correspondence between the edge contour of the human body and the edge contour of clothing based on the edge contour of the human body and the edge contour of clothing.

[0090] Exemplarily, in the implementation of the present application, a hand edge contour is obtained according to a hand, and then the correspondence between the hand edge contour and the arm edge contour is determined according to the hand edge contour and the arm edge contour. This may be by performing image segmentation on the hand image in the target image using a pre-deployed image segmentation model to obtain the hand edge contour, and then matching the hand edge contour with the arm edge contour to obtain the correspondence between the hand edge contour and the arm edge contour; and a human body edge contour is obtained according to a human body, and then the correspondence between the human body edge contour and the clothing edge contour is determined according to the human body edge contour and the clothing edge contour. This may be by performing image segmentation on the human body image in the target image using a pre-deployed image segmentation model to obtain the human body edge contour, and then matching the human body edge contour with the clothing edge contour to obtain the correspondence between the human body edge contour and the clothing edge contour.

[0091] Furthermore, based on the correspondence between the arm edge contour and the clothing edge contour, the left and right position of the arm edge contour relative to the clothing edge contour, and the correspondence between the hand edge contour and the arm edge contour, the clothing edge contour corresponding to the hand edge contour and the left and right position of the hand edge contour relative to the clothing edge contour are obtained; then, based on the clothing edge contour corresponding to the hand edge contour, the left and right position of the hand edge contour relative to the clothing edge contour, and the correspondence between the human body edge contour and the clothing edge contour, the human body corresponding to the human hand and the left and right position of the hand relative to the human body are obtained.

[0092] In this way, based on the edge contour of the arm, the edge contour of the clothing, the edge contour of the hand, and the edge contour of the human body, the human body corresponding to the hand and the left and right position of the hand relative to the human body are determined, and then the user corresponding to the hand and the left and right position of the hand relative to the user are determined. In a multi-user scenario, based on the user's arms, clothing, hands, and human body, the user to whom the gesture belongs and whether the gesture is a left-hand gesture or a right-hand gesture can be more accurately determined.

[0093] Further, regarding the above-mentioned step S500, the user corresponding to the hand and the left-right position of the hand relative to the user are determined based on the correspondence between the hand and the human body, the left-right position of the hand relative to the human body, and the correspondence between the face, the human body and the user. The correspondence between the face, the human body and the user can be obtained first based on the region where the face and the human body are located and the correspondence between the region and the user, and then the correspondence between the hand and the human body and the left-right position of the hand relative to the human body can be obtained based on the edge contour of the arm and the edge contour of the clothes. Then, the user corresponding to the hand and the left-right position of the hand relative to the user can be obtained based on the correspondence between the face, the human body and the user, the correspondence between the hand and the human body and the left-right position of the hand relative to the human body.

[0094] Further, regarding the above-mentioned step S600, the gesture type is determined based on the user corresponding to the hand and the left and right position of the hand relative to the user. The above-mentioned corresponding relationship can be determined first, and then the user corresponding to the hand and the left and right position of the hand relative to the user are determined based on the corresponding relationship, and then the user's left hand gesture or the user's right hand gesture is determined.

[0095] The gesture recognition method provided by the implementation of the present application is based on target images including multiple users and the determined correspondence between faces and bodies and users, the correspondence between hands and bodies, and the left and right positions of hands relative to bodies. It can accurately determine the user to which the hand corresponds and the left and right positions of the hand relative to the user. It can accurately determine the user to whom the gesture belongs and whether the gesture is a left-hand gesture or a right-hand gesture, thereby obtaining a more accurate gesture type, improving the accuracy of gesture recognition, and further improving the accuracy of human-computer interaction processing such as control.

[0096] Furthermore, the implementation of the present application also provides a gesture control method, see Figure 6 , specifically including the following steps.

[0097] S10, determining the gesture type of the user.

[0098] Exemplarily, the aforementioned gesture recognition method is used to determine the user's gesture type.

[0099] S20, performing corresponding gesture control processing based on the gesture type.

[0100] Exemplarily, gesture control processing based on a determined user's gesture type can obtain a more accurate gesture type while accurately identifying the user to whom the gesture belongs and whether the gesture is a left-hand gesture or a right-hand gesture, thereby improving the accuracy of human-computer interaction processing such as gesture control.

[0101] Furthermore, in another implementation of the present application, the gesture control method can also be applied to vehicles to realize gesture recognition of users in the vehicle.

[0102] For example, see Figure 7 , the gesture control method specifically includes the following steps.

[0103] S1, find the user position of the control area in the target image (that is, the corresponding relationship between the area and the user) through the full-vehicle monitoring camera in the car, where the control area is determined according to the seat position in the car, including the main driver control area and the co-driver control area, and detect the face, hand and body in the car to obtain the face, hand and body in the target image, and then create a temporary ID for the face to bind the face ID with the hand, and bind the body with the face ID according to the positional relationship between the face and the body (that is, the area where the face and the body are located).

[0104] Exemplarily, after determining the face, hand and body in the target image, a temporary ID is created for the face, and based on the area where the face and body are located, the body is bound to the face ID. Then, based on the correspondence between the area and the user, the binding relationship between the hand and the face ID, and the binding relationship between the body and the face ID, the correspondence between the hand, face ID, body and the user, and whether the user is in the driver's seat, the co-driver's seat or other position in the car can be determined.

[0105] S2, using the in-vehicle full-vehicle monitoring camera to perform image segmentation on the human body appearing in the in-vehicle camera, obtain arms and clothes (that is, the edge contour of the arm and the edge contour of the clothes), and identify the main driver's clothing and the co-driver's clothing according to the matching degree and size of the human body and the clothes, and then discretize the edge points of the main driver's clothing, the co-driver's clothing and the arms to obtain discrete points corresponding to the main driver's clothing, the co-driver's clothing and the arms, and calculate the curvature of the discrete points to obtain the inflection points of the segmented edges corresponding to the main driver's clothing, the co-driver's clothing and the arms, and then divide different small segments of curves (that is, segment curves) according to the inflection points, wherein the small segment curves of the arms do not exceed 6 segments, and the small segment curves of the clothes do not exceed 12 segments.

[0106] Exemplarily, after obtaining the edge contour of the arm and the edge contour of the clothing based on the human body and determining the correspondence between the driver's seat, the co-driver's seat and the clothing edge contour, the edge points corresponding to the arm edge contour and the driver's and co-driver's clothing edge contours are discretized, and after obtaining the corresponding discrete points, the curvature of the discrete points is calculated to obtain the corresponding inflection points, and then based on the inflection points, multiple segment curves of the arm edge contour and multiple segment curves of the driver's and co-driver's clothing edge contours are obtained.

[0107] S3, through the DTW (dynamic time warping) algorithm and the Hausdorff algorithm, the curve similarity and curve distance of each small segment curve of all arms and each small segment curve on the left and right sides of the driver's clothing and the co-driver's clothing are calculated to obtain the corresponding curve similarity and curve distance (the curve similarity and curve distance can also be calculated by other methods), so that when a certain segment curve of the arm and a certain segment curve on the left and right sides of the driver's or co-driver's clothing simultaneously meet the curve similarity greater than s (that is, the preset curve similarity threshold) and the curve distance less than d (that is, the preset curve distance threshold ), determine the correspondence between a certain curve of the arm and a certain curve on the left and right sides of the driver's or co-driver's clothing (that is, the correspondence between the edge contour of the arm and the edge contour of the driver's or co-driver's clothing, and the left and right positions relative to the edge contour of the clothing), and then determine the correspondence between the human hand and the left and right sides of the driver's or co-driver's clothing (that is, the correspondence between the edge contour of the human hand and the edge contour of the driver's or co-driver's clothing, and the left and right positions relative to the edge contour of the clothing) according to the correspondence between the human hand and the arm obtained based on image segmentation (that is, the correspondence between the edge contour of the human hand and the edge contour of the arm).

[0108] Exemplarily, the similarity and distance between the curves of each segment of the arm edge contour and the curves of the edge contour of the driver's or passenger's clothes are calculated based on an algorithm. After the curve similarity and curve distance are obtained, when the curve similarity is greater than a preset curve similarity threshold and the curve distance is less than the curve distance threshold, the correspondence between the arm edge contour and the edge contour of the driver's or passenger's clothes, and the left and right positions of the arm edge contour relative to the edge contour of the driver's or passenger's clothes are determined. Then, based on the correspondence between the hand edge contour and the arm edge contour, the correspondence between the hand edge contour and the edge contour of the driver's or passenger's clothes, and the left and right positions of the hand edge contour relative to the edge contour of the driver's or passenger's clothes are determined.

[0109] For example, Figure 8 As shown, based on the inflection point determined above, the segment curves of the arm edge contour at the main driver's position (that is, the potential control arm segment curves) 1, 2, 3 (that is, arm curves 1, 2, 3) and the segment curves on the right side of the main driver's clothing edge contour (that is, the main driver's right clothing segment curves) 1, 2, 3 (that is, clothing curves 1, 2, 3) are obtained, and then the arm segment curves 1, 2, 3 are matched with the clothing segment curves 1, 2, 3 one by one. Based on the calculation, it can be seen that the arm segment curves 2 and 3 and the clothing segment curve 2 both meet the curve similarity greater than s and the curve distance less than d, and then the arm is determined to be the arm on the right side of the main driver's clothing. Among them, the potential control arm is determined based on the correspondence between the human hand and the arm obtained based on image segmentation, that is, only the arm that can see the palm of the human hand is the potential control arm.

[0110] S4, according to the correspondence between the human hand and the left and right sides of the driver's or co-driver's clothing, the correspondence between the human body and the clothing obtained based on image segmentation (that is, the correspondence between the human body edge contour and the clothing edge contour), and the binding relationship between the human hand and the face ID, and the binding relationship between the human body and the face ID, determine the gesture type to perform gesture control in diversified scenarios.

[0111] Exemplarily, based on the correspondence between the edge contour of a human hand and the edge contour of the driver's or co-driver's clothing, the left and right position of the edge contour of the human hand relative to the edge contour of the driver's or co-driver's clothing, the correspondence between the edge contour of the human body and the edge contour of the clothing, and the correspondence between the hand, face ID, body and user determined based on the binding relationship, and the user being in the driver's seat, co-driver seat or other positions in the car, the gesture type is determined, that is, the left-hand gesture or right-hand gesture of the user corresponding to the face ID when in the driver's seat, or the left-hand gesture or right-hand gesture of the user corresponding to the face ID when in the co-driver seat, and gesture control is performed based on the gesture type.

[0112] The gesture control method provided in the implementation mode of the present application determines the face, hand and body in the target image, and after creating an ID for the face, based on the binding relationship between the hand and the face ID, the binding relationship between the body and the face ID and the user position to which the control area belongs (that is, the corresponding relationship between the area and the user), determines the correspondence between the hand, the face ID, the body and the user, and whether the user is in the main driver's seat or the co-driver's seat or other positions in the car, and then determines the correspondence between the hand and the left and right sides of the main driver's or co-driver's clothing through the arms and clothing (that is, the arm edge contour and the clothing edge contour) obtained by image segmentation of the human body. (that is, the correspondence between the edge contour of the human hand and the edge contour of the driver's or co-driver's clothes, and the left and right positions relative to the edge contour of the clothes), and then based on the correspondence between the human body and the clothes obtained based on image segmentation (that is, the correspondence between the edge contour of the human body and the edge contour of the clothes), determine the correspondence between the human hand and the driver's or co-driver's body and the left and right positions of the human hand relative to the human body, and then determine the gesture type according to the correspondence between the human hand, face ID, the body and the user, and whether the user is in the driver's or co-driver's position in the car, as well as the correspondence between the human hand and the driver's or co-driver's body, and the left and right positions of the human hand relative to the human body.

[0113] In this way, based on the binding relationship between the face ID and the human hand and the human body, as well as the correspondence between the human hand and the segmented arm through image segmentation, the curve similarity and curve distance calculated, and the segmented arm, the correspondence between the human hand and the left and right sides of the driver's or co-driver's clothing is determined. According to the correspondence between the human body and the segmented clothing, the left-hand gesture or right-hand gesture of the user when he is in the driver's or co-driver position can be determined. This makes it possible to accurately distinguish the driver's and co-driver control areas and the user's left and right hands when multiple hands are performing gesture control at the same time in the car, so as to accurately identify whether the gesture is a left-hand gesture or a right-hand gesture, which can meet the user's gesture control needs in different scenarios, and perform gesture recognition and control based on the driver's and co-driver control areas, which can eliminate interference from the rear control area.

[0114] The gesture recognition method and gesture control method provided by the implementation of the present application can be applied to gesture recognition in a vehicle smart cockpit, and can also be applied to gesture recognition scenarios corresponding to other electronic devices such as mobile phones.

[0115] See also Fig. 9 , Fig. 9 The figure shows a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig. 9 As shown, the electronic device may include: a transceiver 121 , a processor 122 , and a memory 123 .

[0116] The processor 122 executes the computer execution instructions stored in the memory, so that the processor 122 executes part of the technical solutions of the vehicle offline detection method in the above embodiment. The processor 122 can be a general-purpose processor, including a central processing unit CPU, a network processor (NP), etc.; it can also be a digital data processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0117] The memory 123 is connected to the processor 122 via a system bus and completes communication between them. The memory 123 is used to store computer program instructions.

[0118] By way of example and not limitation, the memory 123 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 123 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 123 may be inside or outside the integrated gateway device. In a particular embodiment, the memory 123 is a non-volatile solid-state memory. In a particular embodiment, the memory 123 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0119] The transceiver 121 may be used to obtain tasks to be executed and configuration information of the tasks to be executed.

[0120] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory.

[0121] The embodiment of the present application also provides a chip for running instructions, which is used to execute the technical solutions of the gesture recognition method and / or gesture control method in the above embodiments.

[0122] An embodiment of the present application also provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on a processor of an electronic device, the processor of the electronic device executes the technical solution of the gesture recognition method and / or gesture control method of the above-mentioned embodiment.

[0123] In some possible implementations, various aspects of the method provided in the present application may also be implemented in the form of a program product, which includes program code. When the program product runs on a processor of an electronic device, the program code is used to enable the processor of the electronic device to execute the steps of the method according to various exemplary implementations of the present application described above in this specification. For example, the electronic device may execute the gesture recognition method and / or gesture control method described in the embodiments of the present application.

[0124] The program product may employ any combination of one or more readable media. The readable medium may be a readable data medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0125] The implementation method of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When the at least one processor executes the computer program, the technical solutions of the gesture recognition method and / or gesture control method in the above-mentioned embodiments can be implemented.

[0126] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable information processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable information processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0127] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable information processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0128] These computer program instructions can also be loaded onto a computer or other programmable information processing device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0129] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary technical means in the art that are not disclosed in the present application.

[0130] It should be noted that, in addition to the implementation of the present application described in the above-mentioned specific embodiments, those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. Although the description of the present application is introduced in conjunction with the preferred embodiment, this does not mean that the features of this invention are limited to the implementation. On the contrary, the purpose of introducing the invention in conjunction with the implementation is to cover other options or modifications that may extend based on the claims of the present application. In order to provide an in-depth understanding of the present application, the above description contains many specific details, and the present application may also be implemented without using these details. In addition, in order to avoid confusion or blurring the focus of the present application, some specific details will be omitted in the description. It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0131] It should be noted that in this specification, similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0132] The terms “first”, “second”, etc. are only used for distinguishing descriptions and should not be understood as indicating or implying relative importance.

[0133] Although the present application has been illustrated and described by referring to certain preferred implementations of the present application, it should be understood by those skilled in the art that the above contents are further detailed descriptions of the present application in combination with specific implementations, and it cannot be determined that the specific implementation of the present application is limited to these descriptions. Those skilled in the art may make various changes in form and details, including making several simple deductions or substitutions, without departing from the spirit and scope of the present application.

Claims

1. A gesture recognition method, characterized in that: The method comprises: Acquire a target image, wherein the target image includes a plurality of users; Recognize the face, hands, and human body other than the face and hands of the user in the target image; Determine the user corresponding to the face and the body based on the area where the face and the body are located and the correspondence between the preset area and the user; Obtaining an arm edge contour and a clothing edge contour according to the human body, and determining the human body corresponding to the human hand and the left and right positions of the human hand relative to the human body according to the arm edge contour, the clothing edge contour, and the human hand; Determine the user to which the human hand corresponds, and the left-right position of the human hand relative to the user, according to the correspondence between the human hand and the human body, the left-right position of the human hand relative to the human body, and the correspondence between the human face, the human body and the user; The gesture type is determined according to the user corresponding to the human hand and the left and right position of the human hand relative to the user.

2. The gesture recognition method according to claim 1, characterized in that: Determining the human body corresponding to the human hand and the left and right positions of the human hand relative to the human body according to the edge contour of the arm and the edge contour of the clothing, as well as the human hand, includes: Determining the inflection point of the arm edge contour and the inflection point of the clothing edge contour; According to the inflection point of the arm edge contour and the inflection point of the clothing edge contour, a plurality of segment curves of the arm edge contour and a plurality of segment curves of the clothing edge contour are obtained; Calculating the similarity and distance between each segment curve of the arm edge contour and each segment curve of the clothing edge contour to obtain corresponding curve similarity and curve distance; The human body to which the human hand corresponds and the left and right position of the human hand relative to the human body are determined based on the curve similarity, the curve distance, the human hand, and the correspondence between the preset human hand edge contour and the arm edge contour, and the correspondence between the human body edge contour and the clothing edge contour.

3. The gesture recognition method according to claim 2, characterized in that: Determining the inflection point of the arm edge contour and the inflection point of the clothing edge contour comprises: Determine discrete points of the arm edge contour and discrete points of the clothing edge contour; Performing a first curvature calculation on discrete points of the arm edge contour to obtain a first curvature calculation value; Based on the first curvature calculation value, obtaining the inflection point of the arm edge contour; and Performing a second curvature calculation on discrete points of the edge contour of the clothing to obtain a second curvature calculation value; Based on the second curvature calculation value, the inflection point of the edge contour of the clothing is obtained.

4. The gesture recognition method according to claim 3, characterized in that: Determining discrete points of the arm edge contour and discrete points of the clothing edge contour includes: Determine edge points of the arm edge contour and edge points of the clothing edge contour; Discretization processing is performed on the edge points of the arm edge contour and the edge points of the clothing edge contour to obtain discrete points of the arm edge contour and the discrete points of the clothing edge contour.

5. The gesture recognition method according to any one of claims 2 to 4, characterized in that: Calculating the similarity and distance between each segment curve of the arm edge contour and each segment curve of the clothing edge contour to obtain corresponding curve similarity and curve distance, including: Calculating the similarity between each segment curve of the arm edge contour and each segment curve of the clothing edge contour based on a curve similarity algorithm to obtain corresponding curve similarity; and The distances between the segment curves of the arm edge contour and the segment curves of the clothing edge contour are calculated based on a curve distance algorithm to obtain corresponding curve distances.

6. The gesture recognition method according to claim 5, characterized in that: The curve similarity algorithm is a dynamic time warping algorithm, and the curve distance algorithm is a Hausdorff algorithm.

7. The gesture recognition method according to any one of claims 2 to 6, characterized in that: Determining the human body corresponding to the human hand and the left and right positions of the human hand relative to the human body according to the curve similarity, the curve distance, the human hand, and the preset correspondence between the human hand edge contour and the arm edge contour, and the correspondence between the human body edge contour and the clothing edge contour, including: When the curve similarity is greater than a preset curve similarity threshold, and the curve distance is less than a preset curve distance threshold, the clothing edge contour corresponding to the arm edge contour and the left and right positions of the arm edge contour relative to the clothing edge contour are obtained; Determine the clothing edge contour corresponding to the human hand edge contour, and the left and right position of the human hand edge contour relative to the clothing edge contour, according to the corresponding relationship between the arm edge contour and the clothing edge contour, the left and right position of the arm edge contour relative to the clothing edge contour, and the preset corresponding relationship between the human hand edge contour and the arm edge contour; The human body to which the human hand corresponds and the left and right position of the human hand relative to the human body are determined based on the correspondence between the edge contour of the human hand and the edge contour of the clothing, the left and right position of the human hand edge contour relative to the clothing edge contour, the human hand, and the correspondence between the preset human body edge contour and the clothing edge contour.

8. The gesture recognition method according to claim 7, characterized in that: The corresponding relationship between the edge contour of the hand and the edge contour of the arm, and the corresponding relationship between the edge contour of the human body and the edge contour of the clothing are obtained by: Obtaining a human hand edge contour according to the human hand, and determining a corresponding relationship between the human hand edge contour and the arm edge contour according to the human hand edge contour and the arm edge contour; A human body edge contour is obtained according to the human body, and a corresponding relationship between the human body edge contour and the clothing edge contour is determined according to the human body edge contour and the clothing edge contour.

9. A gesture control method, characterized in that: The method comprises: Determining a gesture type of the user, wherein the gesture type is determined based on the gesture recognition method according to any one of claims 1 to 8; Perform corresponding gesture control processing based on the gesture type.

10. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores a computer program; The processor executes the computer program stored in the memory so that the electronic device performs the gesture recognition method according to any one of claims 1 to 8 and / or the gesture control method according to claim 9.

Images