Gesture recognition method and device, equipment and storage medium
By obtaining the key points of the hand, wrist and shoulder, and combining gestures and body shapes to determine gesture commands, the problem of low accuracy of gesture recognition in the prior art is solved, and the accuracy and stability of recognition is improved. It is suitable for multi-person scenarios and angle changes.
Patent Information
- Application Number
- CN202411001236.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-24
- Publication Date
- 2024-11-12
AI Technical Summary
In the prior art, in the process of hand gesture recognition in images, the accuracy is poor based on the positional relationship of the key points of the hand, especially when there is error in image acquisition, the accuracy of gesture recognition is low.
By obtaining the hand, wrist and shoulder key points characteristics in the target image, determine the gesture shape and limb shape, and combining the hand and limb shape to determine gesture commands, avoiding inaccurate gesture recognition problems caused by the loss of hand key points.
It improves the accuracy of gesture recognition, enhances the recognition stability under multi-person scenes and angle changes, reduces misidentification, and improves the fairness and efficiency of scenes such as competitions.
Smart Images

Figure CN118942116A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image recognition technology, and in particular to a gesture recognition method, device, equipment and storage medium. Background Art
[0002] In recent years, with the development of science and technology, the application of image recognition technology has become more and more extensive. For example, it can recognize the gesture instructions made by human hands in images.
[0003] Currently, in the process of identifying gesture instructions made by a person's hand in an image, the shape of the gesture made by the hand is determined based on the positional relationship between the key points of the hand, and then the corresponding gesture instruction is determined based on the gesture shape.
[0004] However, in the case where there are errors in image acquisition, the recognition of the key points of the hand in the image may not be accurate enough, so that the gesture recognition accuracy of determining the gesture instructions based only on the key points of the hand is poor. Summary of the invention
[0005] The present application provides a gesture recognition method, apparatus, device and storage medium, which are used to solve the problem of low accuracy of gesture recognition.
[0006] In order to achieve the above objectives, this application adopts the following technical solutions:
[0007] In a first aspect, the present application provides a gesture recognition method, the method comprising: obtaining a human body key point feature of a first object in a target image, the human body key point feature comprising: a hand key point feature, a wrist key point feature, and a shoulder key point feature. Determine a target gesture shape of the first object based on the hand key point feature. Determine a target limb shape of the first object based on the wrist key point feature and the shoulder key point feature. Determine a target gesture instruction of the first object based on the target gesture shape and the target limb shape.
[0008] The technical solution provided by the present application brings at least the following beneficial effects: by combining the key points of the hand, the key points of the wrist and the key points of the shoulder, the hand shape and the limb shape are determined, and then the corresponding gesture instructions are determined by combining the hand shape and the limb shape. In this way, by referring to the key points other than the hand, the problem of inaccurate gesture recognition caused by the loss of some key points in the hand can be avoided. In this way, the accuracy of gesture recognition can be improved.
[0009] Optionally, the target image includes multiple second objects. The above-mentioned method of "obtaining the key point features of the human body of the first object in the target image" includes: obtaining the clothing color features of each second object in the target image. Determine the clothing color similarity between the multiple second objects based on the clothing color features of the multiple second objects. Determine the first object from the multiple second objects based on the clothing color similarity between the multiple second objects, the clothing color similarity between the first object and the other objects is less than a preset similarity threshold, and the other objects are any object other than the first object among the multiple second objects.
[0010] Optionally, the clothing color feature is a color feature corresponding to clothing on the upper body of the object in the image.
[0011] Optionally, the human body key point features include all key point features of the first object. The method further includes: determining the human body center point features of the first object based on other key point features except the hand key point features and the wrist key point features among all the key point features. Determine the third object pointed to by the target gesture instruction among the multiple second objects based on the angle relationship between the wrist key point features and the human body center point features.
[0012] Optionally, the target gesture instruction is used to manage status information of the third object, and the status information includes at least one of the following: position information, moving speed, reward score, and penalty score.
[0013] Optionally, the above-mentioned method of "determining the target gesture instruction of the first object based on the target gesture shape and the target limb shape" includes: determining the target gesture instruction corresponding to the target gesture shape and the target limb shape according to a preset gesture recognition library, the preset gesture recognition library includes multiple preset gesture shapes, multiple preset limb shapes and multiple preset gesture instructions, and one preset gesture instruction corresponds to one preset gesture shape and one preset limb shape.
[0014] In a second aspect, the present application provides a gesture recognition device, which includes: an acquisition module and a processing module.
[0015] The acquisition module is used to acquire the key point features of the human body of the first object in the target image, and the key point features of the human body include: hand key point features, wrist key point features and shoulder key point features. The processing module is used to determine the target gesture shape of the first object according to the hand key point features. The processing module is also used to determine the target limb shape of the first object according to the wrist key point features and the shoulder key point features. The processing module is also used to determine the target gesture instruction of the first object according to the target gesture shape and the target limb shape.
[0016] Optionally, the target image includes multiple second objects. The acquisition module is specifically used to acquire clothing color features of each second object in the target image. The processing module is also used to determine the clothing color similarity between the multiple second objects based on the clothing color features of the multiple second objects. The processing module is also used to determine a first object from the multiple second objects based on the clothing color similarity between the multiple second objects, the clothing color similarity between the first object and the other objects is less than a preset similarity threshold, and the other objects are any object other than the first object in the multiple second objects.
[0017] Optionally, the clothing color feature is a color feature corresponding to clothing on the upper body of the object in the image.
[0018] Optionally, the human body key point features include all key point features of the first object. The processing module is further used to determine the human body center point features of the first object based on other key point features except the hand key point features and the wrist key point features among all the key point features. The processing module is also used to determine the third object pointed to by the target gesture instruction among the multiple second objects based on the angle relationship between the wrist key point features and the human body center point features.
[0019] Optionally, the target gesture instruction is used to manage status information of the third object, and the status information includes at least one of the following: position information, moving speed, reward score, and penalty score.
[0020] Optionally, the processing module is specifically used to determine the target gesture instruction corresponding to the target gesture shape and the target limb shape according to a preset gesture recognition library, the preset gesture recognition library includes multiple preset gesture shapes, multiple preset limb shapes and multiple preset gesture instructions, and one preset gesture instruction corresponds to one preset gesture shape and one preset limb shape.
[0021] In a third aspect, the present application provides a gesture recognition device, comprising: a processor and a memory, the processor and the memory are coupled, the memory is used to store one or more programs, the one or more programs include computer-executable instructions, when the gesture recognition device is running, the processor executes the computer-executable instructions stored in the memory to implement the gesture recognition method described in the first aspect or any optional one in the first aspect.
[0022] In a fourth aspect, the present application provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer executes the gesture recognition method described in the first aspect or any optional one in the first aspect.
[0023] In a fifth aspect, the present application provides a computer program product applied to a server, the computer program product comprising computer instructions. When the computer instructions are executed on the server, the server implements the gesture recognition method described in the first aspect or any optional aspect in the first aspect.
[0024] In the above solution, the technical problems that can be solved and the technical effects achieved by the gesture recognition device, equipment, computer storage medium or computer program product can refer to the technical problems and technical effects solved by the above first aspect, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 A schematic diagram of a communication system provided in an embodiment of the present application;
[0026] Figure 2 A flowchart of a gesture recognition method provided in an embodiment of the present application;
[0027] Figure 3 An example schematic diagram of key points corresponding to different areas of a hand provided in an embodiment of the present application;
[0028] Figure 4 A flowchart of another gesture recognition method provided in an embodiment of the present application;
[0029] Figure 5 A flowchart of another gesture recognition method provided in an embodiment of the present application;
[0030] Figure 6 A schematic diagram of the structure of a gesture recognition device provided in an embodiment of the present application;
[0031] Figure 7 A schematic diagram of the structure of a gesture recognition device provided in an embodiment of the present application;
[0032] Figure 8 A conceptual partial view of a computer program product provided for an embodiment of the present application. DETAILED DESCRIPTION
[0033] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0034] In this article, the character " / " generally indicates that the objects before and after are in an "or" relationship. For example, A / B can be understood as A or B.
[0035] The terms “first” and “second” in the description and claims of the present application are used to distinguish different objects rather than to describe a specific order of the objects.
[0036] In addition, the terms "including" and "having" and any variations thereof mentioned in the description of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not limited to the listed steps or modules, but may optionally include other steps or modules that are not listed, or may optionally include other steps or modules that are inherent to these processes, methods, products or devices.
[0037] In addition, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present concepts in a specific way.
[0038] Before introducing the gesture recognition method provided in the embodiment of the present application in detail, the implementation environment and application scenarios of the embodiment of the present application are first introduced.
[0039] First, the application scenarios of the embodiments of the present application are introduced.
[0040] In the current competition scenario, there is a significant time difference between the time when the contestant performs the corresponding action and the referee makes a gesture. In addition, for the recorder, the task is heavy and the possibility of operating errors is high. Therefore, in order to ensure the fairness and accuracy of the competition, it is necessary to take corresponding measures to shorten the time difference and reduce the workload of the recorder. The referee's gesture can be recognized by using key point detection algorithms (such as high resolution network (HRNet) algorithm and convolutional pose machine (CPM) algorithm), and key points at different positions are defined as different gestures. Then the recognition data is compared with the gesture library, and the results are obtained and output for easy scoring.
[0041] However, due to the large number of people in the game scene, the images captured by the camera contain a lot of people, and gesture recognition may mistakenly identify spectators or athletes, affecting the scoring. In addition, gesture recognition based on key point detection is prone to losing some key points due to the influence of angle and posture, resulting in misidentification of key points, which makes the gesture recognition results vary greatly.
[0042] Therefore, how to improve the accuracy of gesture recognition has become a technical problem that needs to be solved urgently.
[0043] In order to solve the above problems, an embodiment of the present application provides a gesture recognition method, and the gesture recognition method provided by the embodiment of the present application is applied to the scene of gesture recognition. The embodiment of the present application determines the hand shape and limb shape by combining the key points of the hand, the key points of the wrist and the key points of the shoulder, and then determines the corresponding gesture instructions by combining the hand shape and the limb shape. In this way, by referring to the key points of non-hands, the problem of inaccurate gesture recognition caused by the loss of some key points in the hand can be avoided. In this way, the accuracy of gesture recognition can be improved.
[0044] The implementation environment of the embodiments of the present application is introduced below.
[0045] like Figure 1 , which is a schematic diagram of a communication system provided in an embodiment of the present application, the communication system may include: a gesture recognition device 101 and a collection device 102. The gesture recognition device 101 may perform wired / wireless communication with the collection device 102.
[0046] Specifically, the gesture recognition device 101 can obtain the video image captured by the acquisition device 102 by accessing the acquisition device 102, and recognize the gesture action made by the user based on the captured video image, thereby obtaining the corresponding gesture instruction.
[0047] It should be noted that the embodiment of the present application does not limit the acquisition device 102. For example, the acquisition device 102 may be a camera. For another example, the acquisition device 102 may be a camera. For another example, the acquisition device 102 may be a laser radar.
[0048] In the embodiment of the present application, the gesture recognition device 101 may be a terminal, or the gesture recognition device 101 may be a server.
[0049] The server may be a single physical server, or a server cluster consisting of multiple servers. Alternatively, the server cluster may be a distributed cluster. Alternatively, the server may be a cloud server. The embodiments of the present application do not limit the specific implementation of the server.
[0050] The terminal may be a mobile phone, tablet computer, desktop, laptop, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, etc. with transceiver functions. The present application does not impose any special restrictions on the specific form of the terminal. It may interact with the user through one or more methods such as keyboard, touch pad, touch screen, remote control, voice interaction or handwriting device.
[0051] After introducing the application scenario and implementation environment of the embodiment of the present application, the gesture recognition method provided by the embodiment of the present application is described in detail below in combination with the above implementation environment.
[0052] The methods in the following embodiments can all be implemented in the above application scenarios and implementation environments. The embodiments of the present application are described in detail below in conjunction with the accompanying drawings.
[0053] Figure 2 A flowchart of a gesture recognition method provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the method may include: S201-S204.
[0054] S201: A gesture recognition device obtains key point features of a human body of a first object in a target image.
[0055] The human body key point features are used to indicate the position information of different key points in the human body in the image, and the human body key point features may include all key point features of the first object.
[0056] In the embodiment of the present application, the key point features of the human body may include: hand key point features, wrist key point features and shoulder key point features.
[0057] Among them, the key point features of the hand may include: key point features corresponding to the base of the palm, key point features corresponding to multiple finger joints, key point features corresponding to multiple fingertips, key point features corresponding to multiple metacarpal bones, and key point features corresponding to the thenar eminence.
[0058] For example, Figure 3 As shown, it shows the key points corresponding to different areas of the hand, including: the base of the palm key point A, thenar key point B, and 5 metacarpal key points (metacarpal key point C, metacarpal key point D, metacarpal key point E, metacarpal key point F and metacarpal key point G) corresponding to the palm, the knuckle key point H and fingertip key point I corresponding to the thumb, the knuckle key point J, knuckle key point K and fingertip key point L corresponding to the index finger, the knuckle key point M, knuckle key point N and fingertip key point O corresponding to the middle finger, the knuckle key point P, knuckle key point Q and fingertip key point R corresponding to the ring finger, and the knuckle key point S, knuckle key point T and fingertip key point U corresponding to the little finger.
[0059] It should be noted that for the introduction of wrist key point features and shoulder key point features, you can refer to the division of key points corresponding to different areas of the hand in the hand key point features. The key points of different areas of the wrist and different areas of the shoulder are divided, which will not be repeated here.
[0060] As a possible implementation method, the gesture recognition device can recognize the target image, determine the image area in the target image that displays the first object, and perform human key point recognition on the image area corresponding to the first object in the target image to obtain the human key point features of the first object in the target image.
[0061] It should be noted that, for the above-mentioned process of identifying the image area corresponding to the first object in the target image and identifying the key points of the human body in the image, reference may be made to the introduction of image recognition in the prior art, which will not be elaborated here.
[0062] Optionally, the embodiment of the present application does not limit the method for acquiring the target image. For example, the gesture recognition device may receive the target image input by the user. For another example, the gesture recognition device may access a capture device (such as a camera) to acquire the target image captured by the capture device.
[0063] S202: The gesture recognition device determines a target gesture shape of the first object according to key point features of the hand.
[0064] As a possible implementation method, the gesture recognition device can determine the positional relationship between each hand key point based on the hand key point characteristics to determine the distribution of all hand key points, and then determine the target gesture shape composed of all hand key points.
[0065] S203: The gesture recognition device determines the target limb shape of the first object according to the wrist key point features and the shoulder key point features.
[0066] As a possible implementation method, the gesture recognition device can determine the positional relationship between the wrist key point and the shoulder key point based on the wrist key point features and the shoulder key point features to determine the distribution of the wrist key point and the shoulder key point, and then determine the target limb shape corresponding to the posture shape formed by the wrist and shoulder.
[0067] S204: The gesture recognition device determines a target gesture instruction of the first object according to the target gesture shape and the target limb shape.
[0068] As a possible implementation, the gesture recognition device may store a preset gesture recognition library, which includes: multiple preset gesture shapes, multiple preset limb shapes, and multiple preset gesture instructions, and one preset gesture instruction corresponds to one preset gesture shape and one preset limb shape. The gesture recognition device may determine a gesture shape that is identical to the target gesture shape from multiple preset gesture shapes, and determine a limb shape that is identical to the target limb shape from multiple preset limb shapes, based on the target gesture shape and the target limb shape. Then, the gesture recognition device may use the preset gesture instruction corresponding to the gesture shape identical to the target gesture shape from multiple preset gesture shapes and the limb shape identical to the target limb shape from multiple preset limb shapes as the target gesture instruction, to obtain the target gesture instruction.
[0069] The technical solution provided by the above embodiment brings at least the following beneficial effects: by combining the key points of the hand, the key points of the wrist and the key points of the shoulder, the hand shape and the limb shape are determined, and then the corresponding gesture instructions are determined by combining the hand shape and the limb shape. In this way, by referring to the key points other than the hand, the problem of inaccurate gesture recognition caused by the loss of some key points in the hand can be avoided. In this way, the accuracy of gesture recognition can be improved.
[0070] In some embodiments, the target image may include multiple second objects. In the process of the gesture recognition device acquiring the key point features of the human body of the first object in the target image (i.e., S201), the gesture recognition device may acquire the clothing color features of each second object in the target image, and determine the clothing color similarity between the multiple second objects based on the clothing color features of the multiple second objects. Then, the gesture recognition device may determine the first object from the multiple second objects based on the clothing color similarity between the multiple second objects, the clothing color similarity between the first object and the other objects is less than a preset similarity threshold, and the other objects are any object other than the first object in the multiple second objects.
[0071] Exemplarily, the plurality of second objects in the target image include: object A, object B, object C, and object D. Among them, the clothing color similarity between object A and object B is 0.1, the clothing color similarity between object A and object C is 0.2, the clothing color similarity between object A and object D is 0.3, the clothing color similarity between object B and object C is 0.4, the clothing color similarity between object B and object D is 0.5, and the clothing color similarity between object C and object D is 0.6. If the preset similarity threshold is 0.4, the first object is object A.
[0072] As a possible implementation manner, the clothing color feature may be a color feature corresponding to the clothing on the upper body of the object in the image.
[0073] It can be understood that by dividing the video acquisition area, determining the images of the upper body and the whole body of the human body in the acquisition area, and automatically or manually dividing the color of the upper body image, the referee's position can be identified, and then gesture recognition can be performed. In other words, when there are multiple objects in the image, since each object has its own gesture, how to filter out the gesture to be recognized from the image is to filter the objects in the image based on the color characteristics of the clothing, and then obtain the gesture of the object. In this way, the efficiency of gesture recognition can be improved.
[0074] In some embodiments, Figure 4 As shown, after the gesture recognition device determines the target gesture instruction of the first object according to the target gesture shape and the target limb shape (ie, S204), the method may further include: S401-S402.
[0075] S401. The gesture recognition device determines the body center point feature of the first object according to the key point features other than the hand key point feature and the wrist key point feature among all the key point features.
[0076] S402: The gesture recognition device determines a third object pointed to by the target gesture instruction among the plurality of second objects according to the angular relationship between the wrist key point feature and the body center point feature.
[0077] As a possible implementation method, the gesture recognition device can determine the arm pointing direction of the first object based on the angular relationship between the wrist key point feature and the body center point feature, and determine the third object indicated by the first object from multiple second objects based on the arm pointing direction, and then determine the third object pointed to by the target gesture instruction.
[0078] It can be understood that, by determining the pointing direction of the hand gesture, the object pointed to by the corresponding gesture instruction can be determined, so that the pointed object can be managed based on the gesture instruction later.
[0079] It should be noted that, in a game scenario, the first object may be a referee, and the third object may be an athlete judged by the referee. The gesture recognition device may manage the athlete's score based on the referee's gesture instructions recognized by the image.
[0080] In some embodiments, the target gesture command is used to manage the state information of the third object, and the state information includes at least one of the following: position information, movement speed, bonus score, and penalty score. The bonus score may indicate the number of goals scored by the athlete, and the penalty score may indicate the number of fouls committed by the athlete.
[0081] As a possible implementation method, after the gesture recognition device determines the third object pointed to by the target gesture instruction among multiple second objects (i.e., S402) based on the angular relationship between the wrist key point features and the body center point features, the gesture recognition device can manage the state information of the third object according to the target gesture instruction.
[0082] The following takes the target object as an image as an example to introduce the process of determining target abnormal information, including:
[0083] The following uses a game scenario as an example, combined with a specific example, to introduce the gesture recognition method provided by the embodiment of the present application. Figure 5 As shown, including:
[0084] Step 1: The gesture recognition device uses a camera to obtain a video stream.
[0085] Step 2: The gesture recognition device uses the video segmentation function of OpenCV (open source computer vision library, an open source computer vision and machine learning software library) to manually limit the area that needs to be recognized (such as the stadium area).
[0086] Step 3: The gesture recognition device uses algorithms such as YOLOv8 (the latest iteration of the YOLO (you only look once) series of real-time object detection algorithms) to recognize the upper body image of the human body.
[0087] Step 4: The gesture recognition device uses OpenCV color to filter out the only shirt color and obtain the referee image.
[0088] Alternatively, the gesture recognition device can filter out the corresponding referee image based on the manually defined referee's clothing color.
[0089] Step 5: The gesture recognition device performs gesture recognition on the referee image, compares it with the defined gesture library, and recognizes the action.
[0090] Step 6: The gesture recognition device performs gesture recognition near the key points of the hand for referee gesture recognition and compares it with the gesture library.
[0091] Step 7: The gesture recognition device uses the wrist key points and other hand key points to determine the finger angles.
[0092] Step 8: The gesture recognition device transmits the recognition result to the scoring system in real time.
[0093] That is to say, the gesture recognition device uses a high-definition camera to shoot the competition venue, divides the captured video into regions, limits the recognition area to the competition venue, and then uses target recognition algorithms such as YOLOv8 to recognize the human body in the region, and recognizes the upper body image of the human body at the same time. The gesture recognition device then uses the color selection function of opencv to filter the color of the upper body image, defines the color with the largest proportion in the image as the color of the clothes, and selects the only image with a different color from other images as the referee's half-body image, and then performs gesture recognition based on the full-body image corresponding to the referee's half-body image.
[0094] The above method can also be implemented by manually defining the referee's clothes color and then finding the referee's half-body image by comparing the clothes color.
[0095] The method for gesture recognition device to identify referee actions uses key point detection algorithm for posture recognition and gesture recognition. First, the gesture recognition device uses key point detection algorithms such as HRNet or CPM to perform key point detection on the referee's full-body image to obtain the position information of the key points. Then, the gesture recognition device defines key points at different positions as different gestures. For example, the wrist key point is higher than the shoulder key point (arm raised) as "score", and the wrist key point is parallel to the shoulder key point and has a certain distance (arm parallel and straight) as "mistake", etc. Next, the gesture recognition device uses key point detection to identify the hand key points near the wrist key point, compares the identified key point position information with the gesture library, obtains the gesture recognition result, and outputs the result.
[0096] In order to deal with the problem of losing the key points of the hand due to changes in angles and postures or excessive distance during the game, the gesture recognition device can define the wrist key point as point 0, and considering the posture and recognition accuracy of the gesture, the program only judges the case where the number of detected key points is greater than 5. Since point 0 and the first 5 key points of the hand belong to the palm, they will affect the finger pointing angle information. The gesture recognition device can calculate the center point of all key points of the hand except point 0 and the first 5 points of the wrist, and calculate the azimuth of the center point and point 0. Then, the gesture recognition device can determine the direction of the referee's pointing based on the information of point 0 and the azimuth, and the player identified in this direction is the offending player.
[0097] Finally, the gesture recognition device transmits the gesture recognition results to the scoring system in real time to achieve automatic scoring of the game. This method can greatly shorten the time difference from the referee making a gesture to the scoring system recording the results, improving the fairness and accuracy of the game. At the same time, it can also reduce the workload of the recorder and improve the efficiency of the game.
[0098] For example, in a football match, the referee is wearing blue clothes. First, the high-definition camera is pointed at the football field, and then the field area is manually marked. During the game, the human bodies on the field are identified, and the identified human bodies are screened to select the referee wearing blue clothes, and then the referee's posture is recognized. If the referee's arm is recognized to be raised diagonally upward with five fingers together, the detection gesture angle is 45 degrees to the upper right, then it is recorded as a foul ball, and the ball belongs to the right team.
[0099] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of a computer device. It is understandable that in order to implement the above functions, the computer device includes a hardware structure and / or software module corresponding to each function. Those skilled in the art should easily realize that, in combination with the gesture recognition method steps of each example described in the embodiment disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0100] The embodiment of the present application also provides a gesture recognition device, which can be a computer device, a CPU in the computer device, a processing module for gesture recognition in the computer device, or a client for gesture recognition in the computer device.
[0101] The embodiment of the present application can divide the gesture recognition device into functional modules or functional units according to the above method example. For example, each functional module or functional unit can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules or functional units. Among them, the division of modules or units in the embodiment of the present application is schematic, which is only a logical function division. There may be other division methods in actual implementation.
[0102] like Figure 6 FIG. 1 is a schematic diagram of a gesture recognition device provided in an embodiment of the present application. The gesture recognition device is used to perform Figure 2 or Figure 4 In the gesture recognition method shown, the gesture recognition device 600 may include: an acquisition module 601 and a processing module 602 .
[0103] The acquisition module 601 is used to acquire the human body key point features of the first object in the target image, and the human body key point features include: hand key point features, wrist key point features and shoulder key point features. The processing module 602 is used to determine the target gesture shape of the first object according to the hand key point features. The processing module 602 is also used to determine the target limb shape of the first object according to the wrist key point features and the shoulder key point features. The processing module 602 is also used to determine the target gesture instruction of the first object according to the target gesture shape and the target limb shape.
[0104] Optionally, the target image includes multiple second objects. The acquisition module 601 is specifically used to acquire the clothing color features of each second object in the target image. The processing module 602 is also used to determine the clothing color similarity between the multiple second objects based on the clothing color features of the multiple second objects. The processing module 602 is also used to determine a first object from the multiple second objects based on the clothing color similarity between the multiple second objects, the clothing color similarity between the first object and the other objects is less than a preset similarity threshold, and the other objects are any object other than the first object in the multiple second objects.
[0105] Optionally, the clothing color feature is a color feature corresponding to clothing on the upper body of the object in the image.
[0106] Optionally, the human body key point features include all key point features of the first object. The processing module 602 is further used to determine the human body center point features of the first object based on other key point features except the hand key point features and the wrist key point features in all the key point features. The processing module 602 is also used to determine the third object pointed to by the target gesture instruction among the multiple second objects based on the angle relationship between the wrist key point features and the human body center point features.
[0107] Optionally, the target gesture instruction is used to manage status information of the third object, and the status information includes at least one of the following: position information, moving speed, reward score, and penalty score.
[0108] Optionally, the processing module 602 is specifically used to determine the target gesture instruction corresponding to the target gesture shape and the target limb shape according to a preset gesture recognition library, the preset gesture recognition library includes multiple preset gesture shapes, multiple preset limb shapes and multiple preset gesture instructions, and one preset gesture instruction corresponds to one preset gesture shape and one preset limb shape.
[0109] Figure 7 7 is a schematic diagram showing a structure of a gesture recognition device according to an exemplary embodiment. The gesture recognition device may include a processor 702, and the processor 702 is used to execute application code to implement the gesture recognition method in the present application.
[0110] The processor 702 may be a CPU, a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.
[0111] like Figure 7 As shown, the gesture recognition device may further include a memory 703. The memory 703 is used to store application program codes for executing the solution of the present application, and the execution is controlled by the processor 702.
[0112] The memory 703 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 703 may exist independently and be connected to the processor 702 via the bus 704. The memory 703 may also be integrated with the processor 702.
[0113] like Figure 7 As shown, the gesture recognition device may further include a communication interface 701, wherein the communication interface 701, the processor 702, and the memory 703 may be coupled to each other, for example, via a bus 704. The communication interface 701 is used to exchange information with other devices, for example, to support information exchange between the gesture recognition device and other devices.
[0114] It should be pointed out that Figure 7 The device structure shown in the figure does not constitute a limitation on the gesture recognition device. Figure 7 In addition to the components shown, the gesture recognition device may include more or fewer components than shown, or combine certain components, or arrange the components differently.
[0115] In actual implementation, the functions implemented by the processing module 602 can be Figure 7 The processor 702 shown calls the program code in the memory 703 to implement.
[0116] The present application also provides a computer-readable storage medium, on which instructions are stored. When the instructions in the computer-readable storage medium are executed by a processor of a computer device, the computer can perform the gesture recognition provided by the above-mentioned embodiment. For example, the computer-readable storage medium may be a memory 703 including instructions, and the above instructions may be executed by a processor 702 of a computer device to complete the above method. Optionally, the computer-readable storage medium may be a non-temporary computer-readable storage medium, for example, a non-temporary computer-readable storage medium may be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.
[0117] Figure 8 A conceptual partial view of a computer program product provided by an embodiment of the present application is exemplarily shown, where the computer program product includes a computer program for executing a computer process on a computing device.
[0118] In one embodiment, the computer program product is provided using a signal bearing medium 800. The signal bearing medium 800 may include one or more program instructions that, when executed by one or more processors, may provide the above-described Figure 2 or Figure 4 Thus, for example, reference to Figure 2 In the embodiment shown in , one or more features of S201 to S204 may be undertaken by one or more instructions associated with the signal bearing medium 800. In addition, Figure 8 The program instructions in also describe example instructions.
[0119] In some examples, the signal bearing medium 800 may include a computer readable medium 801, such as, but not limited to, a hard drive, a compact disk (CD), a digital video disk (DVD), a digital tape, a memory, a read-only memory (ROM), or a random access memory (RAM), and the like.
[0120] In some implementations, the signal bearing medium 800 may include a computer recordable medium 802 such as, but not limited to, a memory, a read / write (R / W) CD, R / W, DVD, or the like.
[0121] In some implementations, signal bearing medium 800 may include communication medium 803 such as, but not limited to, digital and / or analog communication media (eg, fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).
[0122] The signal bearing medium 800 may be communicated by a wireless form of communication medium 803. The one or more program instructions may be, for example, computer executable instructions or logic implemented instructions.
[0123] In some examples, such as for Figure 6 The described gesture recognition apparatus may be configured to provide various operations, functions, or actions in response to one or more program instructions via computer-readable medium 801 , computer-recordable medium 802 , and / or communication medium 803 .
[0124] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.
[0125] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0126] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0127] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0128] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or the full classification part or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute the full classification part or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes various media that can store program codes, such as-U disk, mobile hard disk, ROM, RAM, disk or CD.
[0129] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto, and any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A gesture recognition method, characterized in that: The method comprises: Acquire key point features of a human body of a first object in a target image, wherein the key point features of the human body include: key point features of a hand, key point features of a wrist, and key point features of a shoulder; Determining a target gesture shape of the first object according to the hand key point features; Determining a target limb shape of the first object according to the wrist key point features and the shoulder key point features; A target gesture instruction of the first object is determined according to the target gesture shape and the target limb shape.
2. The method according to claim 1, characterized in that The target image includes a plurality of second objects; and the step of obtaining the key point features of the human body of the first object in the target image includes: Acquire clothing color features of each second object in the target image; Determining clothing color similarities between the plurality of second objects according to clothing color features of the plurality of second objects; The first object is determined from the multiple second objects according to the clothing color similarity between the multiple second objects, the clothing color similarity between the first object and other objects is less than a preset similarity threshold, and the other objects are any object among the multiple second objects except the first object.
3. The method according to claim 2, characterized in that The clothing color feature is a color feature corresponding to the clothing of the upper body of the object in the image.
4. The method according to claim 2, characterized in that: The human body key point features include all key point features of the first object; After determining the target gesture instruction of the first object according to the target gesture shape and the target limb shape, the method further includes: Determine a body center point feature of the first object according to other key point features except the hand key point feature and the wrist key point feature among all the key point features; A third object pointed to by the target gesture instruction among the plurality of second objects is determined according to an angular relationship between the wrist key point feature and the body center point feature.
5. The method according to claim 4, characterized in that The target gesture instruction is used to manage the state information of the third object, and the state information includes at least one of the following: position information, moving speed, reward score, and penalty score.
6. The method according to any one of claims 1 to 5, characterized in that The step of determining a target gesture instruction of the first object according to the target gesture shape and the target limb shape includes: According to a preset gesture recognition library, the target gesture instruction corresponding to the target gesture shape and the target limb shape is determined, wherein the preset gesture recognition library includes multiple preset gesture shapes, multiple preset limb shapes and multiple preset gesture instructions, and one preset gesture instruction corresponds to one preset gesture shape and one preset limb shape.
7. A gesture recognition device, characterized in that: The device comprises: An acquisition module, used for acquiring key point features of a human body of a first object in a target image, wherein the key point features of the human body include: key point features of a hand, key point features of a wrist, and key point features of a shoulder; A processing module, configured to determine a target gesture shape of the first object according to the key point features of the hand; The processing module is further used to determine the target limb shape of the first object according to the wrist key point feature and the shoulder key point feature; The processing module is further used to determine a target gesture instruction of the first object according to the target gesture shape and the target limb shape.
8. A gesture recognition device, characterized in that: include: Processor and memory; The processor is coupled to the memory; The memory is used to store one or more programs, and the one or more programs include computer-executable instructions. When the gesture recognition device is running, the processor executes the computer-executable instructions stored in the memory to enable the gesture recognition device to perform the gesture recognition method according to any one of claims 1 to 6.
9. A computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, characterized in that: When a computer executes the instructions, the computer performs the gesture recognition method according to any one of claims 1 to 6.
10. A computer program product, characterized in that The computer program product comprises computer program instructions, and when the computer program instructions are executed, the gesture recognition method according to any one of claims 1 to 6 is implemented.
