Hand recognition method and device, equipment and storage medium

Abstract

The invention relates to the technical field of hand recognition, and provides a hand recognition method and device, equipment and a storage medium, and the method comprises the steps: obtaining a hand image; according to the hand image, determining a first preset position of a finger tip point and a second preset position of a finger joint point adjacent to the finger tip point; according to the first preset position and the second preset position, determining a finger mask area corresponding to a finger tip point and a finger joint point from the hand image; determining the distance between a third preset position of the contour point in the finger mask area and a preset external connection area corresponding to the finger mask area; according to the distances corresponding to the contour points, the target position of the fingertip point is determined from the third preset positions of the contour points, so that the position recognition accuracy of the fingertip point is improved.

Description

Technical Field

[0001] The present application relates to the field of hand recognition technology, and in particular to a hand recognition method, apparatus, device and storage medium. Background Art

[0002] Currently, when recognizing and tracking hand movements of a user's hand, it is often necessary to mark the key points of the user's hand.

[0003] In the related art, a deep neural network model can be used to identify key points of the hand in a hand image. When training a deep neural network model, a large number of hand images with hand key points annotated are required as training samples. For example, the hand key points in the hand image can be annotated by auxiliary annotation to obtain training samples. However, when annotating the hand key points in the hand image by auxiliary annotation, it is easy for the identified fingertip points to deviate from the actual fingertip points, resulting in poor accuracy in the position recognition of the fingertip points. Summary of the Invention

[0004] The main purpose of this application is to provide a hand recognition method, device, equipment and storage medium, aiming to solve the technical problem of poor accuracy in position recognition of fingertip points due to the deviation of the identified fingertip points from the actual fingertips.

[0005] In a first aspect, the present application provides a hand recognition method, comprising:

[0006] Acquire hand images;

[0007] Determining, based on the hand image, a first preset position of a fingertip point and a second preset position of a finger joint point adjacent to the fingertip point;

[0008] Determining, from the hand image, a finger mask area corresponding to the fingertip point and the finger joint point according to the first preset position and the second preset position;

[0009] Determining a distance between a third preset position of a contour point in the finger mask area and a preset circumscribed area corresponding to the finger mask area;

[0010] According to the distances corresponding to the plurality of contour points, the target position of the fingertip point is determined from the third preset positions of the plurality of contour points.

[0011] In a second aspect, the present application provides a hand recognition device, comprising:

[0012] An acquisition module, used for acquiring a hand image;

[0013] a first determining module, configured to determine, based on the hand image, a first preset position of a fingertip point and a second preset position of a finger joint point adjacent to the fingertip point;

[0014] a second determining module, configured to determine, from the hand image, a finger mask area corresponding to the fingertip point and the finger joint point according to the first preset position and the second preset position;

[0015] a third determining module, configured to determine a distance between a third preset position of a contour point in the finger mask area and a preset circumscribed area corresponding to the finger mask area;

[0016] The fourth determining module is configured to determine the target position of the fingertip point from respective third preset positions of the plurality of contour points according to respective corresponding distances of the plurality of contour points.

[0017] In a third aspect, the present application provides an electronic device, the electronic device comprising a memory and a processor;

[0018] The memory is used to store computer programs;

[0019] The processor is configured to execute the computer program and implement the steps of the above-mentioned hand recognition method when executing the computer program.

[0020] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the hand recognition method as described above are implemented.

[0021] The present application provides a hand recognition method, apparatus, device and storage medium, the hand recognition method comprising: acquiring a hand image; determining, based on the hand image, a first preset position of a fingertip point and a second preset position of a finger joint point adjacent to the fingertip point; determining, from the hand image, a finger mask area corresponding to the fingertip point and the finger joint point based on the first preset position and the second preset position; determining the distance between a third preset position of a contour point in the finger mask area and a preset circumscribed area corresponding to the finger mask area; and determining, based on the corresponding distances of each of the multiple contour points, a target position of the fingertip point from each of the third preset positions of the multiple contour points.

[0022] When determining a first preset position of a fingertip and a second preset position of a finger joint based on a hand image, the electronic device can determine the finger mask area corresponding to the fingertip and the finger joint. Accordingly, the electronic device can calibrate the first preset position of the fingertip by combining the contour points of the finger mask area with the preset circumscribed area corresponding to the finger mask area to obtain a target position of the fingertip, thereby improving the accuracy of fingertip position recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0024] Figure 1 This is a flow chart of a hand recognition method provided in an embodiment of the present application;

[0025] Figure 2 is a schematic diagram of a hand image involved in an embodiment of the present application;

[0026] Figure 3 is a schematic diagram of a hand mask area involved in an embodiment of the present application;

[0027] Figure 4 is a schematic diagram of a finger mask area involved in an embodiment of the present application;

[0028] Figure 5 is a schematic diagram of a preset circumscribed area corresponding to a finger mask area according to an embodiment of the present application;

[0029] Figure 6 is a schematic diagram of a target search area involved in an embodiment of the present application;

[0030] Figure 7 is a schematic block diagram of a hand recognition device provided in an embodiment of the present application;

[0031] Figure 8 This is a schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0033] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0034] The embodiments of the present application provide a hand recognition method, apparatus, device and storage medium. The hand recognition method can be applied to electronic devices. The electronic device can be a mobile phone, tablet computer, laptop computer, desktop computer, wearable device and the like. Wearable devices can include smart bracelets, smart watches, near-eye display devices and the like. Near-eye display devices can include virtual reality (VR) glasses, augmented reality (AR) glasses, mixed reality (MR) glasses, VR helmets, AR helmets, MR helmets, and the like, without limitation. The hand recognition method can also be applied to a server, which can be a separate server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0035] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments may be combined with each other.

[0036] See also Figure 1 , Figure 1 It should be noted that the hand recognition method provided in the embodiment of the present application can be used in electronic devices or servers, without limitation.

[0037] like Figure 1 As shown, the hand recognition method includes steps S101 to S105.

[0038] S101: Acquire a hand image.

[0039] For example, an electronic device can capture a hand image. For example, the electronic device can be provided with a camera module. The electronic device can capture a user's hand using the camera module to obtain a hand image. The user corresponding to the user's hand can include the user of the electronic device or a non-user of the electronic device, without limitation. Of course, this is not limiting. For example, the electronic device can obtain hand images of different users via a network, without limitation.

[0040] When a hand image is acquired, the hand image can be used to subsequently determine the first preset position of the fingertip point.

[0041] S102: Determine, based on the hand image, a first preset position of a fingertip point and a second preset position of a finger joint point adjacent to the fingertip point.

[0042] For example, the finger tip point may include at least one of the thumb tip point, index finger tip point, middle finger tip point, ring finger tip point, and little finger tip point. The finger joint point may include at least one of the thumb joint point adjacent to the thumb tip point, the index finger joint point adjacent to the index finger tip point, the middle finger joint point adjacent to the middle finger tip point, the ring finger joint point adjacent to the ring finger tip point, and the little finger joint point adjacent to the little finger tip point.

[0043] In some embodiments, the hand image is recognized and processed according to a preset image recognition algorithm to obtain a first preset position of a fingertip point and a second preset position of a finger joint point adjacent to the fingertip point.

[0044] For example, the image recognition algorithm may include a deep learning algorithm, which is not limited here.

[0045] like Figure 2 As shown, the electronic device recognizes the hand image and can obtain the first preset position of the thumb tip, the second preset position of the thumb joint, the first preset position of the index finger tip and the second preset position of the index finger joint. Figure 2 It can be seen that the first preset position of the thumb joint point deviates from the actual position of the thumb fingertip point in the hand image. The first preset position of the index fingertip point deviates from the actual position of the index fingertip point in the hand image.

[0046] When an electronic device determines a first preset position of a fingertip and a second preset position of a finger joint adjacent to the fingertip based on a hand image, if the first preset position deviates from the actual position of the fingertip, the electronic device will need to subsequently calibrate the first preset position of the fingertip to obtain the actual position of the fingertip, i.e., the target position. Based on this, the electronic device can subsequently determine the target position of the fingertip in combination with the first preset position and the second preset position to subsequently improve the accuracy of fingertip position recognition.

[0047] S103: Determine finger mask areas corresponding to fingertip points and finger joint points from the hand image according to the first preset position and the second preset position.

[0048] For example, the electronic device can determine the position range of the finger area corresponding to the finger tip points and the finger joint points in the hand image based on the first preset position and the second preset position. When the position range of the finger area is determined, the electronic device can perform image segmentation processing on the hand image based on the position range to obtain the corresponding finger mask area. For example, the electronic device can perform cropping processing on the hand image based on the position range to obtain a cropped hand image. The cropped hand image may include the finger area corresponding to the finger tip points and the finger joint points and the background area where the finger area is located. The electronic device can separate the finger area from the background area by performing image segmentation processing on the cropped hand image to obtain the corresponding finger mask area. Of course, the method of determining the finger mask area is not limited to this and is not limited here.

[0049] In some embodiments, image segmentation processing is performed on the hand image to obtain a hand mask area corresponding to the hand image; and a finger mask area is determined from the hand mask area based on the first preset position and the second preset position.

[0050] When the electronic device acquires a hand image, the hand image may include the hand area and the background area where the hand area is located. Figure 3 As shown, the electronic device can perform image segmentation processing on the hand image to separate the hand area from the background area in which it is located, and obtain the corresponding hand mask area. The electronic device can then determine the finger mask area in combination with the first preset position, the second preset position and the hand mask area. For example, since there is a correspondence between the hand image and the hand mask area, the electronic device can determine the finger mask area from the hand mask area based on the first preset position and the second preset position. For example, the electronic device can determine the finger mask area from the hand mask area based on the position range of the first preset position and the second preset position, and there is no limitation here.

[0051] In some embodiments, a corresponding preset rectangular area is determined from the hand mask area according to the first preset position and the second preset position; and the outline of the preset rectangular area is expanded to obtain the finger mask area.

[0052] For example, since the first preset position and the second preset position are determined based on the hand image, the hand image may include at least one of a two-dimensional image and a three-dimensional image. Taking the hand image as a two-dimensional image as an example, the first preset position and the second preset position may include 2D coordinates. The electronic device may determine the horizontal coordinate of the first preset position as the starting horizontal coordinate of the preset rectangular area, and the horizontal coordinate of the second preset position as the ending horizontal coordinate of the preset rectangular area. Correspondingly, the electronic device may determine the vertical coordinate of the first preset position as the starting vertical coordinate of the preset rectangular area, and the vertical coordinate of the second preset position as the ending vertical coordinate of the preset rectangular area. Based on this, the electronic device may divide the preset rectangular area from the hand mask area according to the starting horizontal coordinate, the ending horizontal coordinate, the starting vertical coordinate and the ending vertical coordinate.

[0053] Since the first preset position may deviate from the actual position of the fingertip, the preset rectangular area divided by the electronic device based on the first preset position and the second preset position does not cover the actual position of the fingertip. Based on this, the electronic device can expand the outline of the preset rectangular area so that the preset rectangular area after expansion can cover the actual position of the fingertip. For example, the electronic device can expand the outline of the preset rectangular area according to a preset expansion ratio. The preset expansion ratio may include 10%, 20%, 30%, etc., which are not limited here. Taking the preset expansion ratio of 20% as an example, the electronic device can expand the size of the outline of the preset rectangular area by 20%, and so on. The preset expansion ratio can be pre-set or user-set, which is not limited here. Accordingly, the electronic device can determine the preset rectangular area after expansion as the finger mask area. The finger mask area is, for example, Figure 4 shown.

[0054] In some embodiments, based on the hand mask area, region growing processing is performed on the first preset position and the second preset position to obtain the finger mask area.

[0055] The finger area has similar attributes among the multiple pixels in the position range corresponding to the hand mask area, such as color or gray value similarity. Figure 3As shown, the finger region corresponding to the thumb tip and the thumb joint has multiple pixels with similar colors and grayscale values ​​within the hand mask region. Based on this, the electronic device can perform region growing processing on the first and second preset locations. For example, during the region growing processing on the first and second preset locations, the electronic device can use the pixels corresponding to each of the first and second preset locations as starting points and, based on region growing criteria based on color or grayscale similarity and spatial proximity, iteratively merge adjacent pixels with similar colors or grayscale values ​​to the pixels corresponding to each of the first and second preset locations into the same region until no more pixels can be included. For example, all pixels outside the region fail to meet the region growing criteria, making it impossible to merge additional pixels into the region. For another example, the shape of the region is constrained by a predetermined shape, making it impossible to merge additional pixels into the region. For another example, the number of pixels included in the region reaches a predetermined threshold, making it impossible to merge additional pixels into the region. Based on this, the electronic device can determine that the region growing processing for the first and second preset locations is complete. Accordingly, the electronic device may determine the area determined by performing region growing processing on the first preset position and the second preset position as the finger mask area. The finger mask area is, for example, Figure 4 shown.

[0056] In this way, the electronic device can determine the finger mask area in different ways based on the first preset position and the second preset position, which is conducive to improving the convenience and flexibility of determining the finger mask area. When the electronic device determines the finger mask area corresponding to the fingertip point and the finger joint point, the finger mask area can be used by the electronic device to calibrate the first preset position of the fingertip point to determine the target position of the fingertip point, which is conducive to subsequently improving the accuracy of fingertip position recognition.

[0057] S104: Determine the distance between the third preset position of the contour point in the finger mask area and the preset circumscribed area corresponding to the finger mask area.

[0058] For example, the finger mask area may include multiple contour points. Figure 4 As shown, the finger mask area may have a clear outline. The electronic device can then determine each pixel located on the outline of the finger mask area as a contour point of the finger mask area. When determining the finger mask area, the electronic device can determine the third preset position of each contour point. Accordingly, the target position of the fingertip point can be determined based on the third preset position of one of the contour points in the finger mask area.

[0059] In some embodiments, since the fingertip point and the finger joint point are a rigid body, and the shape formed by the fingertip point and the finger joint point is similar to a rectangle, the electronic device can identify the minimum bounding rectangle corresponding to the finger mask area. Accordingly, the electronic device can determine the identified minimum bounding rectangle as a preset bounding area. The preset bounding area has a direction, and the direction of the preset bounding area is consistent with the direction of the finger mask area. For example, the electronic device can use the minimum bounding rectangle recognition function provided by the Open Source Computer Vision Library (OpenCV), such as the minAreaRect function, to find the minimum bounding rotated rectangle of the finger mask area, thereby determining the preset bounding area corresponding to the finger mask area. The preset bounding area is, for example, Figure 5 shown.

[0060] In the case of determining the preset circumscribed area, the electronic device can assist in determining the target position of the fingertip point based on the preset circumscribed area. For example, the electronic device can determine the distance between the third preset position of the contour point in the finger mask area and the preset circumscribed area. Regardless of the direction of the hand in the hand image, the fingertip points of the fingers in the hand are all located at one of the edge positions of the hand, which is equivalent to the fingertip points being on the contour of the finger mask area. Based on this, in the case of the electronic device determining the preset circumscribed area corresponding to the finger mask area, the preset circumscribed area is equivalent to the area closest to the contour of the finger mask area, and the electronic device can determine the distance between the third preset position of the contour point in the finger mask area and the preset circumscribed area for subsequent determination of the target position of the fingertip point.

[0061] In some embodiments, the preset circumscribed area is divided into a first search area and a second search area according to the center point of the preset circumscribed area; the target search area is determined from the first search area and the second search area according to the first preset position; when there is an overlapping area where a contour point is located between the target search area and the finger mask area, the distance between the third preset position of the contour point located in the overlapping area and the preset circumscribed area is determined.

[0062] For example, since the finger mask area can have a large number of contour points, and the target position of the finger tip point is generally near the first preset position of the finger tip point, the electronic device can determine the target search area based on the preset external area and the first preset position to reduce the range of searching for the finger tip point from the contour points of the finger mask area.

[0063] In the process of determining the target search area, the electronic device can divide the preset circumscribed area into a first search area and a second search area based on the center point of the preset circumscribed area. For example, the electronic device can make a center line perpendicular to the long side of the preset circumscribed area based on the center point of the preset circumscribed area. Accordingly, the electronic device can determine the part of the preset circumscribed area located on the left side of the center line as the first search area, and determine the other part of the preset circumscribed area located on the right side of the center line as the second search area. Of course, the method of determining the first search area and the second search area is not limited to this. For example, the electronic device can determine the part of the preset circumscribed area located on the left side of the center line as the second search area, and determine the other part of the preset circumscribed area located on the right side of the center line as the first search area. There is no limitation here.

[0064] When determining the first search area and the second search area, the electronic device may determine the target search area from the first search area and the second search area according to the first preset position of the fingertip point. The electronic device may determine the search area where the first preset position of the fingertip point is located as the target search area. Figure 6 As shown, taking the first search area as a partial preset external area located on the left side of the center line and the second search area as another partial preset external area located on the right side of the center line as an example, since the first preset position is located in the first search area, the electronic device can determine the first search area as the target search area.

[0065] When determining the target search area, the electronic device can determine the overlap area between the target search area and the finger mask area. Accordingly, when determining the overlap area, the electronic device only needs to determine the distance between the third preset position of the contour point located in the overlap area and the preset circumscribed area to subsequently determine the target position of the fingertip point.

[0066] For example, the electronic device may draw a perpendicular line from the contour point to the edge of the preset circumscribed area based on the third preset position of the contour point in the overlapped area. The preset circumscribed area may be a rectangle, and the edges of the preset circumscribed area may include a long side and a short side. Because the width of the fingertip of a finger on the user's hand is generally smaller than the length between the fingertip point and the adjacent finger joint point, the electronic device may determine the distance between the third preset position of the contour point and the preset circumscribed area based on the length of the perpendicular line from the contour point to the short side of the preset circumscribed area.

[0067] In an exemplary embodiment, the electronic device may determine, from among the contour points located in the overlapping area, the contour point with the smallest distance from the first preset position as the search starting point. Starting from the search starting point, the electronic device sequentially determines the distance between the third preset position of the contour points located in the overlapping area and the preset circumscribed area according to a preset search direction. The electronic device may also select any contour point from among the contour points located in the overlapping area as the search starting point, and sequentially determine the distance corresponding to each contour point according to the preset search direction, without limitation. The preset search direction may be pre-set or user-defined, without limitation.

[0068] Of course, this is not limited to this. In another exemplary embodiment, the electronic device may determine the distance between the third preset position of all contour points located in the overlapping area and the preset circumscribed area. The electronic device may also determine the distance between the third preset position of some contour points located in the overlapping area and the preset circumscribed area. For example, if there is a distance less than or equal to a preset distance threshold, the electronic device may stop determining the distance between the third preset position of other contour points located in the overlapping area and the preset circumscribed area, which is not limited here. The preset distance threshold may be pre-set or user-defined, which is not limited here.

[0069] In this way, the electronic device determines the target search area by combining the center point of the preset external area and the first preset position. Then, in the process of determining the target position of the finger tip point, the electronic device only needs to combine the target search area and the finger mask area to determine the overlapping area between the target search area and the finger mask area, and determine the distance between the third preset position of the contour point located in the overlapping area and the preset external area, without having to determine the distance between the third preset position of the contour point located outside the overlapping area and the preset external area, which is beneficial to improving the efficiency of determining the distance corresponding to the contour point, and thus beneficial to subsequently improving the convenience and efficiency of determining the target position of the finger tip point.

[0070] S105 : Determine the target position of the fingertip point from the third preset positions of the plurality of contour points according to the distances corresponding to the plurality of contour points.

[0071] Because the fingertip is located on the contour of the finger mask area and is the most prominent part of the user's finger, the fingertip is closest to the preset circumscribed area compared to other finger contour points other than the fingertip. Based on this, when the electronic device determines the distances corresponding to multiple contour points on the finger mask area, it can compare the distances corresponding to the multiple contour points to determine the target position of the fingertip. For example, the electronic device can determine the third preset position of the contour point with the smallest distance from the preset circumscribed area as the target position of the fingertip.

[0072] In some embodiments, a third preset position of a contour point having the smallest distance from a short side of a preset circumscribed area is determined as the target position of the fingertip point.

[0073] The preset circumscribed area may include a rectangle, and the sides of the rectangle may include a long side and a short side. The size information of the long side is greater than the size information of the short side. Since the finger mask area is determined based on the first preset position of the fingertip point and the second preset position of the finger joint point, the distance between the first preset position and the second preset position is generally greater than the width corresponding to the fingertip. Therefore, compared with the long side of the preset circumscribed area, the size information of the short side of the preset circumscribed area will be more consistent with the width corresponding to the fingertip, that is, the short side of the preset circumscribed area will be closer to the fingertip point. Based on this, the electronic device can draw a perpendicular line from the contour point as the starting point to the short side of the preset circumscribed area, and determine the distance between the third preset position of the contour point and the preset circumscribed area based on the length of the perpendicular line between the contour point and the short side of the preset circumscribed area. When it is determined that the distance between the third preset position where the contour point exists and the short side of the preset circumscribed area is the smallest, the third preset position of the contour point can be determined as the target position of the fingertip point.

[0074] For example, when the distance between the third preset position of multiple contour points and the short side of the preset circumscribed area is the smallest, the electronic device can determine the target position of the fingertip point based on the distance between the contour point and the first preset position.

[0075] like Figure 6As shown, since the length of the perpendicular line between the third preset position of contour point 1 and the short side of the preset circumscribed area is 0, the distance between the third preset position of contour point 1 and the short side of the preset circumscribed area can be determined to be 0. Since the length of the perpendicular line between the third preset position of contour point 2 and the short side of the preset circumscribed area is 0, the distance between the third preset position of contour point 2 and the short side of the preset circumscribed area is 0. The electronic device can determine that the distances between the third preset positions of contour point 1 and contour point 2 and the short side of the preset circumscribed area are both smaller than the distances between the third preset positions of other contour points in the finger mask area and the short side of the preset circumscribed area. Based on this, the electronic device can determine the distances between the third preset positions of contour point 1 and contour point 2 and the first preset position of the fingertip point. If the distance between the third preset position of contour point 1 and the first preset position is smaller than the distance between the third preset position of contour point 2 and the first preset position, the electronic device can determine the third preset position of contour point 1 as the target position of the fingertip point.

[0076] In this way, the electronic device determines the target position of the fingertip point from the third preset positions of each of the multiple contour points based on the distances corresponding to each of the multiple contour points. It can calibrate the first preset position of the fingertip point in combination with the distances corresponding to each of the multiple contour points and the third preset positions of each of the multiple contour points to obtain the target position of the fingertip point, which is conducive to improving the accuracy of position recognition of the fingertip point.

[0077] The hand recognition method provided in the above embodiment includes: acquiring a hand image; determining a first preset position of a finger tip point and a second preset position of a finger joint point adjacent to the finger tip point based on the hand image; determining a finger mask area corresponding to the finger tip point and the finger joint point from the hand image based on the first preset position and the second preset position; determining a distance between a third preset position of a contour point in the finger mask area and a preset circumscribed area corresponding to the finger mask area; and determining a target position of the finger tip point from each of the third preset positions of the multiple contour points based on the distances corresponding to each of the multiple contour points.

[0078] When determining a first preset position of a fingertip and a second preset position of a finger joint based on a hand image, the electronic device can determine the finger mask area corresponding to the fingertip and the finger joint. Accordingly, the electronic device can calibrate the first preset position of the fingertip by combining the contour points of the finger mask area with the preset circumscribed area corresponding to the finger mask area to obtain a target position of the fingertip, thereby improving the accuracy of fingertip position recognition.

[0079] See also Figure 7 , Figure 7This is a schematic block diagram of a hand recognition device provided in an embodiment of the present application. The hand recognition device can be configured in an electronic device or a server to execute the aforementioned hand recognition method. The electronic device can be a mobile phone, a tablet computer, a laptop computer, a desktop computer, a wearable device, and the like. Wearable devices can include smart bracelets, smart watches, near-eye display devices, and the like. Near-eye display devices can include VR glasses, AR glasses, MR glasses, VR helmets, AR helmets, MR helmets, and the like, which are not limited here. The server can be a separate server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data and artificial intelligence platforms.

[0080] like Figure 7 As shown, the hand recognition device includes an acquisition module 110 , a first determination module 120 , a second determination module 130 , a third determination module 140 and a fourth determination module 150 .

[0081] The acquisition module 110 is configured to acquire a hand image.

[0082] The first determining module 120 is configured to determine, based on the hand image, a first preset position of a fingertip point and a second preset position of a finger joint point adjacent to the fingertip point.

[0083] The second determining module 130 is configured to determine, from the hand image, a finger mask area corresponding to the fingertip point and the finger joint point according to the first preset position and the second preset position.

[0084] The third determining module 140 is configured to determine a distance between a third preset position of a contour point in the finger mask area and a preset circumscribed area corresponding to the finger mask area.

[0085] The fourth determining module 150 is configured to determine the target position of the fingertip point from the third preset positions of the plurality of contour points according to the distances corresponding to the plurality of contour points.

[0086] Exemplarily, the second determination module 130 includes a hand mask determination submodule and a finger mask determination submodule.

[0087] The hand mask determination submodule is used to perform image segmentation processing on the hand image to obtain a hand mask area corresponding to the hand image.

[0088] The finger mask determination submodule is configured to determine a finger mask area from the hand mask area according to the first preset position and the second preset position.

[0089] Exemplarily, the finger mask determination submodule includes a rectangular area determination submodule and a border expansion processing submodule.

[0090] The rectangular area determination submodule is configured to determine a corresponding preset rectangular area from the hand mask area according to the first preset position and the second preset position.

[0091] The edge expansion processing submodule is used to perform edge expansion processing on the outline of the preset rectangular area to obtain a finger mask area.

[0092] Exemplarily, the finger mask determination submodule includes a region growing submodule.

[0093] The region growing submodule is used to perform region growing processing on the first preset position and the second preset position based on the hand mask area to obtain a finger mask area.

[0094] Exemplarily, the third determination module 140 includes a region division submodule, a region determination submodule, and a distance determination submodule.

[0095] The area division submodule is configured to divide the preset circumscribed area into a first search area and a second search area according to a center point of the preset circumscribed area.

[0096] The area determination submodule is configured to determine a target search area from the first search area and the second search area according to the first preset position.

[0097] The distance determination submodule is used to determine the distance between the third preset position of the contour point located in the overlapping area and the preset circumscribed area when there is a contour point located in the overlapping area between the target search area and the finger mask area.

[0098] Exemplarily, the fourth determination module 150 includes a position determination submodule.

[0099] The position determination submodule is configured to determine a third preset position of a contour point having the smallest distance from a short side of the preset circumscribed area as a target position of the fingertip point.

[0100] Exemplarily, the first determination module 120 includes a location identification submodule.

[0101] The position recognition submodule is used to perform recognition processing on the hand image according to a preset image recognition algorithm to obtain a first preset position of the fingertip point and a second preset position of a finger joint point adjacent to the fingertip point.

[0102] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0103] The method of the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0104] Exemplarily, the above-mentioned methods and devices can be implemented in the form of a computer program, which can be run on an electronic device or a server to control the electronic device. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a desktop computer, a wearable device, and the like. Wearable devices can include smart bracelets, smart watches, near-eye display devices, and the like. Near-eye display devices can include VR glasses, AR glasses, MR glasses, VR helmets, AR helmets, MR helmets, and the like, without limitation. The server can be a separate server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data and artificial intelligence platforms.

[0105] See also Figure 8 , Figure 8 This is a schematic block diagram of the structure of an electronic device provided in an embodiment of the present application.

[0106] like Figure 8 As shown, the electronic device includes a memory and a processor. The memory and the processor may be connected via a system bus, and the memory may include a storage medium and an internal memory.

[0107] The storage medium may store an operating system and a computer program. When the computer program is executed, the processor may execute any one of the hand recognition methods.

[0108] The processor is used to provide computing and control capabilities to support the operation of the entire electronic device.

[0109] The internal memory provides an environment for the operation of the computer program in the storage medium. When the computer program is executed by the processor, the processor can execute any hand recognition method.

[0110] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0111] It should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0112] In one embodiment, the processor is configured to execute a computer program and implement the following steps when executing the computer program:

[0113] Acquire hand images;

[0114] Determining, based on the hand image, a first preset position of a fingertip point and a second preset position of a finger joint point adjacent to the fingertip point;

[0115] Determining, from the hand image, a finger mask area corresponding to the fingertip point and the finger joint point according to the first preset position and the second preset position;

[0116] Determining a distance between a third preset position of a contour point in the finger mask area and a preset circumscribed area corresponding to the finger mask area;

[0117] According to the distances corresponding to the plurality of contour points, the target position of the fingertip point is determined from the third preset positions of the plurality of contour points.

[0118] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of hand recognition described above can refer to the corresponding process in the aforementioned hand recognition method embodiment, and will not be repeated here.

[0119] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. The method implemented when the computer program is executed by a processor can refer to the various embodiments of the hand recognition method of the present application.

[0120] The computer-readable storage medium may be an internal storage unit of the electronic device described in the aforementioned embodiment, such as a hard disk or memory of the electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash memory card, etc. equipped on the electronic device.

[0121] It should be understood that the terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0122] It should also be understood that the term "and / or" used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system that includes the element.

[0123] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments. The above description is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A hand recognition method, characterized in that: include: Acquire a hand image; Determine, according to the hand image, a first preset position of a fingertip point and a second preset position of a finger joint point adjacent to the fingertip point; Determining a finger mask area corresponding to the fingertip point and the finger joint point from the hand image according to the first preset position and the second preset position; Determine a distance between a third preset position of a contour point in the finger mask area and a preset circumscribed area corresponding to the finger mask area; According to the distances corresponding to the plurality of contour points, the target position of the fingertip point is determined from the third preset positions of the plurality of contour points.

2. The hand recognition method according to claim 1, characterized in that: The determining, from the hand image, a finger mask area corresponding to the fingertip point and the finger joint point according to the first preset position and the second preset position, comprises: Performing image segmentation processing on the hand image to obtain a hand mask area corresponding to the hand image; A finger mask area is determined from the hand mask area according to the first preset position and the second preset position.

3. The hand recognition method according to claim 2, characterized in that: The determining the finger mask area from the hand mask area according to the first preset position and the second preset position includes: Determining a corresponding preset rectangular area from the hand mask area according to the first preset position and the second preset position; The outline of the preset rectangular area is expanded to obtain a finger mask area.

4. The hand recognition method according to claim 2, characterized in that: The determining the finger mask area from the hand mask area according to the first preset position and the second preset position includes: Based on the hand mask area, a region growing process is performed on the first preset position and the second preset position to obtain a finger mask area.

5. The hand recognition method according to any one of claims 1 to 4, characterized in that: The determining of the distance between the third preset position of the contour point in the finger mask area and the preset circumscribed area corresponding to the finger mask area includes: Dividing the preset circumscribed area into a first search area and a second search area according to a center point of the preset circumscribed area; Determine a target search area from the first search area and the second search area according to the first preset position; When there is an overlapped area where a contour point is located between the target search area and the finger mask area, a distance between a third preset position of the contour point located in the overlapped area and the preset circumscribed area is determined.

6. The hand recognition method according to any one of claims 1 to 4, characterized in that: The step of determining the target position of the fingertip point from respective third preset positions of the plurality of contour points according to respective corresponding distances of the plurality of contour points comprises: A third preset position of a contour point having a minimum distance from a short side of the preset circumscribed area is determined as a target position of the fingertip point.

7. The hand recognition method according to any one of claims 1 to 4, characterized in that: The step of determining a first preset position of a fingertip point and a second preset position of a finger joint point adjacent to the fingertip point according to the hand image includes: According to a preset image recognition algorithm, the hand image is recognized and processed to obtain a first preset position of the finger tip point and a second preset position of a finger joint point adjacent to the finger tip point.

8. A hand recognition device, characterized in that: The hand recognition device comprises: An acquisition module, used for acquiring a hand image; A first determining module, used to determine a first preset position of a fingertip point and a second preset position of a finger joint point adjacent to the fingertip point according to the hand image; a second determining module, configured to determine, from the hand image, a finger mask area corresponding to the fingertip point and the finger joint point according to the first preset position and the second preset position; A third determination module, used to determine the distance between a third preset position of the contour point in the finger mask area and a preset circumscribed area corresponding to the finger mask area; The fourth determination module is used to determine the target position of the fingertip point from the third preset positions of the plurality of contour points according to the distances corresponding to the plurality of contour points.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the steps of the hand recognition method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the hand recognition method according to any one of claims 1 to 7 are implemented.

Images