Palm key point recognition method and device, computer device and medium

Abstract

The application relates to the technical field of artificial intelligence, in particular to a palm key point identification method and device, computer equipment and a medium, a current image requiring gesture recognition is acquired; human key point detection is performed on the current image, and a plurality of sets of human key point coordinates are obtained; a plurality of palm region images are extracted from the current image according to the plurality of sets of human key point coordinates; the plurality of palm region images are combined into an input image suitable for a multi-target palm key point model; palm key point detection is performed on the input image through the multi-target palm key point model, and palm key point coordinates corresponding to each palm region image are obtained; the palm key point coordinates corresponding to each palm region image are mapped into the current image, and target palm key point coordinates are obtained, and the application can realize efficient and real-time multi-target palm key point identification.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method for recognizing key points on the palm, a device for recognizing key points on the palm, a computer device, and a computer-readable storage medium. Background Technology

[0002] Currently, with the rapid development of human-computer interaction technology, gesture recognition, as a natural and intuitive interaction method, has been widely used in smart homes, virtual reality, in-vehicle systems, and industrial control. In real-world recognition scenarios (such as living rooms, large conference rooms, and in-vehicle environments), there is often a need to simultaneously address complex recognition requirements involving long distances (e.g., 3-4 meters) and multiple targets (multiple people and hands). However, traditional solutions suffer from significant resource consumption and computational latency when handling such scenarios, making it difficult to meet the performance demands of real-time interaction. Summary of the Invention

[0003] This application provides a method, device, computer equipment, and computer-readable storage medium for palm key point recognition, which can achieve efficient and real-time multi-target palm key point recognition.

[0004] Firstly, the palm key point recognition method provided in this application includes: Obtain the current image from which gesture recognition needs to be performed; Human keypoint detection is performed on the current image to obtain multiple sets of human keypoint coordinates; Based on multiple sets of human body key point coordinates, extract multiple palm region images from the current image; Multiple palm region images are combined into an input image that adapts to a multi-target palm keypoint model; The input image is used to detect palm key points by a multi-target palm key point model, and the coordinates of palm key points corresponding to each palm region image are obtained. Map the coordinates of key points of the palm corresponding to each palm region image to the current image to obtain the coordinates of key points of the target palm.

[0005] Secondly, the palm key point recognition device provided in this application includes: The image acquisition module is used to acquire the current image that needs to be used for gesture recognition; The first detection module is used to detect human key points in the current image and obtain multiple sets of human key point coordinates. The region extraction module is used to extract multiple palm region images from the current image based on multiple sets of human body key point coordinates; The image binning module is used to combine multiple palm region images into an input image that adapts to a multi-target palm keypoint model; The second detection module is used to detect palm key points in the input image using a multi-target palm key point model, and obtain the palm key point coordinates corresponding to each palm region image. The coordinate mapping module is used to map the coordinates of key points of the palm corresponding to each palm region image to the current image, so as to obtain the coordinates of key points of the target palm.

[0006] Optionally, in one embodiment, when extracting multiple palm region images from the current image based on multiple sets of human key point coordinates, the region extraction module is used to: Obtain the coordinates of key points of the hand of the target from previous historical images; Wrist coordinates were extracted from the coordinates of key human body points in each group to obtain multiple wrist coordinates. Based on the coordinates of key points of the target hand and multiple wrist points in the historical image, extract multiple hand region images from the current image.

[0007] Optionally, in one embodiment, when extracting multiple palm region images from the current image based on historical target palm keypoint coordinates and multiple wrist point coordinates, the region extraction module is used to: For each wrist point coordinate, the hand movement speed is calculated based on the historical wrist point coordinates in the corresponding historical image. If the hand movement speed corresponding to the wrist point coordinate does not exceed the speed threshold, then the corresponding hand area image is extracted from the current image based on the historical target hand key point coordinates corresponding to the wrist point coordinate. If the hand movement speed corresponding to the wrist point coordinate exceeds the speed threshold, the initial cropping range is determined from the current image based on the historical target hand key point coordinates corresponding to the wrist point coordinate, and the initial cropping range is corrected based on the wrist point coordinate, and the corresponding hand region image is extracted from the current image based on the corrected cropping range.

[0008] Optionally, in one embodiment, when extracting multiple palm region images from the current image based on multiple sets of human key point coordinates, the region extraction module is used to: Wrist coordinates were extracted from the coordinates of key human body points in each group to obtain multiple wrist coordinates. Based on the depth value of each wrist point coordinate, determine the cropping range of the palm to which each wrist point coordinate belongs; Based on the cropping range corresponding to the coordinates of each wrist point, extract multiple palm region images from the current image.

[0009] Optionally, in one embodiment, when combining multiple palm region images into an input image adapted to a multi-target palm keypoint model, the image binning module is used to: When the number of extracted palm region images exceeds a preset threshold, the preset threshold is used as the maximum number of combinations to combine multiple palm region images into multiple input images.

[0010] Optionally, in one embodiment, when combining multiple palm region images into an input image adapted to a multi-target palm keypoint model, the image binning module is used to: When the number of extracted palm region images exceeds a preset threshold, a quality score is given to each palm region image to obtain a quality score for each palm region image. Based on the quality scores, images of the palm region with the highest quality scores (a preset threshold number) are selected and combined into the input image.

[0011] Optionally, in one embodiment, when performing human keypoint detection on the current image to obtain multiple sets of human keypoint coordinates, the first detection module is used to: Using a zero-copy method, human keypoints are detected in the current image, resulting in multiple sets of human keypoint coordinates.

[0012] Thirdly, the computer device provided in this application includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the palm key point recognition method provided in this application.

[0013] Fourthly, the computer-readable storage medium provided in this application stores a computer program that, when executed by a processor, implements the palm key point recognition method provided in this application.

[0014] This application provides a hand keypoint recognition scheme, which involves acquiring the current image for gesture recognition; performing human keypoint detection on the current image to obtain multiple sets of human keypoint coordinates; extracting multiple hand region images from the current image based on the multiple sets of human keypoint coordinates; combining the multiple hand region images into an input image adapted to a multi-target hand keypoint model; performing hand keypoint detection on the input image using the multi-target hand keypoint model to obtain the hand keypoint coordinates corresponding to each hand region image; and mapping the hand keypoint coordinates corresponding to each hand region image to the current image to obtain the target hand keypoint coordinates. In this way, by using human keypoints to locate the hand region and extracting multiple hand region images, integrating these images into a single input image, and then using a multi-target hand keypoint model for parallel detection, not only can background interference be eliminated, but parallel detection of multiple target hand keypoints can also be completed within a single detection overhead. This not only improves detection accuracy but also significantly reduces computational resource consumption, achieving efficient and real-time multi-target hand keypoint recognition. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram illustrating an application scenario of the palm key point recognition method provided in this application embodiment; Figure 2 This is a flowchart illustrating the palm key point recognition method provided in an embodiment of this application; Figure 3 This is a schematic diagram of the hand key point recognition device provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of the computer device provided in the embodiments of this application. Detailed Implementation

[0017] To make the technical problems, technical solutions, and beneficial effects solved by this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0018] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0019] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0020] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0021] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0022] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0023] This application provides a palm key point recognition method, a palm key point recognition device, a computer device, a computer-readable storage medium, and a computer program product. The palm key point recognition method can be executed by the palm key point recognition device or by a computer device integrating the palm key point recognition device.

[0024] Please refer to Figure 1 This application also provides a palm key point recognition system, such as Figure 1 As shown, the palm keypoint recognition system includes a computer device, which integrates the palm keypoint recognition device provided in this application. For example, the computer device acquires the current image for which gesture recognition is required; performs human keypoint detection on the current image to obtain multiple sets of human keypoint coordinates; extracts multiple palm region images from the current image based on the multiple sets of human keypoint coordinates; combines the multiple palm region images into an input image adapted to a multi-target palm keypoint model; performs palm keypoint detection on the input image using the multi-target palm keypoint model to obtain the palm keypoint coordinates corresponding to each palm region image; and maps the palm keypoint coordinates corresponding to each palm region image to the current image to obtain the target palm keypoint coordinates.

[0025] The computer device can be any device equipped with a processor and capable of processing, such as mobile computer devices with processors, such as smartphones, tablets, handheld computers, laptops, virtual reality devices, augmented reality devices, or mixed reality devices, or fixed computer devices with processors, such as desktop computers, televisions, servers, and industrial equipment.

[0026] In addition, such as Figure 1As shown, the palm key point recognition system may also include a storage device, which may be built into the computer device or external to the computer device, for storing relevant data in the palm key point recognition process. For example, the computer device stores relevant data in the palm key point recognition process, such as the current image acquired, the coordinates of the detected human body key points, the extracted palm area image, and the coordinates of the generated palm key points, into the storage device.

[0027] It should be noted that, Figure 1 The schematic diagram of the palm key point recognition system shown is merely an example. The palm key point recognition system and scenario described in this application embodiment are for the purpose of more clearly illustrating the technical solutions of this application embodiment and do not constitute a limitation on the technical solutions provided in this application embodiment. As those skilled in the art will know, with the evolution of palm key point recognition systems and the emergence of new business scenarios, the technical solutions provided in this application embodiment are also applicable to similar technical problems.

[0028] Please refer to Figure 2 This is a flowchart illustrating a method for recognizing key points of the palm disclosed in an embodiment of this application. Figure 2 As shown, the process of this hand key point recognition method can be as follows: In S110, the current image for which gesture recognition needs to be performed is acquired.

[0029] The following section uses a computer device as the execution subject to elaborate on the palm key point recognition method provided in this application.

[0030] In this embodiment, the computer device can automatically trigger the palm key point recognition process, or it can initiate the process in response to a user's trigger command. For example, the computer device can automatically trigger the process when it detects a user entering the camera's field of view, or it can initiate the process in response to user operations such as voice commands or physical button clicks. In addition, users can also manually initiate recognition requests through virtual buttons in the application interface provided by the computer device to meet personalized needs in different interaction scenarios.

[0031] After triggering the palm key point recognition process, the computer device first acquires the current image that needs to be recognized for gesture recognition. This current image can be a video frame captured in real time by the computer device through a camera, or a static image file pre-stored locally or in the cloud, as long as it contains the palm information to be recognized. After acquiring the current image, the computer device can perform preprocessing operations on it, including but not limited to at least one operation such as noise reduction, contrast enhancement, or resolution adjustment.

[0032] In S120, human keypoint detection is performed on the current image to obtain multiple sets of human keypoint coordinates.

[0033] After acquiring the current image, the computer device analyzes it using a pre-trained human keypoint detection model, identifying the locations of all visible key parts of the human body in the image. This yields multiple sets of human keypoint coordinates, including those for the wrist, elbow, and shoulder. These coordinates not only indicate the overall posture of the human body but also provide crucial spatial reference for subsequent precise positioning of the palm area.

[0034] It should be noted that the embodiments of this application do not limit the specific structure, training method, or detection accuracy of the human keypoint detection model, as long as it can accurately output the coordinate information of human keypoints. For example, the lightweight YOLO-Pose can be used to detect human keypoints in the current image.

[0035] In S130, multiple palm region images are extracted from the current image based on multiple sets of human body key point coordinates.

[0036] As shown above, after detecting multiple sets of human keypoint coordinates, the computer device can extract the wrist coordinates from each set of human keypoint coordinates, and use these coordinates to determine the cropping range of the palm region, thereby cropping out an image of the palm region containing palm information. This cropping process effectively removes background interference, focuses on palm features, and can significantly reduce the computational complexity of subsequent palm keypoint detection.

[0037] The cropping area can be determined by calculating the bounding box based on the geometric relationship between the wrist point and adjacent key points (such as the elbow) using a preset ratio or an adaptive algorithm, to ensure that the cropping area completely covers the palm and minimizes background redundancy. For example, the height of the bounding box can be determined by extending a preset pixel distance along the forearm direction with the wrist point as the center, and the width can be determined by scaling the vertical distance from the wrist point to the elbow proportionally, thus forming a rectangular cropping area that fits the shape of the palm.

[0038] In S140, multiple palm region images are combined into an input image that adapts to a multi-target palm keypoint model.

[0039] In this embodiment, the computer device maintains a blank canvas with the same input resolution as the multi-target palm keypoint detection model. After extracting multiple palm region images from the current image, the computer device sequentially fills these palm region images into the blank canvas according to a preset arrangement rule (such as grid or linear tiling), thereby generating a combined input image containing all palm information to be detected.

[0040] In practice, the computer device can fill the blank canvas with images of each palm area in descending order of size, so as to make full use of the resolution space of the input image and avoid losing details due to insufficient resolution of small palms.

[0041] It should be noted that the embodiments of this application do not limit the specific structure, training method, and detection accuracy of the multi-target palm keypoint detection model, as long as it can accurately achieve the function of multi-target palm keypoint detection. For example, a lightweight detection network based on convolutional neural networks can be used.

[0042] In S150, the input image is used to detect palm key points through a multi-target palm key point model, and the coordinates of the palm key points corresponding to each palm region image are obtained.

[0043] As shown above, after combining the input image containing all the palm information to be detected, the computer device inputs the input image into the multi-target palm key point detection model. Through the forward inference of the multi-target palm key point detection model, the coordinates of the palm key points corresponding to each palm region image are obtained.

[0044] In S160, the coordinates of the key points of the palm corresponding to each palm region image are mapped to the current image to obtain the coordinates of the key points of the target palm.

[0045] It is understandable that the coordinates of the key points of the palm corresponding to each palm region image are obtained based on the image coordinate system of the input image. Therefore, the computer device further reverses the coordinates of the key points of the palm corresponding to each palm region image to the original image coordinate system of the current image based on the geometric transformation relationship (such as scaling ratio and translation offset) in the aforementioned cropping and combination process, so as to obtain the target palm key point coordinates representing each palm in the current image.

[0046] As described above, the hand keypoint recognition scheme provided in this application acquires the current image for gesture recognition; performs human keypoint detection on the current image to obtain multiple sets of human keypoint coordinates; extracts multiple hand region images from the current image based on the multiple sets of human keypoint coordinates; combines the multiple hand region images into an input image adapted to a multi-target hand keypoint model; performs hand keypoint detection on the input image using the multi-target hand keypoint model to obtain the hand keypoint coordinates corresponding to each hand region image; and maps the hand keypoint coordinates corresponding to each hand region image to the current image to obtain the target hand keypoint coordinates. In this way, by using human keypoints to locate the hand region and extracting multiple hand region images, integrating these multiple hand region images into a single input image, and then using the multi-target hand keypoint model for parallel detection, not only can background interference be eliminated, but parallel detection of multiple target hand keypoints can also be completed within a single detection overhead. This not only improves detection accuracy but also significantly reduces computational resource consumption, achieving efficient and real-time multi-target hand keypoint recognition.

[0047] Optionally, in one embodiment, multiple palm region images are extracted from the current image based on multiple sets of human body key point coordinates, including: Obtain the coordinates of key points of the hand of the target from previous historical images; Wrist coordinates were extracted from the coordinates of key human body points in each group to obtain multiple wrist coordinates. Based on the coordinates of multiple historical target hand key points and multiple wrist point coordinates, multiple hand region images are extracted from the current image.

[0048] Based on the above embodiments, this application provides an optional regional image extraction strategy.

[0049] In the process of extracting multiple palm region images from the current image based on multiple sets of human body key point coordinates, the computer device can obtain the coordinates of multiple historical target palm key points corresponding to the previous historical images, and combine them with the wrist point coordinates detected in the current image to determine the cropping range of each palm in the current image, thereby cropping out multiple palm region images.

[0050] For example, computer devices can predict the expected position of each hand in the current image based on the historical target hand key point coordinates, and then fine-tune the expected position by combining the wrist point coordinates of each hand in the current image, thereby determining a more accurate cropping range.

[0051] Optionally, in one embodiment, multiple palm region images are extracted from the current image based on historical target palm key point coordinates and multiple wrist point coordinates, including: For each wrist point coordinate, the hand movement speed is calculated based on the historical wrist point coordinates in the corresponding historical image. If the hand movement speed corresponding to the wrist point coordinate does not exceed the speed threshold, then the corresponding hand area image is extracted from the current image based on the historical target hand key point coordinates corresponding to the wrist point coordinate. If the hand movement speed corresponding to the wrist point coordinate exceeds the speed threshold, the initial cropping range is determined from the current image based on the historical target hand key point coordinates corresponding to the wrist point coordinate, and the initial cropping range is corrected based on the wrist point coordinate, and the corresponding hand region image is extracted from the current image based on the corrected cropping range.

[0052] Based on the above embodiments, this application provides a palm region extraction strategy based on motion speed adaptation.

[0053] Specifically, for each wrist point coordinate, the hand movement speed represented by that wrist point coordinate is calculated based on the historical wrist point coordinates in the corresponding historical image. Then, the hand movement speed is compared with a preset speed threshold to determine the current movement state of the hand.

[0054] If the hand movement speed corresponding to the wrist point coordinate does not exceed the speed threshold, the hand is determined to be in a low-speed or stationary state. At this time, the computer device can directly determine the cropping range based on the historical target hand key point coordinates corresponding to the wrist point coordinate and crop the corresponding hand area image from the current image.

[0055] If the hand movement speed corresponding to the wrist point coordinate exceeds a speed threshold, the hand is determined to be in a high-speed movement state. At this time, the computer device can first determine the initial cropping range based on the historical target hand key point coordinates corresponding to the wrist point coordinate, and then correct the initial cropping range in combination with the wrist point coordinate to compensate for the displacement deviation caused by high-speed movement, thereby ensuring that the corrected cropping range can completely cover the hand area. Then, the corresponding hand area image is extracted from the current image based on the corrected cropping range.

[0056] Specifically, when correcting the initial cropping range based on wrist point coordinates, the computer device can determine the translation amount and direction corresponding to the initial cropping range based on the displacement vector between the wrist point coordinates and their corresponding historical wrist point coordinates, and then make corresponding translation adjustments to the initial cropping range. Furthermore, the scaling ratio of the cropping range can be dynamically adjusted based on the magnitude of the displacement vector to further adapt to the rapid movement of the palm in the depth direction, ensuring the integrity of the extracted area.

[0057] Optionally, in one embodiment, multiple palm region images are extracted from the current image based on multiple sets of human body key point coordinates, including: Wrist coordinates were extracted from the coordinates of key human body points in each group to obtain multiple wrist coordinates. Based on the depth value of each wrist point coordinate, determine the cropping range of the palm to which each wrist point coordinate belongs; Based on the cropping range corresponding to the coordinates of each wrist point, extract multiple palm region images from the current image.

[0058] Based on the above embodiments, this application also provides another optional palm region extraction strategy.

[0059] For each set of human keypoint coordinates, the computer device extracts the wrist point coordinates. Then, based on the depth value of each wrist point coordinate and a preset correspondence between depth values ​​and scaling factors, the computer device determines the scaling factor corresponding to each wrist point coordinate. Based on the scaling factor corresponding to each wrist point coordinate, the computer device scales and adjusts a preset baseline cropping range to obtain the cropping range corresponding to each wrist point coordinate. The baseline cropping range can be preset according to the actual application scenario, such as being determined based on standard palm size or historical average palm size.

[0060] Finally, the computer device extracts the corresponding palm region image from the current image based on the cropping range corresponding to each wrist point coordinate, resulting in multiple palm region images.

[0061] Optionally, in one embodiment, combining multiple palm region images into an input image adapted to a multi-target palm keypoint model includes: When the number of extracted palm region images exceeds a preset threshold, the preset threshold is used as the maximum number of combinations to combine multiple palm region images into multiple input images.

[0062] Based on the above embodiments, this application provides an optional packing strategy.

[0063] To ensure the accuracy and computational efficiency of key point recognition, this application embodiment sets a fixed maximum upper limit for the number of hands in each input image, i.e., a preset threshold.

[0064] When combining multiple palm region images into an input image that adapts to a multi-target palm key point model, if the number of extracted palm region images exceeds a preset threshold, the computer device will use the preset threshold as the maximum number of combinations and combine the multiple palm region images into multiple input images.

[0065] For example, assuming the preset threshold is configured to 3, when extracting and outputting 4 palm region images, the computer device can combine 3 of the palm region images into one input image, and combine the remaining palm region image into another input image separately.

[0066] Accordingly, in the embodiments of this application, the computer device inputs each input image into a multi-target palm key point model for key point detection, thereby obtaining the palm key point coordinates corresponding to each palm region image.

[0067] Optionally, in one embodiment, combining multiple palm region images into an input image adapted to a multi-target palm keypoint model includes: When the number of extracted palm region images exceeds a preset threshold, a quality score is given to each palm region image to obtain a quality score for each palm region image. Based on the quality scores, images of the palm region with the highest quality scores (a preset threshold number) are selected and combined into the input image.

[0068] Based on the above embodiments, this application provides an optional packing strategy.

[0069] To ensure the accuracy and computational efficiency of key point recognition, this application embodiment sets a fixed maximum upper limit for the number of hands in each input image, i.e., a preset threshold.

[0070] When combining multiple palm region images into an input image that adapts to a multi-target palm keypoint model, if the number of extracted palm region images exceeds a preset threshold, the computer device will score the quality of each palm region image according to the configured quality scoring strategy, and select the palm region images with the highest quality scores and the preset threshold number of images to combine into an input image.

[0071] It should be noted that the embodiments of this application do not specifically limit the quality scoring strategy, which can be configured by those skilled in the art according to actual needs. For example, a scoring function can be constructed based on indicators such as image clarity, palm integrity and / or lighting conditions, and the scoring results can be weighted to obtain the quality score of each palm region image.

[0072] By employing the bin packing strategy based on quality scoring, low-quality palm region images can be effectively filtered out, thereby significantly improving the overall accuracy and robustness of key point detection while ensuring the rational use of computing resources.

[0073] Optionally, in one embodiment, human keypoint detection is performed on the current image to obtain multiple sets of human keypoint coordinates, including: Using a zero-copy method, human keypoints are detected in the current image, resulting in multiple sets of human keypoint coordinates.

[0074] Zero-copy mode refers to accessing data directly through pointers or handles without memory copying, thereby reducing the data copying overhead between the CPU and GPU and improving detection efficiency. Taking the current image as a video frame as an example, the video frame data can reside directly in the video memory. The computer device can directly reference the video frame address in the video memory as the input of the human keypoint detection model, without going through the system memory. This avoids the redundant operation of transferring data from the video memory back to the main memory and then reloading it, thus improving the overall efficiency of palm keypoint recognition.

[0075] To facilitate better implementation of the above palm key point recognition method, this application also provides a corresponding palm key point recognition device. The meanings of the terms used are the same as in the above palm key point recognition method; for specific implementation details, please refer to the descriptions in the above method embodiments.

[0076] Please refer to Figure 3 The hand key point recognition device may include an image acquisition module 210, a first detection module 220, a region extraction module 230, an image binning module 240, a second detection module 250, and a coordinate mapping module 260. Detailed descriptions of each functional module are as follows: Image acquisition module 210 is used to acquire the current image that needs to be used for gesture recognition; The first detection module 220 is used to detect human key points in the current image and obtain multiple sets of human key point coordinates. The region extraction module 230 is used to extract multiple palm region images from the current image based on multiple sets of human body key point coordinates; Image binning module 240 is used to combine multiple palm region images into an input image that adapts to a multi-target palm keypoint model; The second detection module 250 is used to detect palm key points in the input image through a multi-target palm key point model, and obtain the palm key point coordinates corresponding to each palm region image. The coordinate mapping module 260 is used to map the coordinates of the key points of the palm corresponding to each palm region image to the current image to obtain the coordinates of the key points of the target palm.

[0077] Optionally, in one embodiment, when extracting multiple palm region images from the current image based on multiple sets of human key point coordinates, the region extraction module 230 is used to: Obtain the coordinates of key points of the hand of the target from previous historical images; Wrist coordinates were extracted from the coordinates of key human body points in each group to obtain multiple wrist coordinates. Based on the coordinates of key points of the target hand and multiple wrist points in the historical image, extract multiple hand region images from the current image.

[0078] Optionally, in one embodiment, when extracting multiple palm region images from the current image based on historical target palm key point coordinates and multiple wrist point coordinates, the region extraction module 230 is used to: For each wrist point coordinate, the hand movement speed is calculated based on the historical wrist point coordinates in the corresponding historical image. If the hand movement speed corresponding to the wrist point coordinate does not exceed the speed threshold, then the corresponding hand area image is extracted from the current image based on the historical target hand key point coordinates corresponding to the wrist point coordinate. If the hand movement speed corresponding to the wrist point coordinate exceeds the speed threshold, the initial cropping range is determined from the current image based on the historical target hand key point coordinates corresponding to the wrist point coordinate, and the initial cropping range is corrected based on the wrist point coordinate, and the corresponding hand region image is extracted from the current image based on the corrected cropping range.

[0079] Optionally, in one embodiment, when extracting multiple palm region images from the current image based on multiple sets of human key point coordinates, the region extraction module 230 is used to: Wrist coordinates were extracted from the coordinates of key human body points in each group to obtain multiple wrist coordinates. Based on the depth value of each wrist point coordinate, determine the cropping range of the palm to which each wrist point coordinate belongs; Based on the cropping range corresponding to the coordinates of each wrist point, extract multiple palm region images from the current image.

[0080] Optionally, in one embodiment, when combining multiple palm region images into an input image adapted to a multi-target palm keypoint model, the image binning module 240 is used to: When the number of extracted palm region images exceeds a preset threshold, the preset threshold is used as the maximum number of combinations to combine multiple palm region images into multiple input images.

[0081] Optionally, in one embodiment, when combining multiple palm region images into an input image adapted to a multi-target palm keypoint model, the image binning module 240 is used to: When the number of extracted palm region images exceeds a preset threshold, a quality score is given to each palm region image to obtain a quality score for each palm region image. Based on the quality scores, images of the palm region with the highest quality scores (a preset threshold number) are selected and combined into the input image.

[0082] Optionally, in one embodiment, when performing human keypoint detection on the current image to obtain multiple sets of human keypoint coordinates, the first detection module 220 is used to: Using a zero-copy method, human keypoints are detected in the current image, resulting in multiple sets of human keypoint coordinates.

[0083] Specific limitations regarding the palm key point recognition device can be found in the limitations of the palm key point recognition method described above, and will not be repeated here. Each module in the aforementioned palm key point recognition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0084] In one embodiment, a computer device is provided, which may be a monitoring device, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and the database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The network interface connects to an external wireless client, providing wireless network access services to the connected client. When executed by the processor, the computer program implements the palm key point recognition method provided in this application.

[0085] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the palm key point recognition method described in the above embodiment.

[0086] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the palm key point recognition method described above.

[0087] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0088] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0089] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for recognizing key points on the palm, characterized in that, The palm key point recognition method includes: Obtain the current image from which gesture recognition needs to be performed; Human key point detection is performed on the current image to obtain multiple sets of human key point coordinates; Based on the coordinates of multiple sets of key human body points, extract multiple palm region images from the current image; The multiple palm region images are combined into an input image that adapts to a multi-target palm key point model; The input image is detected by the multi-target palm key point model to obtain the palm key point coordinates corresponding to each palm region image. The coordinates of key points of the palm corresponding to each palm region image are mapped to the current image to obtain the coordinates of key points of the target palm.

2. The hand key point recognition method according to claim 1, characterized in that, The step of extracting multiple palm region images from the current image based on multiple sets of human body key point coordinates includes: Obtain the coordinates of multiple historical target hand key points corresponding to historical images preceding the current image; Wrist coordinates were extracted from the coordinates of key human body points described in each group to obtain multiple wrist coordinates. Based on the coordinates of multiple historical target hand key points and multiple wrist point coordinates, multiple hand region images are extracted from the current image.

3. The hand key point recognition method according to claim 2, characterized in that, The step of extracting multiple palm region images from the current image based on multiple historical target palm key point coordinates and multiple wrist point coordinates includes: For each wrist point coordinate, the hand movement speed is calculated based on the historical wrist point coordinates in the corresponding historical image. If the hand movement speed corresponding to the wrist point coordinate does not exceed the speed threshold, then the corresponding hand area image is extracted from the current image based on the historical target hand key point coordinates corresponding to the wrist point coordinate. If the hand movement speed corresponding to the wrist point coordinate exceeds the speed threshold, then the initial cropping range is determined from the current image based on the historical target hand key point coordinates corresponding to the wrist point coordinate, and the initial cropping range is corrected based on the wrist point coordinate, and the corresponding hand region image is extracted from the current image based on the corrected cropping range.

4. The hand key point recognition method according to claim 1, characterized in that, The step of extracting multiple palm region images from the current image based on multiple sets of human body key point coordinates includes: Wrist coordinates were extracted from the coordinates of key human body points described in each group to obtain multiple wrist coordinates. Based on the depth value of each wrist point coordinate, determine the cropping range of the palm to which each wrist point coordinate belongs; Based on the cropping range corresponding to the coordinates of each wrist point, multiple palm region images are extracted from the current image.

5. The hand key point recognition method according to claim 1, characterized in that, The step of combining the multiple palm region images into an input image adapted to a multi-target palm keypoint model includes: When the number of extracted palm region images exceeds a preset threshold, the preset threshold is used as the maximum number of combinations to combine multiple palm region images into multiple input images.

6. The hand key point recognition method according to claim 1, characterized in that, The step of combining the multiple palm region images into an input image adapted to a multi-target palm keypoint model includes: When the number of extracted palm region images exceeds a preset threshold, a quality score is given to each palm region image to obtain a quality score for each palm region image. Based on the quality scores, images of the palm regions with the highest quality scores (a preset threshold number) are selected and combined into the input image.

7. The hand key point recognition method according to claim 1, characterized in that, The process of detecting human key points in the current image yields multiple sets of human key point coordinates, including: Using a zero-copy method, human key point detection is performed on the current image to obtain multiple sets of human key point coordinates.

8. A hand palm key point recognition device, characterized in that, include: The image acquisition module is used to acquire the current image that needs to be used for gesture recognition; The first detection module is used to detect human key points in the current image and obtain multiple sets of human key point coordinates. The region extraction module is used to extract multiple palm region images from the current image based on multiple sets of human body key point coordinates; The image binning module is used to combine the multiple palm region images into an input image that adapts to a multi-target palm key point model; The second detection module is used to detect palm key points in the input image through the multi-target palm key point model to obtain the palm key point coordinates corresponding to each palm region image. The coordinate mapping module is used to map the coordinates of key points of the palm corresponding to each palm region image to the current image to obtain the coordinates of key points of the target palm.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the palm key point recognition method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the palm key point recognition method according to any one of claims 1 to 7.

Images