Palm contour extraction method and device, control command generation method and device, computer device, and computer program

Abstract

This application relates to a palm contour extraction method, a control command generation method, an apparatus, a computer device, a storage medium, and a computer program product. The method includes a step of acquiring skeleton point information corresponding to each palm skeleton point in a target palm image, the skeleton point information including a skeleton point position and a skeleton point type (step S202), a step of matching each palm skeleton point based on the skeleton point type to obtain a plurality of palm skeleton point sets, the palm skeleton point sets having a corresponding geometric processing type (step S204), and a step of extracting geometrical values ​​corresponding to the palm skeleton point sets based on the skeleton point positions corresponding to each skeleton point in the same palm skeleton point set. The method includes a step of generating a palm contour auxiliary line that matches the geometric processing type to obtain a palm contour auxiliary line corresponding to each palm skeleton point set (step S206); a step of determining palm contour points from the palm contour auxiliary line corresponding to the palm skeleton point set based on a reference step size and skeleton point positions corresponding to the same palm skeleton point set to obtain contour point positions corresponding to the palm contour points (step S208); and a step of generating a palm contour corresponding to the target palm image based on each contour point position (step S210).

Description

[Technical field]

[0001] (Reference to Related Application) This application claims the benefit of priority to Chinese Patent Application No. 202210587233.0, entitled "Palm contour extraction method, control command generation method, device and computer device," filed with the State Intellectual Property Office of the People's Republic of China on May 27, 2022, the entire contents of which are incorporated herein by reference.

[0002] (Technical Field) The present application relates to the technical field of computers, and in particular to a palm contour extraction method, a control command generation method, an apparatus, a computer device, a storage medium, and a computer program product. [Background technology]

[0003] With the development of computer technology, palm extraction techniques, including techniques for recognizing palm contours from palm images, have emerged.

[0004] In the prior art, the pixel information of a palm image is usually subjected to various data processing to extract the palm outline in the palm image, for example, by performing binarization processing and edge detection on the palm image to extract the palm outline. However, methods that extract the palm outline based on the pixel information of the palm image are prone to interference from the background of the image, making it impossible to extract an accurate palm outline. Summary of the Invention [Problem to be solved by the invention]

[0005] SUMMARY OF THE DISCLOSURE The embodiments of the present application provide a palm contour extraction method, a control command generation method, an apparatus, a computer device, a computer-readable storage medium, and a computer program product. [Means for solving the problem]

[0006] A palm contour extraction method implemented by a computing device, comprising: acquiring skeleton point information corresponding to each palm skeleton point in the target palm image, the skeleton point information including a skeleton point position and a skeleton point type; matching each palm skeleton point based on the skeleton point type to obtain a plurality of palm skeleton point sets, each palm skeleton point set having a corresponding geometric processing type; generating palm contour assistant lines conforming to a geometric processing type corresponding to the palm skeleton point set based on skeleton point positions corresponding to each skeleton point in the same palm skeleton point set to obtain palm contour assistant lines corresponding to each of the palm skeleton point sets; determining palm contour points from the palm contour auxiliary line corresponding to the set of palm skeleton points based on a reference step size and skeleton point positions corresponding to the same set of palm skeleton points, thereby obtaining contour point positions corresponding to the palm contour points; generating a palm contour corresponding to the target palm image based on each contour point location.

[0007] a skeleton point information acquisition module configured to acquire skeleton point information corresponding to each palm skeleton point in a target palm image, the skeleton point information including a skeleton point position and a skeleton point type; a palm skeleton point matching module configured to match each palm skeleton point based on the skeleton point type to obtain a plurality of palm skeleton point sets, the palm skeleton point sets having corresponding geometric processing types; a palm contour extension determining module configured to generate a palm contour extension conforming to a geometric processing type corresponding to the palm skeleton point set based on skeleton point positions corresponding to each skeleton point in the same palm skeleton point set, thereby obtaining a palm contour extension corresponding to each of the palm skeleton point sets; a palm contour point determination module configured to determine palm contour points from palm contour auxiliary lines corresponding to the same palm skeleton point set based on a reference step size and skeleton point positions corresponding to the same palm skeleton point set, and obtain contour point positions corresponding to the palm contour points; a palm contour generation module configured to generate a palm contour corresponding to the target palm image based on each contour point position.

[0008] There is provided a computing device including a memory in which computer-readable instructions are stored and one or more processors, the computing device realizing the steps of the palm contour extraction method described above when the computer-readable instructions are executed by the one or more processors.

[0009] The present invention provides a computer-readable storage medium having computer-readable instructions stored thereon, the computer-readable instructions being executed by one or more processors to realize the steps of the palm contour extraction method described above.

[0010] The present invention provides a computer program product including computer-readable instructions, which, when executed by one or more processors, causes the steps of the palm contour extraction method to be realized.

[0011] 1. A method for generating control instructions executed by a computer device, comprising: displaying a palm contour determined based on the target palm image; generating a target control command based on the display information of the palm outline; The palm outline extraction process includes: acquiring skeleton point information corresponding to each palm skeleton point in the target palm image, the skeleton point information including a skeleton point position and a skeleton point type; matching each palm skeleton point based on the skeleton point type to obtain a plurality of palm skeleton point sets, each palm skeleton point set having a corresponding geometric processing type; generating palm contour assistant lines conforming to a geometric processing type corresponding to the palm skeleton point set based on skeleton point positions corresponding to each skeleton point in the same palm skeleton point set to obtain palm contour assistant lines corresponding to each of the palm skeleton point sets; determining palm contour points from the palm contour auxiliary line corresponding to the set of palm skeleton points based on a reference step size and skeleton point positions corresponding to the same set of palm skeleton points, thereby obtaining contour point positions corresponding to the palm contour points; generating a palm contour corresponding to the target palm image based on each contour point location.

[0012] a palm contour display module configured to display a palm contour determined based on the target palm image; and a control command generation module configured to generate a target control command based on the palm outline display information, The palm outline extraction process includes: acquiring skeleton point information corresponding to each palm skeleton point in the target palm image, the skeleton point information including a skeleton point position and a skeleton point type; matching each palm skeleton point based on the skeleton point type to obtain a plurality of palm skeleton point sets, each palm skeleton point set having a corresponding geometric processing type; generating palm contour assistant lines conforming to a geometric processing type corresponding to the palm skeleton point set based on skeleton point positions corresponding to each skeleton point in the same palm skeleton point set to obtain palm contour assistant lines corresponding to each of the palm skeleton point sets; determining palm contour points from the palm contour auxiliary line corresponding to the set of palm skeleton points based on a reference step size and skeleton point positions corresponding to the same set of palm skeleton points, thereby obtaining contour point positions corresponding to the palm contour points; and generating a palm contour corresponding to the target palm image based on each contour point position.

[0013] The present invention provides a computer device including a memory in which computer-readable instructions are stored and one or more processors, the computer device realizing the steps described in the above-mentioned control instruction generation method when the computer-readable instructions are executed by the one or more processors.

[0014] A computer-readable storage medium having computer-readable instructions stored thereon, the computer-readable instructions, when executed by one or more processors, realizing the steps described in the above control instruction generation method, is provided.

[0015] There is provided a computer program product comprising computer readable instructions, which, when executed by one or more processors, effectuate the steps of the above control instruction generation method.

[0016] The details of one or more embodiments of the application are set forth in the drawings and description below. Other features, objects, and advantages of the application will become apparent from the description, drawings, and claims. [Brief description of the drawings]

[0017] In order to more clearly describe the technical means in the embodiments of the present application, the following briefly describes the drawings that need to be used in the description of the embodiments, and it is obvious that the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings based on these drawings without creative labor.

[0018] [Figure 1] FIG. 2 is a diagram illustrating an application environment of a palm contour extraction method and a control command generation method according to an embodiment. [Diagram 2] FIG. 1 is a schematic flow chart of a palm contour extraction method according to an embodiment. [Diagram 3] FIG. 2 is a schematic diagram of palm skeleton points in one embodiment. [Figure 4] 13 is a schematic diagram of a movement direction corresponding to a palm outline extension line in one embodiment. FIG. [Diagram 5] FIG. 1 is a schematic diagram of a contour point sorting identifier and a palm contour in one embodiment. [Figure 6] FIG. 2 is a schematic flow chart diagram of a control command generation method according to one embodiment. [Figure 7A] FIG. 13 is a schematic diagram of a palm skeleton point recognition result according to an embodiment; [Figure 7B] 10 is a schematic diagram of a palm contour point recognition result according to an embodiment; [Figure 8A] FIG. 13 is a schematic diagram illustrating calculating coordinates of the outer contour of a palm in one embodiment; [Figure 8B] FIG. 13 is a schematic diagram illustrating calculating interdigital coordinates in one embodiment. [Figure 8C] 11 is a schematic diagram showing calculation of coordinates of edges of a finger contour in one embodiment; FIG. [Figure 8D] FIG. 2 is a schematic diagram of a palm profile in one embodiment. [Figure 9] FIG. 1 is a schematic diagram of palm authentication payment in one embodiment. [Figure 10] 1 is a block diagram showing the configuration of a palm contour extraction device according to an embodiment. [Figure 11] FIG. 13 is a configuration block diagram of a palm contour extraction device according to another embodiment. [Figure 12] 1 is a block diagram illustrating a configuration of a control instruction generating device according to an embodiment. [Figure 13] FIG. 2 is a diagram illustrating an internal configuration of a computer device according to an embodiment. [Figure 14] FIG. 11 is a diagram illustrating an internal configuration of a computer device according to another embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] In order to make the objectives, technical means and advantages of the present application clearer, the present application will be described in more detail below with reference to the drawings and embodiments. It is understood that the specific embodiments described herein are only for the purpose of illustrating the present application, and are not intended to limit the present application.

[0020] The palm outline extraction method and the control command generation method according to the embodiment of the present application can be applied to an application environment as shown in FIG. 1. Here, the terminal 102 communicates with the server 104 via a network. The data storage system may store data that the server 104 needs to process. The data storage system may be integrated into the server 104 or may be located in the cloud or other servers. The terminal 102 may be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices, and the IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. The portable wearable devices may be smart watches, smart bands, head-mounted devices, etc. The server 104 may be realized by an independent server, a server cluster consisting of multiple servers, or a cloud server.

[0021] Both the terminal and the server can independently execute the palm contour extraction method and the control command generation method according to the embodiment of the present application.

[0022] For example, the terminal obtains skeleton point information corresponding to each of the palm skeleton points in the target palm image, the skeleton point information including a skeleton point position and a skeleton point type, the terminal matches each palm skeleton point based on the skeleton point type to obtain a plurality of palm skeleton point sets, the palm skeleton point sets having corresponding geometric processing types, the terminal generates a palm contour auxiliary line matching the geometric processing type corresponding to the palm skeleton point set based on the skeleton point positions corresponding to each skeleton point in the same palm skeleton point set to obtain a palm contour auxiliary line corresponding to each of the palm skeleton point sets, determines a palm contour point from the palm contour auxiliary line corresponding to the palm skeleton point set based on the reference step size and skeleton point positions corresponding to the same palm skeleton point set, and obtains a contour point position corresponding to the palm contour point, the terminal generates a palm contour corresponding to the target palm image based on each contour point position.

[0023] The terminal displays a palm outline determined based on the target palm image, and generates a target control command based on the palm outline display information.

[0024] The terminal and the server may cooperate to execute the palm contour extraction method and the control command generation method according to the embodiment of the present application.

[0025] For example, the server obtains from the terminal skeleton point information corresponding to each of the palm skeleton points in the target palm image, the skeleton point information including a skeleton point position and a skeleton point type, and the server matches each palm skeleton point based on the skeleton point type to obtain a plurality of palm skeleton point sets, each of which has a corresponding geometric processing type. The server generates a palm contour auxiliary line that matches the geometric processing type corresponding to the palm skeleton point set based on the skeleton point position corresponding to each skeleton point in the same palm skeleton point set to obtain a palm contour auxiliary line corresponding to each of the palm skeleton point sets, and determines a palm contour point from the palm contour auxiliary line corresponding to the palm skeleton point set based on the reference step size and skeleton point position corresponding to the same palm skeleton point set to obtain a contour point position corresponding to the palm contour point. The server generates a palm contour corresponding to the target palm image based on each contour point position.

[0026] The terminal acquires the palm outline determined based on the target palm image from the server, displays the palm outline, and generates a target control command based on the display information of the palm outline.

[0027] The above palm contour extraction method does not require complex data processing of pixel information of a palm image, and by performing geometric processing based on skeleton point information corresponding to each palm skeleton point in the palm image, it is possible to easily determine contour point positions corresponding to palm contour points from the palm image, and further generate a palm contour based on the contour point positions corresponding to each palm contour point. The positions and types of palm skeleton points are not easily interfered with by the background of the image, and by extracting the palm contour based on the positions and types of palm skeleton points in the palm image, it is possible to effectively improve the accuracy of palm contour extraction.

[0028] In the above control command generation method, the palm contour is generated based on the contour point positions corresponding to the palm contour points by performing geometric processing based on the skeleton point information corresponding to each of the palm skeleton points in the palm image, and the palm contour is generated based on the contour point positions corresponding to each of the palm contour points. The palm contour is extracted not only without complex data processing of the pixel information of the palm image, but also with high accuracy because it is not easily interfered with by the background of the image. Extracting an accurate palm contour is also useful for ensuring the accuracy of the display, and further for improving the accuracy of the generation of the target control command. Automatically generating the target control command based on the display information of the palm contour can also improve the efficiency of generating the target control command.

[0029] In one embodiment, a palm contour extraction method is provided, as shown in Fig. 2, and the application of the method to a computer device is described as an example, which may be the terminal 102 or the server 104 in the above Fig. 1. As shown in Fig. 2, the palm contour extraction method includes the following steps S202 to S210.

[0030] In step S202, skeleton point information corresponding to each palm skeleton point in the target palm image is obtained, and the skeleton point information includes a skeleton point position and a skeleton point type.

[0031] Here, the target palm image is a palm image from which a palm contour is to be extracted. The palm image is an image including a palm. The palm image may include only the palm, for example, the palm image is an image obtained by photographing the palm area. The palm image may include more body parts, for example, the palm image is an image obtained by photographing the upper body of a subject who raises his palm. The palm image may be one photographed in real time, for example, a photograph of the palm taken in real time may be the palm image, or a video frame including the palm in a video taken in real time may be the palm image. The palm image may be one obtained from data photographed in advance, for example, a photograph of the palm stored in an album may be the palm image, or a video frame including the palm in a video edited in advance may be the palm image. The palm image may also be one photographed of a real palm or a virtual palm.

[0032] Palm skeleton points are skeleton points on the palm of the hand. A palm includes a plurality of palm skeleton points. By performing palm skeleton point recognition on a palm image, skeleton point information corresponding to each palm skeleton point in the palm image can be obtained. The skeleton point information includes skeleton point positions and skeleton point types. The skeleton point positions are used to identify the positions of the palm skeleton points in the palm image, and the skeleton point types are used to identify the types of the palm skeleton points.

[0033] Specifically, the computer device may obtain a target palm image locally or from another device, and perform palm skeletal point recognition on the target palm image in real time to obtain skeleton point information corresponding to each palm skeletal point in the target palm image, or may directly obtain skeleton point information corresponding to each palm skeletal point in the target palm image that has been recognized in advance locally or from another device. Furthermore, the computer device may perform palm contour recognition on the target palm image based on the skeleton point information corresponding to each palm skeletal point, to obtain the palm contour in the target palm image.

[0034] In step S204, matching is performed for each palm skeleton point according to the skeleton point type to obtain a plurality of palm skeleton point sets, each of which has a corresponding geometric processing type.

[0035] Here, the palm skeleton point set includes at least two palm skeleton points. Each palm skeleton point set has a corresponding geometric processing type. The geometric processing type is for determining a geometric processing method corresponding to the palm skeleton point set. The geometric processing method corresponding to the palm skeleton point set is used to perform geometric processing on the palm skeleton point set to obtain a palm outline point corresponding to the palm skeleton point set. Note that the same palm skeleton point set can correspond to at least one geometric processing type.

[0036] Specifically, the computer device can match each palm skeleton point according to the skeleton point type, and the successfully matched palm skeleton points form a palm skeleton point set, and finally obtain a plurality of palm skeleton point sets, each of which has a corresponding geometric processing type.

[0037] In one embodiment, the computer device can obtain skeleton point type matching information that records the matching relationship between skeleton point types, and the skeleton point type matching information can quickly match each palm skeleton point based on the skeleton point type to quickly obtain each palm skeleton point set. In one embodiment, the skeleton point type matching information can further include a geometric processing type corresponding to each matching relationship, so that when determining the palm skeleton point set based on the matching relationship, the geometric processing type corresponding to the palm skeleton point set can be synchronously determined.

[0038] In one embodiment, a geometric processing type corresponding to the palm skeleton point set can be determined based on the skeleton point type corresponding to each palm skeleton point in the palm skeleton point set. For example, at least one target palm skeleton point type corresponding to each of the geometric processing types is preset, and when the palm skeleton point set includes the target palm skeleton point type, the geometric processing type corresponding to the target palm skeleton point type is set as the geometric processing type corresponding to the palm skeleton point set.

[0039] In one embodiment, a geometric processing type corresponding to the palm skeleton point set can be determined based on a skeleton point position corresponding to each palm skeleton point in the palm skeleton point set. For example, the palm portion in the palm image is divided into a number of palm regions, each palm region has a corresponding geometric processing type, and the geometric processing type corresponding to the palm region in which the palm skeleton point set falls is set as the geometric processing type corresponding to the palm skeleton point set.

[0040] In one embodiment, there is a connection relationship between the palm skeleton points. When recognizing the palm skeleton points, the connection relationship between the palm skeleton points and the skeleton point information of the palm skeleton points can be recognized synchronously. The computer device first performs initial matching for each palm skeleton point based on the connection relationship to obtain a plurality of first palm skeleton point sets, and then performs complementary matching for each palm skeleton point based on the skeleton point type to obtain a plurality of second palm skeleton point sets, and finally obtains a palm skeleton point set based on each of the first palm skeleton point sets and the second palm skeleton point set.

[0041] For example, as shown in Fig. 3, the circle points in Fig. 3 indicate palm skeleton points, including a total of 21 palm skeleton points, and the connecting lines between the circle points in Fig. 3 indicate the connection relationships between the palm skeleton points. The computer device can configure a first palm skeleton point set with adjacent palm skeleton points having a connection relationship, for example, circle point 0 and circle point 1 configure one first palm skeleton point set, and circle point 1 and circle point 2 configure one first palm skeleton point set. The computer device can configure a second palm skeleton point set with palm skeleton points corresponding to a predetermined skeleton point type, for example, the skeleton point type corresponding to circle point 13 and circle point 17 is a predetermined skeleton point type, and circle point 13 and circle point 17 configure one second palm skeleton point set.

[0042] In step S206, a palm contour auxiliary line that matches the geometric processing type corresponding to the palm skeleton point set is generated based on the skeleton point positions corresponding to each skeleton point in the same palm skeleton point set, thereby obtaining a palm contour auxiliary line corresponding to each of the palm skeleton point sets.

[0043] Here, the palm contour auxiliary lines are geometric lines and are used to determine the palm contour points.

[0044] Specifically, after determining each palm skeleton point set, the computer device can generate a palm contour auxiliary line that matches the geometric processing type corresponding to the palm skeleton point set for any palm skeleton point set based on the skeleton point positions corresponding to each skeleton point in the palm skeleton point set. The computer device can determine a geometric processing method and a geometric processing algorithm based on the geometric processing type corresponding to the palm skeleton point set, and perform data processing on the skeleton point positions corresponding to each skeleton point in the palm skeleton point set using the geometric processing algorithm to generate a palm contour auxiliary line corresponding to the palm skeleton point set. Finally, the computer device can obtain a palm contour auxiliary line corresponding to each palm skeleton point set.

[0045] Note that one palm skeleton point set may correspond to at least one palm outline extension line.

[0046] In step S208, based on the reference step size and skeleton point positions corresponding to the same palm skeleton point set, palm contour points are determined from the palm contour auxiliary line corresponding to the palm skeleton point set, and contour point positions corresponding to the palm contour points are obtained.

[0047] Here, the reference step size is the movement step size on the palm contour auxiliary line. Each palm skeleton point set has a corresponding reference step size. The contour point positions are used to identify the positions of the palm contour points.

[0048] Specifically, for any palm skeleton point set, the computer device can obtain a contour point position corresponding to the palm contour point by determining a palm contour point from the palm contour auxiliary line corresponding to the palm skeleton point set based on the skeleton point position corresponding to each palm skeleton point in the palm skeleton point set and the reference step size corresponding to the palm skeleton point set. The computer device can determine a movement start point from the palm contour auxiliary line corresponding to the palm skeleton point set based on the skeleton point position corresponding to each palm skeleton point in the palm skeleton point set, move the palm contour auxiliary line corresponding to the palm skeleton point set from the movement start point by the reference step size, and set the movement end point moved by the reference step size as the palm contour point. Finally, the computer device can obtain a plurality of palm contour points and contour point positions corresponding to each of the palm contour points.

[0049] In one embodiment, a palm image with a marked palm outline is acquired as a reference palm image, and a common reference step size can be determined based on the reference palm image. Specifically, a palm outline auxiliary line corresponding to the reference palm image is determined based on the reference palm image, and the intersection point between the palm outline auxiliary line and the marked palm outline is set as a reference outline point that is an accurate palm outline point, and an accurate movement step size is determined based on the distance between the movement start point and the reference outline point, and the accurate movement step size is set as the reference step size. The computer device acquires multiple reference palm images, and collects statistics on the same kind of reference step sizes determined based on different reference palm images to obtain a common reference step size, for example, the average value of the reference step sizes can be calculated to be the common reference step size. Then, when processing a palm image with an unknown palm contour, the common reference step size can be directly obtained to determine the palm contour points in the palm image, or the common reference step size can be fine-tuned based on the personalized information of the palm image to obtain a target reference step size, and the palm contour points in the palm image can be determined based on the target reference step size. The personalized information of the palm image includes at least one kind of data including palm size information, palm depth information, etc.

[0050] In one embodiment, the reference step size increases as the palm size information corresponding to the target palm image increases. The palm size information is for indicating the size of the palm in the target palm image. For a larger palm, the palm contour assistant line needs to move by a larger reference step size before reaching the correct palm contour point. In one embodiment, the palm size information includes at least one of palm length information or palm width information.

[0051] In one embodiment, the reference step size increases as the palm depth information corresponding to the target palm image decreases. The palm depth information is for indicating the distance between the palm and the image collection device, and the closer the distance, the smaller the palm depth information, and the farther the distance, the larger the palm depth information. Note that the smaller the palm depth information corresponding to the target palm image, the closer the distance between the palm and the image collection device, and the larger the palm area displayed in the target palm image. For a larger palm area, the palm contour auxiliary line needs to move by a larger reference step size before reaching the correct palm contour point. In one embodiment, the target palm image may be a depth map, and each pixel value of the depth map indicates the distance between a point in the shooting scene and the image collection device.

[0052] In step S210, a palm contour corresponding to the target palm image is generated based on each contour point position.

[0053] Specifically, after determining the contour point positions corresponding to each of the palm contour points, the computing device can connect the contour point positions to obtain a palm contour corresponding to the target palm image.

[0054] In the above palm contour extraction method, skeleton point information corresponding to each of the palm skeleton points in the target palm image is obtained, the skeleton point information including a skeleton point position and a skeleton point type, and each palm skeleton point is matched based on the skeleton point type to obtain a plurality of palm skeleton point sets, each of which has a corresponding geometric processing type, and a palm contour auxiliary line matching the geometric processing type corresponding to the palm skeleton point set is generated based on the skeleton point position corresponding to each skeleton point in the palm skeleton point set to obtain the palm contour auxiliary line corresponding to each of the palm skeleton point sets, and a palm contour point is determined from the palm contour auxiliary line corresponding to the palm skeleton point set based on a reference step size and skeleton point position corresponding to the same palm skeleton point set to obtain contour point positions corresponding to the palm contour points, and a palm contour corresponding to the target palm image is generated based on each contour point position. In this way, there is no need to perform complex data processing on the pixel information of the palm image, and by performing geometric processing based on skeleton point information corresponding to each of the palm skeleton points in the palm image, it is possible to easily determine contour point positions corresponding to the palm contour points from the palm image, and further to generate a palm contour based on the contour point positions corresponding to each of the palm contour points. The positions and types of palm skeleton points are not easily interfered with by the background of the image, and by extracting the palm contour based on the positions and types of palm skeleton points in the palm image, it is possible to effectively improve the accuracy of palm contour extraction.

[0055] In one embodiment, the skeleton point type is determined by finger parameters and intra-finger joint parameters. The step of matching each palm skeleton point based on the skeleton point type to obtain a plurality of palm skeleton point sets includes:

[0056] The method includes the steps of: establishing a matching relationship with adjacent palm skeleton points whose intra-finger joint parameters are predetermined joint parameters to obtain a palm skeleton point set corresponding to a first geometric processing type; establishing a matching relationship with adjacent palm skeleton points corresponding to the same finger parameters to obtain a palm skeleton point set corresponding to a second geometric processing type; and determining a palm skeleton point corresponding to the predetermined skeleton point type as a predetermined skeleton point, and establishing a matching relationship with the predetermined skeleton point and a palm skeleton point closest to the predetermined skeleton point to obtain a palm skeleton point set corresponding to a third geometric processing type.

[0057] Here, the skeleton point type is determined by finger parameters and intra-finger joint parameters. The finger parameters indicate the finger to which the palm skeleton point belongs. The finger parameters make it possible to distinguish between palm skeleton points located on different fingers. The intra-finger joint parameters indicate the joint within the finger to which the palm skeleton point belongs. The intra-finger joint parameters make it possible to distinguish between different palm skeleton points located on the same finger.

[0058] The predetermined joint parameters are specific intra-finger joint parameters that are preset. The predetermined skeleton point type is a specific skeleton point type that is preset. The predetermined joint parameters and the predetermined skeleton point type can be set according to actual needs. The adjacent palm skeleton points refer to two palm skeleton points whose positions are adjacent to each other.

[0059] Specifically, when matching palm skeleton points, the computer device obtains, from each palm skeleton point, an adjacent palm skeleton point whose intra-finger joint parameters are the predetermined joint parameters, and establishes a matching relationship with the adjacent palm skeleton point whose intra-finger joint parameters are the predetermined joint parameters, thereby obtaining a set of palm skeleton points corresponding to the first geometric processing type. For example, as shown in FIG. 3, the intra-finger joint parameters corresponding to the circled point 13, the circled point 17, the circled point 9, and the circled point 5 are metacarpophalangeal joints, that is, the circled point 13, the circled point 17, the circled point 9, and the circled point 5 are metacarpophalangeal joints corresponding to different fingers. When a certain joint parameter is a metacarpophalangeal joint, a matching relationship between the adjacent circled point 13 and the circled point 17 is established to obtain a palm skeleton point set A1 corresponding to the first geometric processing type, a matching relationship between the adjacent circled point 13 and the circled point 9 is established to obtain a palm skeleton point set A2 corresponding to the first geometric processing type, and a matching relationship between the adjacent circled point 9 and the circled point 5 is established to obtain a palm skeleton point set A3 corresponding to the first geometric processing type.

[0060] When matching palm skeleton points, the computer device obtains adjacent palm skeleton points with the same finger parameters from each palm skeleton point, and establishes a matching relationship with the adjacent palm skeleton points corresponding to the same finger parameters to obtain each palm skeleton point set corresponding to the second geometric processing type. For example, as shown in Figure 3, the finger parameters corresponding to the circle point 1, circle point 2, circle point 3, and circle point 4 are the thumb, that is, the circle point 1, circle point 2, circle point 3, and circle point 4 are different joints in the same finger, and establishes a matching relationship between the adjacent circle point 1 and circle point 2 to obtain a palm skeleton point set B1 corresponding to the second geometric processing type, establishes a matching relationship between the adjacent circle point 2 and circle point 3 to obtain a palm skeleton point set B2 corresponding to the second geometric processing type, and establishes a matching relationship between the adjacent circle point 3 and circle point 4 to obtain a palm skeleton point set B3 corresponding to the second geometric processing type.

[0061] When matching palm skeleton points, the computer device obtains a palm skeleton point corresponding to a predetermined skeleton point type from each palm skeleton point as the predetermined skeleton point, obtains the palm skeleton point closest to the predetermined skeleton point, and establishes a matching relationship between the predetermined skeleton point and the palm skeleton point closest to the predetermined skeleton point, thereby obtaining each palm skeleton point set corresponding to the third geometric processing type. For example, as shown in Figure 3, if the skeleton point type corresponding to circle point 0 is a predetermined skeleton point type, circle point 0 is the predetermined skeleton point, and circle point 1 is the palm skeleton point closest to circle point 0, and a matching relationship is established between circle point 0 and circle point 1 to obtain palm skeleton point set C corresponding to the third geometric processing type.

[0062] In the above embodiment, a matching relationship is established between adjacent palm skeleton points whose intra-finger joint parameters are predetermined joint parameters to obtain each palm skeleton point set corresponding to a first geometric processing type, a matching relationship is established between adjacent palm skeleton points whose intra-finger joint parameters are predetermined joint parameters to obtain each palm skeleton point set corresponding to a second geometric processing type, a palm skeleton point corresponding to a predetermined skeleton point type is set as a predetermined skeleton point, and a matching relationship is established between the predetermined skeleton point and the palm skeleton point closest to the predetermined skeleton point to obtain a palm skeleton point set corresponding to a third geometric processing type. The skeleton point type is determined by the finger parameters and intra-finger joint parameters, and the matching relationship can be quickly established according to the finger parameters, intra-finger joint parameters and skeleton point type, and each palm skeleton point set can be quickly obtained.

[0063] In one embodiment, a first geometric processing type is used to determine palm contour points on the inter-finger areas and the outer contour of the palm, a second geometric processing type is used to determine palm contour points on the edges of the finger contour, and a third geometric processing type is used to determine palm contour points on the edges of the finger contour and the outer contour of the palm.

[0064] In one embodiment, the step of generating palm contour assistant lines conforming to a geometric processing type corresponding to the palm skeleton point set based on skeleton point positions corresponding to each skeleton point in the same palm skeleton point set to obtain palm contour assistant lines corresponding to each of the palm skeleton point sets includes:

[0065] The method includes the steps of: determining a current auxiliary line type based on a geometric processing type corresponding to the current palm skeleton point set; generating a current skeleton point line segment corresponding to the current palm skeleton point set based on skeleton point positions corresponding to each palm skeleton point in the current palm skeleton point set; and performing geometric processing on the current skeleton point line segment based on the current auxiliary line type to obtain a palm contour auxiliary line corresponding to the current palm skeleton point set and belonging to the current auxiliary line type.

[0066] Here, the auxiliary line type is used to determine the type of the auxiliary line. The current auxiliary line type is the auxiliary line type of the palm contour auxiliary line corresponding to the current palm skeleton point set. The skeleton point line segment is a line segment made of palm skeleton points. The current skeleton point line segment is a skeleton point line segment corresponding to the current palm skeleton point set. The geometric processing is used to generate a palm contour auxiliary line of a specific type based on the skeleton point line segment.

[0067] The current palm skeleton point set may be any one of the palm skeleton point sets.

[0068] Specifically, when generating a palm contour auxiliary line, the computer device may determine an auxiliary line type corresponding to the palm contour auxiliary line and then generate the corresponding palm contour auxiliary line. The computer device may determine a current auxiliary line type based on a geometric processing type corresponding to the current palm skeleton point set, and each geometric processing type has a corresponding auxiliary line type. The computer device may generate a current skeleton point line segment corresponding to the current palm skeleton point set based on a skeleton point position corresponding to each palm skeleton point in the current palm skeleton point set, for example, two points form one line segment, and two palm skeleton points are obtained from the current palm skeleton point set, and the skeleton point positions corresponding to each of the two palm skeleton points are connected to generate a current skeleton point line segment. The present application does not limit the order of determining the current auxiliary line type and generating the current skeleton point line segment. Finally, the computer device may perform a geometric processing on the current skeleton point line segment based on the current auxiliary line type to obtain a palm contour auxiliary line corresponding to the current auxiliary line type. For example, if the current auxiliary line type is an extension line, the extension line of the current skeleton point line segment can be set as the palm contour auxiliary line.

[0069] In the above embodiment, a current auxiliary line type is determined based on a geometric processing type corresponding to the current palm skeleton point set, a current skeleton point line segment corresponding to the current palm skeleton point set is generated based on skeleton point positions corresponding to each palm skeleton point in the current palm skeleton point set, and geometric processing is performed on the current skeleton point line segment based on the current auxiliary line type to obtain a palm contour auxiliary line corresponding to the current palm skeleton point set and belonging to the current auxiliary line type. By determining the auxiliary line type and then performing geometric processing on the skeleton point line segment generated based on the palm skeleton points based on the auxiliary line type, the corresponding palm contour auxiliary line can be generated quickly.

[0070] In one embodiment, the geometric processing type includes any one of a first geometric processing type, a second geometric processing type, and a third geometric processing type, the auxiliary line type corresponding to the first geometric processing type includes at least one of a perpendicular bisector type or an extension line type, the auxiliary line type corresponding to the second geometric processing type includes a normal line type, and the auxiliary line type corresponding to the third geometric processing type includes at least one of a normal line type or an extension line type.

[0071] Here, the geometric processing type includes any one of the first geometric processing type, the second geometric processing type, and the third geometric processing type. The auxiliary line type corresponding to the first geometric processing type includes at least one of the perpendicular bisector type and the extension line type. The palm outline auxiliary line corresponding to the perpendicular bisector type is a perpendicular bisector. The perpendicular bisector is a straight line that passes through the midpoint of a line segment and is perpendicular to the line segment. The palm outline auxiliary line corresponding to the extension line type is an extension line, which is a straight line that is extended from an end point of a line segment to a further point. The auxiliary line type corresponding to the second geometric processing type includes a normal line type. The palm outline auxiliary line corresponding to the normal line type is a normal line. The normal line of a line segment may also be called the perpendicular line of the line segment, and the line segment and the corresponding normal line are perpendicular to each other. The auxiliary line type corresponding to the third geometric processing type includes at least one of the normal line type and the extension line type.

[0072] In the above embodiment, different geometric processing types correspond to different auxiliary line types, and the same geometric processing type may correspond to at least one auxiliary line type, which can effectively increase the number of palm contour points and further improve the extraction accuracy of the palm contour.

[0073] In one embodiment, the step of performing a geometric process on the current skeleton point segment based on the current auxiliary line type to obtain a palm contour auxiliary line corresponding to the current palm skeleton point set and belonging to the current auxiliary line type includes:

[0074] The method includes the steps of: if the current auxiliary line type is the perpendicular bisector type, setting the perpendicular bisector corresponding to the current skeleton point line segment as the palm contour auxiliary line; if the current auxiliary line type is the extension line type, determining a target skeleton point from each palm skeleton point corresponding to the current palm skeleton point set, and setting an extension line corresponding to the current skeleton point line segment and having the target skeleton point as the palm contour auxiliary line; and if the current auxiliary line type is the normal type, determining a target skeleton point from each palm skeleton point corresponding to the current palm skeleton point set, and setting a normal to the current skeleton point line segment at the target skeleton point as the palm contour auxiliary line.

[0075] Specifically, if the current auxiliary line type is a perpendicular bisector type, the computer device can set the perpendicular bisector corresponding to the current skeleton point line segment as the palm contour auxiliary line.

[0076] If the current auxiliary line type is an extension line type, the computer device may select a palm skeleton point from each palm skeleton point corresponding to the current palm skeleton point set as a target skeleton point, and set an extension line corresponding to the current skeleton point line segment and starting from the target skeleton point as a palm contour auxiliary line. When determining the target skeleton point, the computer device may set a palm skeleton point that is closer to the center of the palm as the target skeleton point, the computer device may set a palm skeleton point of a specified skeleton point type in the current palm skeleton point set as the target skeleton point, or the computer device may set each palm skeleton point in the current palm skeleton point set as the target skeleton point.

[0077] If the current auxiliary line type is a normal type, the computer device may select a palm skeleton point from each palm skeleton point corresponding to the current palm skeleton point set as a target skeleton point, and set the normal of the current skeleton point line segment at the target skeleton point as the palm contour auxiliary line. When determining the target skeleton point, the computer device may further generate multiple palm contour auxiliary lines by using each palm skeleton point as a target skeleton point, and may further generate one palm contour auxiliary line by selecting one palm skeleton point from among them as the target skeleton point. When selecting the palm skeleton point, it may be selected randomly, or a palm skeleton point closer to the fingertip may be selected.

[0078] In the above embodiment, if the current auxiliary line type is the perpendicular bisector type, the perpendicular bisector corresponding to the current skeleton point line segment is taken as the palm contour auxiliary line, if the current auxiliary line type is the extension line type, a target skeleton point is determined from each palm skeleton point corresponding to the current palm skeleton point set, and an extension line corresponding to the current skeleton point line segment and starting from the target skeleton point is taken as the palm contour auxiliary line, if the current auxiliary line type is the normal type, a target skeleton point is determined from each palm skeleton point corresponding to the current palm skeleton point set, and a normal line of the current skeleton point line segment at the target skeleton point is taken as the palm contour auxiliary line. Different geometric methods can be used for different auxiliary line types to quickly generate the corresponding palm contour auxiliary lines.

[0079] In one embodiment, the step of determining palm contour points from palm contour auxiliary lines corresponding to the same palm skeleton point set based on a reference step size and skeleton point positions corresponding to the same palm skeleton point set to obtain contour point positions corresponding to the palm contour points includes:

[0080] The method includes the steps of: generating a current skeleton point line segment corresponding to the current palm skeleton point set based on skeleton point positions corresponding to each skeleton point in the current palm skeleton point set; determining a reference point from the current skeleton point line segment; and moving, starting from the reference point, on the current palm contour auxiliary line corresponding to the current palm skeleton point set by a reference step size corresponding to the current palm skeleton point set along a moving direction corresponding to the current palm contour auxiliary line, to obtain the current palm contour points and contour point positions corresponding to the current palm contour points.

[0081] Here, the current palm contour extension line is a palm contour extension line that corresponds to the current palm skeleton point set.

[0082] Specifically, when determining the palm contour point, the computer device can determine a reference point from the skeleton point line segment, and move the palm contour auxiliary line by the step size along the movement direction from the reference point as the start point, and set the movement end point as the palm contour point. First, the computer device can generate a current skeleton point line segment corresponding to the current palm skeleton point set based on the skeleton point positions corresponding to each skeleton point in the current palm skeleton point set. Next, the computer device can determine a reference point from the current skeleton point line segment, and for example, any of the end points of the current skeleton point line segment may be set as the reference point, or the intersection point between the current skeleton point line segment and the current palm contour auxiliary line may be set as the reference point. Furthermore, the computer device determines the current palm contour point and the contour point position corresponding to the current palm contour point by moving the current palm contour auxiliary line by the reference step size corresponding to the current palm skeleton point set along the movement direction corresponding to the current palm contour auxiliary line from the reference point as the start point. Different types of palm outline auxiliary lines have corresponding movement directions, and palm outline auxiliary lines of the same type correspond to at least one movement direction.

[0083] In one embodiment, the reference step size increases as the length of the line segment corresponding to the current skeleton point line segment increases. The length of the line segment corresponding to the current skeleton point line segment can reflect the personalized information of the palm from another aspect. The longer the length of the line segment, the larger the palm. For a larger palm, the palm contour assistant line needs to move by a larger reference step size before reaching the correct palm contour point.

[0084] In the above embodiment, a current skeleton point line segment corresponding to the current palm skeleton point set is generated based on the skeleton point positions corresponding to each skeleton point in the current palm skeleton point set, a reference point is determined from the current skeleton point line segment, and the current palm contour auxiliary line corresponding to the current palm skeleton point set is moved from the reference point as a starting point along the movement direction corresponding to the current palm contour auxiliary line by a reference step size corresponding to the current palm skeleton point set to obtain the current palm contour points and contour point positions corresponding to the current palm contour points. In this way, accurate palm contour points can be quickly obtained by moving the palm contour auxiliary line from the reference point determined based on the palm skeleton points as a starting point along the movement direction corresponding to the palm contour auxiliary line by the corresponding reference step size.

[0085] In one embodiment, the current palm contour auxiliary line includes at least one of a perpendicular bisector, an extension line, or a normal line, and the movement direction corresponding to the perpendicular bisector is the direction away from the center of the palm, the movement direction corresponding to the extension line is the extension direction of the extension line, and the movement direction corresponding to the normal line is the perpendicular direction of the current skeleton point line segment.

[0086] Here, the current palm outline extension line includes at least one of a perpendicular bisector, an extension line, or a normal line.

[0087] The movement direction corresponding to the perpendicular bisector is the direction away from the center of the palm, and the direction away from the center of the palm is the direction away from the center of the palm, that is, on the perpendicular bisector type palm contour auxiliary line, it is necessary to determine the palm contour point by moving the reference step size in the direction away from the center of the palm. For example, as shown in Figure 4(a), the dotted line in (a) indicates the perpendicular bisector, the arrow indicates the direction away from the center of the palm, and the circular point 15' indicates the palm contour point determined by moving the reference step size along the perpendicular bisector in the direction away from the center of the palm.

[0088] The movement direction corresponding to the extension line is the extension direction of the extension line, that is, in the extension line type palm contour auxiliary line, it is necessary to determine the palm contour point by moving the extension line by the standard step size in the extension direction of the extension line. For example, as shown in Figure 4(b), the dotted line in (b) indicates the extension line, the arrow indicates the extension direction of the extension line, and the circle 36' indicates the palm contour point determined by moving the extension line by the standard step size along the extension direction of the extension line.

[0089] The movement direction corresponding to the normal is the vertical direction of the current skeleton point segment, that is, in the normal type palm contour auxiliary line, it is necessary to determine the palm contour point by moving the reference step size in the vertical direction of the skeleton point segment. For example, as shown in Fig. 4(c), the dotted line in (c) indicates the normal line, the arrow indicates the vertical direction of the normal line, and the round points 3' and 6' indicate the palm contour points determined by moving the reference step size along the vertical direction of the skeleton point segment in the normal line.

[0090] In the above embodiment, the movement direction corresponding to the perpendicular bisector is the direction away from the center of the palm, the movement direction corresponding to the extension line is the extension direction of the extension line, and the movement direction corresponding to the normal line is the perpendicular direction of the current skeleton point line segment. Different types of palm contour auxiliary lines correspond to different movement directions.

[0091] In one embodiment, each palm skeleton point has a corresponding skeleton point identifier. The step of generating a palm contour corresponding to the target palm image based on each contour point location includes:

[0092] The method includes the steps of: mapping skeleton point identifiers corresponding to palm skeleton points based on predetermined mapping information to obtain contour point sort identifiers corresponding to each palm contour point, where the predetermined mapping information includes identifier mapping relationships between the palm skeleton points in the palm skeleton point set and the palm contour points generated based on the palm skeleton point set; and ordering and connecting contour point positions corresponding to each palm contour point according to the contour point sort identifiers to obtain a palm contour corresponding to the target palm image.

[0093] Here, the skeleton point identifier is used to uniquely identify the palm skeleton points, and the contour point sort identifier is used to identify the sort order of the palm contour points.

[0094] The predetermined mapping information records an identifier mapping relationship between palm skeleton points in the palm skeleton point set and palm outline points generated based on the palm skeleton point set. For example, a palm outline point generated based on palm skeleton point 0 and palm skeleton point 1 is marked as 1', and palm outline points generated based on palm skeleton point 1 and palm skeleton point 2 are marked as 2' and 7'.

[0095] Specifically, when generating a palm contour based on the contour point positions, the palm contour points can be ordered and connected to obtain a palm contour. When determining sorting information for the palm contour points, the computer device obtains predetermined mapping information, maps skeleton point identifiers corresponding to the palm skeleton points, generates contour point identifiers corresponding to the palm contour points generated based on the palm skeleton points, and finally obtains contour point sorting identifiers corresponding to each of the palm contour points. Furthermore, the computer device can order and connect the contour point positions corresponding to each of the palm contour points according to the contour point sorting identifiers to obtain an accurate palm contour.

[0096] To explain with an example, as shown in Fig. 5, black circles in Fig. 5 indicate palm skeleton points, and white circles indicate palm contour points. Fig. 5(a) shows a target palm image, and palm contour points 1' to 38' can be obtained based on palm skeleton points 0 to 20. Fig. 5(b) shows a palm contour, and the palm contour can be obtained by connecting palm contour points 1' to 38' in order.

[0097] In one embodiment, the contour point positions may be connected smoothly or with straight lines.

[0098] In the above embodiment, the skeleton point identifiers corresponding to the palm skeleton points are mapped based on the predetermined mapping information to obtain the contour point sort identifiers corresponding to each palm contour point, the predetermined mapping information includes the identifier mapping relationship between the palm skeleton points in the palm skeleton point set and the palm contour points generated based on the palm skeleton point set, and the contour point positions corresponding to each palm contour point are ordered and connected according to the contour point sort identifiers to obtain the palm contour. In this way, the contour point sort identifiers corresponding to the palm contour points can be quickly determined by the predetermined mapping information, so that the palm contour points can be ordered and connected based on the contour point sort identifiers to generate an accurate palm contour.

[0099] In one embodiment, the step of acquiring skeleton point information corresponding to each of the palm skeleton points in the target palm image includes:

[0100] The method includes inputting the target palm image into a target palm skeleton point recognition model to obtain skeleton point information corresponding to each of the palm skeleton points.

[0101] Here, the target palm skeleton point recognition model is a trained palm skeleton point recognition model, in which input data is a palm image and output data is skeleton point information corresponding to each palm skeleton point in the palm image.

[0102] Specifically, the computer device can recognize palm skeletal points through a machine learning model, and the computer device can obtain a pre-trained target palm skeletal point recognition model, input a target palm image into the target palm skeletal point recognition model, and finally output skeletal point information corresponding to each palm skeletal point from the model through data processing of the model.

[0103] The training process of the target palm skeleton point recognition model includes the steps of: obtaining a training dataset, where the training dataset includes a plurality of training palm images and training skeleton point information corresponding to each training palm image; determining a current palm image from the training dataset; inputting the current palm image into an initial palm skeleton point recognition model to obtain predicted skeleton point information corresponding to the current palm image; adjusting model parameters corresponding to the initial palm skeleton point recognition model based on a difference between the predicted skeleton point information corresponding to the current palm image and the training skeleton point information to obtain an intermediate skeleton point recognition model; and returning to the step of taking the next training palm image as the current palm image and taking the intermediate skeleton point recognition model as the initial palm skeleton point recognition model until a convergence condition is met, to obtain a target skeleton point recognition model.

[0104] Here, the initial palm skeleton point recognition model is the palm skeleton point recognition model to be trained. The training palm image is a palm image with known skeleton point information. The current palm image is any palm image in the training dataset. The training skeleton point information is accurate skeleton point information. The predicted skeleton point information is skeleton point information predicted by the model.

[0105] Specifically, during model training, the computer device obtains a training data set, and performs model training on the initial palm skeleton point recognition model based on the training data set to obtain a target palm skeleton point recognition model. The target training image in the training data set is used as input data of the palm skeleton point recognition model, and the training skeleton point information is used as expected output data of the palm skeleton point recognition model, and the training goal of the model is to make the actual output data of the model as close as possible to the expected output data, so that the model can output predicted skeleton point information close to the training skeleton point information and accurately recognize the palm skeleton points in the palm image.

[0106] The computer device randomly selects a training palm image from the training data set as the current palm image, inputs the current palm image into the initial palm skeleton point recognition model, and outputs predicted skeleton point information corresponding to the current palm image from the model through data processing of the model. Furthermore, the computer device generates loss information based on the difference between the predicted skeleton point information corresponding to the current palm image and the training skeleton point information, and performs backpropagation based on the loss information to adjust model parameters of the initial palm skeleton point recognition model to obtain an intermediate palm skeleton point recognition model. The computer device obtains the next training palm image as a new training palm image, sets the intermediate palm skeleton point recognition model as the new initial palm skeleton point recognition model, and returns to the step of inputting the current palm image into the initial palm skeleton point recognition model to obtain predicted skeleton point information corresponding to the current palm image to perform model iterative training, and adjusts the model parameters through multiple model iterative trainings until the second convergence condition is met, thereby obtaining a target palm skeleton point recognition model.

[0107] Here, the second convergence condition may be at least one of the following conditions: the loss information is smaller than a predetermined threshold, the number of iterations of the model is greater than a predetermined number, etc. For example, if the loss information calculated based on the difference between the predicted skeleton point information and the training skeleton point information in a certain training is smaller than a predetermined threshold, the adjustment of the model parameters is stopped, and the most recently adjusted palm skeleton point recognition model is set as the target palm skeleton point recognition model. If the number of iterations of the model after a certain training is greater than a predetermined number, the most recently adjusted palm skeleton point recognition model is set as the target palm skeleton point recognition model.

[0108] It should be noted that the current palm image determined in each training session may include at least one training palm image.

[0109] In the above embodiment, the palm skeleton points in the palm image are recognized by the target palm skeleton point recognition model, so that the accuracy and efficiency of the recognition can be ensured. An accurate target palm skeleton point recognition model can be obtained by training in a supervised iterative training manner.

[0110] In one embodiment, the palm contour is used to generate a palm animation.

[0111] Specifically, after generating a palm outline, a palm moving image can be generated based on the palm outline and then displayed.

[0112] In one embodiment, the computing device can collect palm images in real time, recognize palm contours in the palm images in real time, and display the dynamic palm contours to form a dynamic palm video. In one embodiment, the computing device can set display parameters of the palm contour to generate the palm video. For example, the palm contour is displayed in a dynamically changing gradient color. In one embodiment, the computing device can set display parameters of a closed area formed by the palm contour to generate the palm video. For example, the closed area formed by the palm contour is displayed in a dynamically changing rainbow color.

[0113] In the above embodiment, the palm contour extracted from the palm image is used to generate palm animation, which can improve entertainment and interactivity.

[0114] In one embodiment, as shown in Fig. 6, a control command generation method is provided, and the method is described as being applied to a terminal as an example. Note that the method may be applied to a server, or to a system including a terminal and a server, and may be realized by an interaction between the terminal and the server. In this embodiment, the control command generation method includes the following steps S602 to S604.

[0115] In step S602, the palm outline determined based on the target palm image is displayed.

[0116] Here, the palm contour extraction process includes the steps of: acquiring skeleton point information corresponding to each of the palm skeleton points in the target palm image, the skeleton point information including skeleton point positions and skeleton point types; matching each palm skeleton point based on the skeleton point type to obtain a plurality of palm skeleton point sets, the palm skeleton point sets having a corresponding geometric processing type; generating a palm contour auxiliary line matching the geometric processing type corresponding to the palm skeleton point set, based on the skeleton point positions corresponding to each skeleton point in the palm skeleton point set, to obtain a palm contour auxiliary line corresponding to each of the palm skeleton point sets; determining palm contour points from the palm contour auxiliary line corresponding to the palm skeleton point set, based on a reference step size and skeleton point positions corresponding to the same palm skeleton point set, to obtain contour point positions corresponding to the palm contour points; and generating a palm contour corresponding to the target palm image based on each contour point position.

[0117] For the specific process of extracting the palm contour, reference can be made to the contents of each embodiment of the palm contour extraction method described above, and a description thereof will be omitted here.

[0118] In step S604, a target control command is generated based on the display information of the palm outline.

[0119] Here, the display information is for reflecting the display status of the palm outline, and includes at least one type of data among the display position, the display size, the display depth, the display attitude, and the like.

[0120] The object control instruction is an instruction for controlling or manipulating an object. The controlled object may be a virtual object, such as a virtual resource, a virtual person, a virtual control, etc. The controlled object may be a real object, such as an electronic device, a mechanical equipment, etc. The object control instruction may be various control instructions. In one embodiment, the object control instruction may be an instruction for turning on or off a function or functions of the object, for example, the object control instruction may be a light command, a light command, a test start command, a test stop command, etc. In one embodiment, the object control instruction may be an instruction for controlling and moving the object, for example, the object control instruction may be a virtual resource transfer command, a virtual person jump command, etc. In one embodiment, the object control instruction may be an instruction for controlling to adjust an attribute parameter of the object, for example, the object control instruction may be a brush color adjustment command, a window size adjustment command, etc.

[0121] Specifically, the terminal displays a palm outline determined based on the target palm image, and generates an object control command based on the display information of the palm outline. The terminal can autonomously execute the object control command and control the corresponding object by the object control command. The terminal may transmit the object control command to another device, and instruct the other device to control the object by the object control command.

[0122] In one embodiment, if the display information of the palm outline satisfies a predetermined condition, a target control command is generated, and if the display information of the palm outline does not satisfy the predetermined condition, prompt information is generated. For example, if the position difference between the display position of the palm outline and the target display position is smaller than a predetermined threshold, a target control command may be generated, and if the display size of the palm outline is larger than a predetermined size, a target control command may be generated. If the difference in the display depth of the palm outline is smaller than a predetermined difference, a target control command may be generated. For example, a closed region formed by the palm outline is divided to obtain at least two sub-regions, and the depth average values ​​corresponding to each sub-region are statistically calculated, and the data difference between each depth average value is statistically calculated to obtain a display depth difference. If the display depth difference is smaller than a predetermined difference, the difference in the depth values ​​corresponding to each pixel point on the palm is not large, indicating that the palm position is correct, and a target control command may be generated. Here, the prompt information may indicate that the current palm position is incorrect, or may indicate that the palm should be moved.

[0123] In one embodiment, a target control command corresponding to the display information can be generated based on the display information of the palm outline. For example, a corresponding target control command can be generated based on the display posture of the palm outline. For example, when the display posture of the palm outline is open fingers, a function on command is generated, and when the display posture corresponding to the palm outline is closed fingers, a function off command is generated.

[0124] In the above control command generating method, the palm outline determined based on the target palm image is displayed, and a target control command is generated based on the display information of the palm outline. Here, the palm contour extraction process includes the steps of: acquiring skeleton point information corresponding to each of the palm skeleton points in the target palm image, the skeleton point information including skeleton point positions and skeleton point types; matching each palm skeleton point based on the skeleton point type to obtain a plurality of palm skeleton point sets, each of which has a corresponding geometric processing type; generating a palm contour auxiliary line matching the geometric processing type corresponding to the palm skeleton point set, based on the skeleton point positions corresponding to each skeleton point in the same palm skeleton point set, to obtain a palm contour auxiliary line corresponding to each of the palm skeleton point sets; determining palm contour points from the palm contour auxiliary line corresponding to the palm skeleton point set, based on the reference step size and skeleton point positions corresponding to the same palm skeleton point set, to obtain contour point positions corresponding to the palm contour points; and generating a palm contour corresponding to the target palm image based on each contour point position. In this way, the palm contour is generated based on the contour point positions corresponding to each palm contour point by performing geometric processing based on the skeleton point information corresponding to each of the palm skeleton points in the palm image, and the palm contour is generated based on the contour point positions corresponding to each of the palm contour points. The palm contour is extracted not only without complex data processing of the pixel information of the palm image, but also with high accuracy because it is less likely to be interfered with by the background of the image. Extracting an accurate palm contour also helps to ensure the accuracy of the display, and further helps to improve the accuracy of the generation of the target control command. Automatically generating the target control command based on the display information of the palm contour can also improve the efficiency of generating the target control command.

[0125] In one embodiment, the step of generating a target control command based on the display information of the palm outline includes:

[0126] The method includes the steps of: generating a target control command when a position difference between the display position of the palm outline and the target display position is smaller than a predetermined threshold; and generating palm movement prompt information based on the position difference when the position difference is equal to or greater than the predetermined threshold, the palm movement prompt information being intended to prompt the user to move the palm to reduce the position difference.

[0127] Here, the target display position is a desired display position. The target display position can be set according to actual needs. The palm movement prompt information may be displayed in at least one of the following expression forms: text, voice, picture, and video.

[0128] Specifically, the display information includes a display position. If the position difference between the display position of the palm outline and the target display position is smaller than a predetermined threshold, it indicates that the palm outline is displayed at a desired position, the target position, and the terminal triggers the generation of a target control command. If the position difference between the display position of the palm outline and the target display position is equal to or greater than a predetermined threshold, it indicates that the palm outline is not displayed at a desired position, the terminal can generate palm movement prompt information based on the position difference, the palm movement prompt information is for prompting the user to move the palm to reduce the position difference so that the palm outline can finally be displayed at the desired position.

[0129] In one embodiment, the terminal can determine the display position of the palm outline based on the spatial position of the palm, for example, if the palm is located directly above the image collection device, the palm outline is displayed in the middle of the screen, if the palm is located to the left of the image collection device, the palm outline is displayed on the left side of the screen, and if the palm is located to the right of the image collection device, the palm outline is displayed on the right side of the screen.

[0130] In the above embodiment, when the position difference between the display position of the palm outline and the target display position is smaller than a predetermined threshold, a target control command is generated; when the position difference is equal to or larger than a predetermined threshold, palm movement prompt information is generated based on the position difference, and the palm movement prompt information is for prompting the palm to move and reduce the position difference. In this way, the target control command is generated only when the position difference between the display position of the palm outline and the target display position is small, thereby avoiding blind generation of the target control command and ensuring effective and accurate triggering of the target control command. When the position difference between the display position of the palm outline and the target display position is large, palm movement prompt information is generated to prompt the palm to move, and the palm movement prompt information can realize accurate guidance, and the palm outline can be displayed at an accurate position, thereby triggering the target control command.

[0131] In one embodiment, the target palm image is a palm image corresponding to a virtual resource transfer source, and the target control command is a virtual resource transfer command, and the virtual resource transfer command is for instructing the transfer of a virtual resource from a resource account corresponding to the virtual resource transfer source to a target account.

[0132] Here, the virtual resource transfer source is one side that needs to transfer the virtual resource from the resource account. The opposite of the virtual resource transfer source is the virtual resource transfer destination, which is the other side that receives the virtual resource. The resource account is an account that can store the virtual resource. The virtual resource is a circulatable resource that exists in the electronic account, such as currency, virtual red envelope, game currency, virtual item, etc. The target account is an account that needs to receive the transferred virtual resource. The target account may be an individual account or an organization account.

[0133] Specifically, in a virtual resource transfer scene, a terminal corresponding to a virtual resource transfer source or a virtual resource transfer destination or other terminal can trigger the generation of a virtual resource transfer command by a palm image, and the virtual resource transfer command is for instructing to transfer the virtual resource from a resource account corresponding to the virtual resource transfer source to a target account.

[0134] For example, the terminal collects a palm image corresponding to a virtual resource transfer source as a target palm image, extracts a palm outline in the target palm image, and displays the palm outline determined based on the target palm image. If the position difference between the display position of the palm outline and the target display position is smaller than a predetermined threshold, the terminal verifies the identity information of the virtual resource transfer source based on the target palm image. If the verification is successful, the terminal generates a virtual resource transfer command, which is for instructing to transfer the virtual resource from the resource account corresponding to the virtual resource transfer source to the target account, and if the verification is unsuccessful, the terminal generates verification failure prompt information. If the position difference is equal to or greater than the predetermined threshold, the terminal generates palm movement prompt information based on the position difference, which is for prompting the user to move the palm to reduce the position difference. After the palm is moved, if the new position difference determined by the terminal based on the new palm image is smaller than a predetermined threshold, the terminal verifies the identity information of the virtual resource transfer source based on the new palm image, and if the verification is successful, generates a virtual resource transfer command.

[0135] In the above embodiment, the palm outline extracted from the palm image can be used to trigger the generation of a virtual resource transfer command, and the triggering method of the virtual resource transfer command is extended.

[0136] In a specific embodiment, the above palm outline extraction method can be applied to a palm authentication payment scenario. Specifically, the palm outline extraction method is applied to the scenario as follows:

[0137] 1. Palm contour recognition

[0138] 1-1. Palm Skeleton Recognition Based on a machine learning algorithm, the palm skeleton points in the palm image are recognized. The palm image is captured by a camera, and the palm image is input to a trained machine learning model, and the model outputs skeleton point information of the palm skeleton points in the palm image. In one embodiment, the machine learning model can be integrated into an SDK (Software Development Kit). The skeleton point information includes the position coordinates of the palm skeleton points. The position coordinates of the palm skeleton points can be mapped to the two-dimensional coordinates of the screen of the current operating system according to the coordinate system in the SDK, and the palm outline can be generated according to the screen coordinates. The current operating system includes the Android system, the Windows system, etc.

[0139] 7A is a schematic diagram of the recognition result of palm skeleton points. As shown in FIG 7A, 21 palm skeleton points (palm skeleton point 0 to palm skeleton point 20) are finally recognized in the palm image.

[0140] 1-2. Recognition of palm outline points

[0141] 7B is a schematic diagram of the recognition result of the palm contour points. As shown in FIG 7B, 38 palm contour points (palm contour point 1' to palm contour point 38') are recognized based on 21 palm skeleton points.

[0142] 1-2-1. Calculating the coordinates of the outer contour of the palm

[0143] The coordinates of the palm outer contour on the extension line are obtained according to the extension line (also called the extension line) of the two palm skeleton points and the relative coefficient (also called the reference step size). For example, as shown in Fig. 8A, a palm contour point 37' is determined on the extension line generated based on palm skeleton point 0 and palm skeleton point 1, and the palm contour point 37' is the palm outer contour point.

[0144] The calculation process of the coordinates of the outer contour of the palm is as follows:

[0145] Palm skeleton start coordinates: (Xa,Ya), Palm skeleton end coordinates: (Xb,Yb) Palm contour target point coordinates: (X,Y) Target point coordinate X:

number

number

[0146] As shown in FIG. 7B, palm contour point 37', palm contour point 38', palm contour point 36', and palm contour point 8' are the outer contour points of the palm.

[0147] 1-2-2. Calculation of interdigital coordinates

[0148] The coordinates of the inter-finger area on the perpendicular bisector are obtained by the perpendicular bisector of the two palm skeleton points and the relative coefficients. For example, as shown in Fig. 8B, a palm outline point 29', which is a palm inter-finger area point, is determined on the perpendicular bisector generated based on palm skeleton point 13 and palm skeleton point 17.

[0149] The calculation process of the interdigital coordinates is as follows.

[0150] Palm skeleton start coordinates: (Xa,Ya), Palm skeleton end coordinates: (Xb,Yb) Midpoint coordinates: (Xmid, Ymid) Xmid:

number

number

number

number

number

number

[0151] As shown in FIG. 7B, palm outline point 29', palm outline point 22', and palm outline point 15' are palm inter-finger points.

[0152] Taking palm skeleton point 13 and palm skeleton point 17 as an example, palm outline point 29' is determined in a clockwise direction if palm skeleton point 13 is the start point and palm skeleton point 17 is the end point. Palm outline point 29' is determined in a counterclockwise direction if palm skeleton point 17 is the start point and palm skeleton point 13 is the end point.

[0153] 1-2-3. Calculation of finger contour edge coordinates

[0154] For vector AB generated by two palm skeleton points, the coordinates of the finger contour edge on the normal vector are obtained based on the relative coefficient in the normal vector of point A or point B. For example, as shown in Fig. 8C, in the normal vector generated based on palm skeleton point 2 and palm skeleton point 3, palm contour point 3' and palm contour point 6' are determined, and palm contour point 3' and palm contour point 6' are finger contour edge points.

[0155] The process of calculating the coordinates of the finger contour edge is as follows.

[0156] Palm skeleton start coordinates: (Xa,Ya), Palm skeleton end coordinates: (Xb,Yb) Palm contour target point coordinates: (X,Y) If the target point is in the clockwise direction, Target point coordinate X:

number

number

number

number

[0157] As shown in FIG. 7B, palm contour point 2' to palm contour point 7', palm contour point 9' to palm contour point 14', palm contour point 16' to palm contour point 21', palm contour point 23' to palm contour point 28', and palm contour point 30' to palm contour point 35' are finger contour edge points.

[0158] Here, k in the above formula is a relative coefficient.

[0159] 1-3. Palm contour generation

[0160] Each palm contour point is connected to obtain the palm contour. Figure 8D is a schematic diagram of the palm contour, and as shown in Figure 8D, 38 palm contour points are connected to obtain the palm contour.

[0161] 2. Palm recognition video generation

[0162] In the scenario of palm recognition payment, after the palm contour is obtained, a palm recognition video can be generated based on the palm contour, thereby improving the interactive and high-tech feeling that palm recognition payment gives to users.

[0163] As shown in Fig. 9, 1002 in Fig. 9 indicates a palm outline displayed on the screen, and 1004 indicates a target display position on the screen. If the palm outline is not displayed at the target display position, the user is prompted to move the palm to the target display position; if the palm outline is displayed at the target display position, the user's identity is verified by the palm image, and payment is made after the verification is successful.

[0164] In the above embodiment, the palm outline points are determined based on the palm skeleton points, and then the palm outline is generated. On the one hand, the palm skeleton points are not easily affected by the background of the image, and the palm outline is extracted from the image based on the palm skeleton points, so that the accuracy of the palm outline extraction can be ensured. On the other hand, the determination of the palm outline points using a geometric method can ensure the efficiency of data processing, ensure the recognition performance, and improve the efficiency of the palm outline extraction.

[0165] Although the steps in the flowcharts according to the above-described embodiments are shown sequentially as indicated by the arrows, it will be understood that the steps are not necessarily performed sequentially in the order indicated by the arrows. Unless otherwise specified in this specification, there is no strict order restriction on the execution of the steps, and the steps may be performed in other orders. Furthermore, at least some of the steps in the flowcharts according to the above-described embodiments may include multiple sub-steps or multiple stages, and these sub-steps or stages do not necessarily have to be performed at the same time, but may be performed at different times, and the order of execution of these sub-steps or stages does not necessarily have to be performed sequentially, but may be performed alternately or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0166] Based on a similar inventive idea, the embodiments of the present application further provide a palm outline extraction device for implementing the above-described palm outline extraction method, and further provide a control command generation device for implementing the above-described control command generation method. Since the implementation means for solving the problem provided by the device are similar to the implementation means described in the above-described method, specific limitations in one or more embodiments of the palm outline extraction device provided below may refer to the limitations described above for the palm outline extraction method, and specific limitations in one or more embodiments of the control command generation device provided below may refer to the limitations described above for the control command generation method, and description thereof will be omitted here.

[0167] In one embodiment, as shown in FIG. 10, a palm contour extraction device is provided, which includes a skeleton point information acquisition module 1002, a palm skeleton point matching module 1004, a palm contour auxiliary line determination module 1006, a palm contour point determination module 1008, and a palm contour generation module 1010.

[0168] The skeleton point information acquisition module 1002 is configured to acquire skeleton point information corresponding to each palm skeleton point in the target palm image, where the skeleton point information includes a skeleton point position and a skeleton point type. The palm skeleton point matching module 1004 is configured to match each palm skeleton point based on a skeleton point type to obtain a plurality of palm skeleton point sets, each of which has a corresponding geometric processing type.

[0169] The palm contour assistant line determination module 1006 is configured to generate palm contour assistant lines conforming to a geometric processing type corresponding to the palm skeleton point set based on skeleton point positions corresponding to each skeleton point in the same palm skeleton point set, to obtain palm contour assistant lines corresponding to each of the palm skeleton point sets.

[0170] The palm contour point determination module 1008 is configured to determine palm contour points from the palm contour auxiliary lines corresponding to the palm skeleton point set based on the reference step size and skeleton point positions corresponding to the same palm skeleton point set, and obtain contour point positions corresponding to the palm contour points.

[0171] The palm contour generation module 1010 is configured to generate a palm contour corresponding to the target palm image based on each contour point location.

[0172] The palm contour extraction device does not require complex data processing of pixel information of the palm image, and by performing geometric processing based on skeleton point information corresponding to each of the palm skeleton points in the palm image, it can easily determine contour point positions corresponding to the palm contour points from the palm image, and further generate a palm contour based on the contour point positions corresponding to each of the palm contour points. The positions and types of palm skeleton points are not easily interfered with by the background of the image, and by extracting the palm contour based on the positions and types of palm skeleton points in the palm image, it is possible to effectively improve the accuracy of palm contour extraction.

[0173] In one embodiment, the skeleton point type is determined by the finger parameters and the intra-finger joint parameters. The palm skeleton point matching module 1004 is further configured to: establish a matching relationship with adjacent palm skeleton points, the intra-finger joint parameters of which are the predetermined joint parameters, to obtain a palm skeleton point set corresponding to a first geometric processing type; establish a matching relationship with adjacent palm skeleton points corresponding to the same finger parameters, to obtain a palm skeleton point set corresponding to a second geometric processing type, and set the palm skeleton point corresponding to the predetermined skeleton point type as the predetermined skeleton point; and establish a matching relationship with the predetermined skeleton point and the palm skeleton point closest to the predetermined skeleton point, to obtain a palm skeleton point set corresponding to a third geometric processing type.

[0174] In one embodiment, the palm contour auxiliary line determination module 1006 is further configured to determine a current auxiliary line type based on a geometric processing type corresponding to the current palm skeleton point set, generate a current skeleton point line segment corresponding to the current palm skeleton point set based on skeleton point positions corresponding to each palm skeleton point in the current palm skeleton point set, and perform geometric processing on the current skeleton point line segment based on the current auxiliary line type to obtain a palm contour auxiliary line corresponding to the current palm skeleton point set, which belongs to the current auxiliary line type.

[0175] In one embodiment, the geometric processing type includes any one of a first geometric processing type, a second geometric processing type, or a third geometric processing type, the auxiliary line type corresponding to the first geometric processing type includes at least one of a perpendicular bisector type or an extension line type, the auxiliary line type corresponding to the second geometric processing type includes a normal line type, and the auxiliary line type corresponding to the third geometric processing type includes at least one of a normal line type or an extension line type.

[0176] In one embodiment, the palm contour auxiliary line determination module 1006 is further configured to: if the current auxiliary line type is a perpendicular bisector type, set the perpendicular bisector corresponding to the current skeleton point line segment as the palm contour auxiliary line; if the current auxiliary line type is an extension line type, determine a target skeleton point from each palm skeleton point corresponding to the current palm skeleton point set, and set an extension line corresponding to the current skeleton point line segment and starting from the target skeleton point, as the palm contour auxiliary line; if the current auxiliary line type is a normal type, determine a target skeleton point from each palm skeleton point corresponding to the current palm skeleton point set, and set a normal to the current skeleton point line segment at the target skeleton point as the palm contour auxiliary line.

[0177] In one embodiment, the palm contour point determination module 1008 is further configured to: generate a current skeleton point line segment corresponding to the current palm skeleton point set based on skeleton point positions corresponding to each skeleton point in the current palm skeleton point set; determine a reference point from the current skeleton point line segment; and move, starting from the reference point, on the current palm contour auxiliary line corresponding to the current palm skeleton point set by a reference step size corresponding to the current palm skeleton point set along a movement direction corresponding to the current palm contour auxiliary line, to obtain the current palm contour point and a contour point position corresponding to the current palm contour point.

[0178] In one embodiment, the current palm contour auxiliary line includes at least one of a perpendicular bisector, an extension line, or a normal line, and the movement direction corresponding to the perpendicular bisector is the direction away from the center of the palm, the movement direction corresponding to the extension line is the extension direction of the extension line, and the movement direction corresponding to the normal line is the perpendicular direction of the current skeleton point line segment.

[0179] In one embodiment, each palm skeleton point has a corresponding skeleton point identifier. The palm contour generation module 1010 is further configured to map the skeleton point identifiers corresponding to the palm skeleton points based on the predetermined mapping information including an identifier mapping relationship between the palm skeleton points in the palm skeleton point set and the palm contour points generated based on the palm skeleton point set to obtain a contour point sort identifier corresponding to each palm contour point, and then order and connect the contour point positions corresponding to each palm contour point according to the contour point sort identifier to obtain a palm contour corresponding to the target palm image.

[0180] In one embodiment, the skeleton point information acquisition module 1002 is further configured to input the target palm image into a target palm skeleton point recognition model to obtain skeleton point information corresponding to each palm skeleton point.

[0181] As shown in FIG. 11, the palm outline extraction device The method further includes a model training module 1001, which obtains a training dataset including a plurality of training palm images and training skeleton point information corresponding to each of the training palm images, determines a current palm image from the training dataset, inputs the current palm image into an initial palm skeleton point recognition model to obtain predicted skeleton point information corresponding to the current palm image, and adjusts model parameters corresponding to the initial palm skeleton point recognition model based on a difference between the predicted skeleton point information corresponding to the current palm image and the training skeleton point information to obtain an intermediate skeleton point recognition model, and returns to the steps of taking a next training palm image as the current palm image and taking the intermediate skeleton point recognition model as the initial palm skeleton point recognition model until a convergence condition is met, to obtain a target skeleton point recognition model.

[0182] In one embodiment, the palm contour is used to generate a palm animation.

[0183] In one embodiment, as shown in FIG. 12, a control command generating device is provided, which includes a palm outline display module 1202 and a control command generating module 1204.

[0184] The palm contour display module 1202 is configured to display the palm contour determined based on the target palm image.

[0185] The control command generating module 1204 is configured to generate a target control command based on the display information of the palm contour.

[0186] The palm contour extraction process includes the steps of: acquiring skeleton point information corresponding to each of the palm skeleton points in the target palm image, the skeleton point information including a skeleton point position and a skeleton point type; matching each of the palm skeleton points based on the skeleton point type to obtain a plurality of palm skeleton point sets, the palm skeleton point sets having a corresponding geometric processing type; generating a palm contour auxiliary line matching the geometric processing type corresponding to the palm skeleton point set, based on the skeleton point positions corresponding to each skeleton point in the palm skeleton point set, to obtain a palm contour auxiliary line corresponding to each of the palm skeleton point sets; determining a palm contour point from the palm contour auxiliary line corresponding to the palm skeleton point set, based on a reference step size and skeleton point positions corresponding to the same palm skeleton point set, to obtain a contour point position corresponding to the palm contour point; and generating a palm contour corresponding to the target palm image based on each contour point position.

[0187] In the control command generation device, the palm contour is generated based on the contour point positions corresponding to each palm contour point by performing geometric processing based on skeleton point information corresponding to each of the palm skeleton points in the palm image, and the palm contour is generated based on the contour point positions corresponding to each of the palm contour points. The palm contour is extracted not only without complex data processing of the pixel information of the palm image, but also with high accuracy because it is not easily interfered with by the background of the image. Extracting an accurate palm contour helps to ensure the accuracy of the display and further helps to improve the accuracy of the generation of the target control command. Automatically generating the target control command based on the display information of the palm contour can also improve the efficiency of generating the target control command.

[0188] In one embodiment, the control command generation module 1204 is further configured to generate a target control command when a position difference between the display position of the palm outline and the target display position is smaller than a predetermined threshold, and generate palm movement prompt information based on the position difference when the position difference is equal to or larger than the predetermined threshold, where the palm movement prompt information is for prompting moving the palm to reduce the position difference.

[0189] In one embodiment, the target palm image is a palm image corresponding to a virtual resource transfer source, and the target control command is a virtual resource transfer command, and the virtual resource transfer command is for instructing the transfer of a virtual resource from a resource account corresponding to the virtual resource transfer source to a target account.

[0190] Each module in the palm contour extraction device and control command generation device may be realized in whole or in part by software, hardware, or a combination thereof. Each module may be incorporated as hardware into one or more processors in a computer device, or may be provided independently of one or more processors in a computer device, or may be stored as software in a memory in a computer device, so that one or more processors call and execute operations corresponding to each module.

[0191] In one embodiment, a computer device is provided, which may be a server, and its internal configuration diagram may be shown as in FIG. 13. The computer device includes a processor, a memory, an input / output interface (abbreviated as Input / Output, I / O), and a communication interface. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. Here, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer-readable instructions, and a database. The internal memory provides an environment for the execution of the operating system and the computer-readable instructions in the non-volatile storage medium. The database of the computer device is used to store data such as a target palm skeleton point recognition model, a reference step size, and a predetermined mapping relationship. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer-readable instructions are executed by a processor, a palm contour extraction method or a control command generation method is realized.

[0192] In one embodiment, a computer device is provided, which may be a terminal, and its internal configuration diagram may be shown as in FIG. 14. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. Here, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system and computer-readable instructions. The internal memory provides an environment for the execution of the operating system and the computer-readable instructions in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be realized by WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer-readable instructions are executed by a processor, a palm contour extraction method or a control command generation method is realized. The display unit of the computing device is used to form a viewable screen, and may be a display screen, a projection device, or a virtual reality imaging device, the display screen may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computing device may be a touch layer covering the display screen, a button, a trackball, or a touch pad installed on the housing of the computing device, or an external keyboard, touch pad, mouse, etc.

[0193] Those skilled in the art can understand that the configurations shown in Figures 13 and 14 are merely block diagrams of parts of the configuration related to the technical means of the present application, and do not limit the computer device to which the technical means of the present application is applied. A specific computer device may include more or less components than those shown in the drawings, combine some components, or have a different component layout.

[0194] In one embodiment, a computing device is further provided that includes a memory and one or more processors, wherein computer readable instructions are stored in the memory and, when executed by the one or more processors, the steps in each of the above method embodiments are realized.

[0195] In one embodiment, a computer-readable storage medium is provided having computer-readable instructions stored thereon that, when executed by one or more processors, effectuate the steps of each of the method embodiments described above.

[0196] The computer program product includes computer readable instructions that, when executed by one or more processors, effectuate the steps of each of the method embodiments described above.

[0197] In addition, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) related to this application are all information and data authorized by the user or fully authorized by each party, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0198] It is obvious to those skilled in the art that all or part of the flow of the method of the above embodiments may be completed by instructing relevant hardware by computer-readable instructions. The computer-readable instructions may be stored in a non-volatile computer-readable storage medium, and the computer-readable instructions may include the flow of each of the above method embodiments when executed. Here, any reference to a memory, database, or other medium used in each embodiment provided in this application may include at least one of a non-volatile memory and a volatile memory. The non-volatile memory may include a read-only memory (ROM), a magnetic tape, a flexible disk, a flash memory, an optical memory, a high-density embedded non-volatile memory, a resistive memory (ReRAM), a magnetoresistive random access memory (MRAM), a ferroelectric random access memory (FRAM), a phase change memory (PCM), a graphene memory, etc. The volatile memory may include a random access memory (RAM), an external cache memory, etc. By way of example and not limitation, the RAM can be in various forms, such as, for example, a static random access memory (SRAM) or a dynamic random access memory (DRAM). The database according to the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a distributed database based on a blockchain. The processor according to the embodiments provided herein may be, but is not limited to, a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, and the like.

[0199] The technical features of the above embodiments can be combined in any manner. For the sake of brevity, all possible combinations of the technical features in the above embodiments have not been described. However, as long as there is no contradiction in the combinations of these technical features, they should all be considered to be within the scope described in this specification.

[0200] The above embodiments only describe some embodiments of the present application, and although the descriptions are specific and detailed, they should not be construed as limiting the scope of the patent of the present application. It should be noted that those skilled in the art may make various modifications and improvements to the present application without departing from the spirit of the present application, and both of these modifications and improvements belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the scope of the appended claims.

Claims

1. A palm contour extraction method implemented by a computing device, comprising: acquiring skeleton point information corresponding to each palm skeleton point in the target palm image, the skeleton point information including a skeleton point position and a skeleton point type; matching each palm skeleton point based on the skeleton point type to obtain a plurality of palm skeleton point sets, each palm skeleton point set having a corresponding geometric processing type; generating palm contour assistant lines conforming to a geometric processing type corresponding to the palm skeleton point set based on skeleton point positions corresponding to each skeleton point in the same palm skeleton point set to obtain palm contour assistant lines corresponding to each of the palm skeleton point sets; determining palm contour points from the palm contour auxiliary line corresponding to the set of palm skeleton points based on a reference step size and skeleton point positions corresponding to the same set of palm skeleton points, thereby obtaining contour point positions corresponding to the palm contour points; and generating a palm contour corresponding to the target palm image based on each contour point location. Palm contour extraction method.

2. The skeleton point type is determined by finger parameters and intra-finger joint parameters, The step of matching each of the palm skeleton points based on the skeleton point type to obtain a plurality of palm skeleton point sets includes: Establishing a matching relationship for adjacent palm skeleton points whose intra-finger joint parameters are the predetermined joint parameters to obtain each palm skeleton point set corresponding to a first geometric processing type; Establishing a matching relationship between adjacent palm skeleton points corresponding to the same finger parameters to obtain each palm skeleton point set corresponding to a second geometric processing type; and determining a palm skeleton point corresponding to a predetermined skeleton point type as a predetermined skeleton point, and establishing a matching relationship between the predetermined skeleton point and a palm skeleton point closest to the predetermined skeleton point to obtain a palm skeleton point set corresponding to a third geometric processing type. The palm contour extraction method according to claim 1 .

3. The step of generating palm contour auxiliary lines conforming to a geometric processing type corresponding to the palm skeleton point set based on skeleton point positions corresponding to each skeleton point in the same palm skeleton point set to obtain palm contour auxiliary lines corresponding to each of the palm skeleton point sets includes: determining a current auxiliary line type based on a geometric processing type corresponding to the current palm skeleton point set; generating a current skeleton point line segment corresponding to the current palm skeleton point set based on a skeleton point position corresponding to each palm skeleton point in the current palm skeleton point set; and performing a geometric process on the current skeleton point line segment based on the current auxiliary line type to obtain a palm contour auxiliary line corresponding to the current palm skeleton point set and belonging to the current auxiliary line type. The palm contour extraction method according to claim 1 .

4. the geometric processing type includes any one of a first geometric processing type, a second geometric processing type, or a third geometric processing type; The auxiliary line type corresponding to the first geometric processing type includes at least one of a perpendicular bisector type or an extension line type; The auxiliary line type corresponding to the second geometric processing type includes a normal line type; The auxiliary line type corresponding to the third geometric processing type includes at least one of a normal line type or an extension line type. The palm contour extraction method according to claim 3.

5. the step of performing a geometric process on the current skeleton point line segment based on the current auxiliary line type to obtain a palm contour auxiliary line corresponding to the current palm skeleton point set and belonging to the current auxiliary line type, if the current auxiliary line type is a perpendicular bisector type, setting the perpendicular bisector corresponding to the current skeleton point line segment as a palm contour auxiliary line; if the current auxiliary line type is an extension line type, determining a target skeleton point from each palm skeleton point corresponding to the current palm skeleton point set, and setting an extension line corresponding to the current skeleton point line segment and starting from the target skeleton point as a palm contour auxiliary line; if the current auxiliary line type is a normal type, determining a target skeleton point from each palm skeleton point corresponding to the current palm skeleton point set, and setting a normal line of the current skeleton point line segment at the target skeleton point as a palm contour auxiliary line. The palm contour extraction method according to claim 3.

6. the step of determining palm contour points from palm contour auxiliary lines corresponding to the same palm skeleton point set based on a reference step size and skeleton point positions corresponding to the same palm skeleton point set to obtain contour point positions corresponding to the palm contour points, generating a current skeleton point line segment corresponding to the current palm skeleton point set based on a skeleton point position corresponding to each skeleton point in the current palm skeleton point set; determining a reference point on the current skeleton point line segment; and moving the reference point as a starting point on the current palm contour auxiliary line corresponding to the current palm skeleton point set by a reference step size corresponding to the current palm skeleton point set along a moving direction corresponding to the current palm contour auxiliary line, thereby obtaining current palm contour points and contour point positions corresponding to the current palm contour points. The palm contour extraction method according to claim 1 .

7. The current palm outline auxiliary line includes at least one of a perpendicular bisector, an extension line, and a normal line; The movement direction corresponding to the perpendicular bisector is a direction opposite to the center of the palm, the movement direction corresponding to the extension line is an extension direction of the extension line, and the movement direction corresponding to the normal line is a perpendicular direction of the current skeleton point line segment. The palm contour extraction method according to claim 6.

8. Each palm skeleton point has a corresponding skeleton point identifier; generating a palm contour corresponding to the target palm image based on each contour point location, Mapping skeleton point identifiers corresponding to palm skeleton points based on predetermined mapping information to obtain contour point sort identifiers corresponding to each palm contour point, the predetermined mapping information including identifier mapping relationships between palm skeleton points in a palm skeleton point set and palm contour points generated based on the palm skeleton point set; and ordering and connecting the contour point positions corresponding to each palm contour point according to the contour point sort identifier to obtain a palm contour corresponding to the target palm image. The palm contour extraction method according to claim 1 .

9. The step of acquiring skeleton point information corresponding to each of the skeleton points of the target palm image includes: inputting the target palm image into a target palm skeleton point recognition model to obtain skeleton point information corresponding to each of the palm skeleton points; The training process of the target palm skeleton point recognition model includes: obtaining a training data set, the training data set including a plurality of training palm images and training skeleton point information corresponding to each of the training palm images; determining a current palm image from the training dataset; inputting the current palm image into an initial palm skeleton point recognition model to obtain predicted skeleton point information corresponding to the current palm image; According to a difference between the predicted skeleton point information corresponding to the current palm image and the training skeleton point information, adjusting a model parameter corresponding to the initial palm skeleton point recognition model to obtain an intermediate skeleton point recognition model; and returning to the step of taking the next training palm image as the current palm image, taking the intermediate skeleton point recognition model as the initial palm skeleton point recognition model, and inputting the current palm image into the initial palm skeleton point recognition model until a convergence condition is satisfied, to obtain the target palm skeleton point recognition model. The palm contour extraction method according to claim 1 .

10. The palm contour extraction method according to claim 1 , wherein the reference step size increases as palm size information corresponding to the target palm image increases.

11. The palm contour extraction method according to claim 1 , wherein the reference step size increases as the palm depth information corresponding to the target palm image decreases.

12. The palm outline extraction method according to any one of claims 1 to 11, characterized in that the palm outline is used to generate a palm video.

13. 1. A method for generating control instructions executed by a computer device, comprising: displaying a palm contour determined based on the target palm image; generating a target control command based on the display information of the palm outline; The palm outline extraction process includes: acquiring skeleton point information corresponding to each palm skeleton point in the target palm image, the skeleton point information including a skeleton point position and a skeleton point type; matching each palm skeleton point based on the skeleton point type to obtain a plurality of palm skeleton point sets, each palm skeleton point set having a corresponding geometric processing type; generating palm contour assistant lines conforming to a geometric processing type corresponding to the palm skeleton point set based on skeleton point positions corresponding to each skeleton point in the same palm skeleton point set to obtain palm contour assistant lines corresponding to each of the palm skeleton point sets; determining palm contour points from the palm contour auxiliary line corresponding to the set of palm skeleton points based on a reference step size and skeleton point positions corresponding to the same set of palm skeleton points, thereby obtaining contour point positions corresponding to the palm contour points; and generating a palm contour corresponding to the target palm image based on each contour point location. A method for generating control instructions.

14. The step of generating a target control command based on the display information of the palm outline includes: generating a target control command when a position difference between the display position of the palm outline and the target display position is smaller than a predetermined threshold; and if the position difference is equal to or greater than the predetermined threshold, generating palm movement prompt information based on the position difference, the palm movement prompt information being for prompting the user to move the palm to reduce the position difference. The control instruction generating method according to claim 13.

15. 15. The control command generating method according to claim 13, wherein the target palm image is a palm image corresponding to a virtual resource transfer source, the target control command is a virtual resource transfer command, and the virtual resource transfer command is for instructing to transfer a virtual resource from a resource account corresponding to the virtual resource transfer source to a target account.

16. a skeleton point information acquisition module configured to acquire skeleton point information corresponding to each palm skeleton point in a target palm image, the skeleton point information including a skeleton point position and a skeleton point type; a palm skeleton point matching module configured to match each palm skeleton point based on the skeleton point type to obtain a plurality of palm skeleton point sets, the palm skeleton point sets having corresponding geometric processing types; a palm contour extension determining module configured to generate a palm contour extension conforming to a geometric processing type corresponding to the palm skeleton point set based on skeleton point positions corresponding to each skeleton point in the same palm skeleton point set, thereby obtaining a palm contour extension corresponding to each of the palm skeleton point sets; a palm contour point determination module configured to determine palm contour points from palm contour auxiliary lines corresponding to the same palm skeleton point set based on a reference step size and skeleton point positions corresponding to the same palm skeleton point set, and obtain contour point positions corresponding to the palm contour points; and a palm contour generation module configured to generate a palm contour corresponding to the target palm image based on each contour point position. Palm contour extraction device.

17. A control instruction generating device, comprising: a palm contour display module configured to display a palm contour determined based on the target palm image; and a control command generation module configured to generate a target control command based on the palm outline display information, The palm outline extraction process includes: acquiring skeleton point information corresponding to each palm skeleton point in the target palm image, the skeleton point information including a skeleton point position and a skeleton point type; matching each palm skeleton point based on the skeleton point type to obtain a plurality of palm skeleton point sets, each palm skeleton point set having a corresponding geometric processing type; generating palm contour assistant lines conforming to a geometric processing type corresponding to the palm skeleton point set based on skeleton point positions corresponding to each skeleton point in the same palm skeleton point set to obtain palm contour assistant lines corresponding to each of the palm skeleton point sets; determining palm contour points from the palm contour auxiliary line corresponding to the set of palm skeleton points based on a reference step size and skeleton point positions corresponding to the same set of palm skeleton points, thereby obtaining contour point positions corresponding to the palm contour points; and generating a palm contour corresponding to the target palm image based on each contour point location. Control instruction generating device.

18. 1. A computing device including a memory having computer readable instructions stored thereon and one or more processors, When the computer-readable instructions are executed by the one or more processors, steps of the palm contour extraction method according to any one of claims 1 to 11 are realized. Computer equipment.

19. A computer program configured to cause a computer to execute the palm contour extraction method according to any one of claims 1 to 11.

Images