Palm contour extraction, control instruction generation method, device and computer equipment

Abstract

The present application relates to a palm contour extraction method, device, computer equipment, storage medium and computer program product. The method comprises: obtaining the skeletal point positions and skeletal point types corresponding to each palm skeletal point in the target palm image; matching each palm skeletal point based on the skeletal point type to obtain multiple palm skeletal point sets; the palm skeletal point set has a corresponding geometric processing type; based on the skeletal point positions corresponding to each skeletal point in the palm skeletal point set, generating a palm contour auxiliary line matching the geometric processing type corresponding to the palm skeletal point set; based on the reference step and skeletal point positions corresponding to the same palm skeletal point set, determining the palm contour points from the palm contour auxiliary lines corresponding to the palm skeletal point set to obtain the contour point positions corresponding to the palm contour points; generating the palm contour corresponding to the target palm image based on each contour point position. The method can improve the accuracy of palm contour extraction.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a palm contour extraction, control instruction generation method, device, computer equipment, storage medium and computer program product. Background Art

[0002] With the development of computer technology, palm extraction technology has emerged. Palm extraction technology includes the technical content of recognizing palm contours from palm images.

[0003] In conventional techniques, various data processing is usually performed on the pixel information of a palm image to extract the palm contour in the palm image, for example, binary processing and edge detection are performed on the palm image to extract the palm contour. However, the palm contour extraction method based on the pixel information of a palm image is easily interfered by the image background, and thus cannot extract an accurate palm contour. Summary of the invention

[0004] Based on this, it is necessary to provide a palm contour extraction, control instruction generation method, device, computer equipment, computer readable storage medium and computer program product that can improve the accuracy of palm contour extraction in response to the above technical problems.

[0005] A palm contour extraction method. The method comprises:

[0006] Obtaining the skeleton point information corresponding to each palm skeleton point in the target palm image; the skeleton point information includes the skeleton point position and the skeleton point type;

[0007] Based on the skeleton point type, each palm skeleton point is matched to obtain a plurality of palm skeleton point sets; the palm skeleton point sets have corresponding geometric processing types;

[0008] Based on the bone point positions corresponding to each bone point in the palm bone point set, a palm contour auxiliary line matching the geometric processing type corresponding to the palm bone point set is generated, and the palm contour auxiliary lines corresponding to each palm bone point set are obtained;

[0009] Based on the reference step length and the bone point position corresponding to the same palm bone point set, the palm contour point is determined from the palm contour auxiliary line corresponding to the palm bone point set, and the contour point position corresponding to the palm contour point is obtained;

[0010] A palm contour corresponding to the target palm image is generated based on the positions of the contour points.

[0011] A palm contour extraction device. The device comprises:

[0012] A skeleton point information acquisition module is used to acquire skeleton point information corresponding to each palm skeleton point in the target palm image; the skeleton point information includes skeleton point position and skeleton point type;

[0013] A palm skeleton point matching module, used for matching the palm skeleton points based on the skeleton point type to obtain a plurality of palm skeleton point sets; the palm skeleton point sets have corresponding geometric processing types;

[0014] A palm contour auxiliary line determination module is used to generate a palm contour auxiliary line matching the geometric processing type corresponding to the palm bone point set based on the bone point position corresponding to each bone point in the palm bone point set, and obtain the palm contour auxiliary lines corresponding to each palm bone point set respectively;

[0015] A palm contour point determination module is used to determine the palm contour points from the palm contour auxiliary lines corresponding to the palm skeleton point set based on the reference step length and the skeleton point position corresponding to the same palm skeleton point set, and obtain the contour point position corresponding to the palm contour point;

[0016] The palm contour generation module is used to generate the palm contour corresponding to the target palm image based on the positions of each contour point.

[0017] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the palm contour extraction method when executing the computer program.

[0018] A computer-readable storage medium stores a computer program, which implements the steps of the palm contour extraction method when executed by a processor.

[0019] A computer program product comprises a computer program, wherein when the computer program is executed by a processor, the steps of the palm contour extraction method are implemented.

[0020] The palm contour extraction method, device, computer equipment, storage medium and computer program product are as follows: by obtaining the skeletal point information corresponding to each palm skeletal point in the target palm image; the skeletal point information includes the skeletal point position and the skeletal point type; based on the skeletal point type, each palm skeletal point is matched to obtain multiple palm skeletal point sets; there is a corresponding geometric processing type for the palm skeletal point set; based on the skeletal point position corresponding to each skeletal point in the palm skeletal point set, a palm contour auxiliary line matching the geometric processing type corresponding to the palm skeletal point set is generated to obtain the palm contour auxiliary lines corresponding to each palm skeletal point set; based on the reference step size and skeletal point position corresponding to the same palm skeletal point set, the palm contour points are determined from the palm contour auxiliary lines corresponding to the palm skeletal point set to obtain the contour point positions corresponding to the palm contour points; based on each contour point position, a palm contour corresponding to the target palm image is generated. In this way, there is no need to perform complex data processing on the pixel information of the palm image. Based on the geometric processing of the bone point information corresponding to each palm bone point in the palm image, the contour point position corresponding to the palm contour point can be easily determined from the palm image, and then the palm contour can be generated based on the contour point position corresponding to each palm contour point. The position and type of the palm bone point are not easily affected by the image background. Extracting the palm contour based on the position and type of the palm bone point in the palm image can effectively improve the accuracy of palm contour extraction.

[0021] A control instruction generation method. The method comprises:

[0022] displaying a palm contour determined based on the target palm image;

[0023] generating an object control instruction based on the displayed information of the palm contour;

[0024] The palm contour extraction process comprises the following steps:

[0025] Obtain the skeletal point information corresponding to each palm skeletal point in the target palm image; the skeletal point information includes the skeletal point position and the skeletal point type; based on the skeletal point type, match the each palm skeletal point to obtain a plurality of palm skeletal point sets; the palm skeletal point sets have corresponding geometric processing types; based on the skeletal point positions corresponding to each skeletal point in the palm skeletal point set, generate palm contour auxiliary lines that match the geometric processing type corresponding to the palm skeletal point set, and obtain palm contour auxiliary lines corresponding to each palm skeletal point set; based on the reference step size and skeletal point positions corresponding to the same palm skeletal point set, determine the palm contour points from the palm contour auxiliary lines corresponding to the palm skeletal point set, and obtain the contour point positions corresponding to the palm contour points; based on the contour point positions, generate the palm contour corresponding to the target palm image.

[0026] A control instruction generating device. The device comprises:

[0027] A palm contour display module, used to display a palm contour determined based on a target palm image;

[0028] A control instruction generating module, used for generating an object control instruction based on the displayed information of the palm contour;

[0029] The palm contour extraction process comprises the following steps:

[0030] Obtain the skeletal point information corresponding to each palm skeletal point in the target palm image; the skeletal point information includes the skeletal point position and the skeletal point type; based on the skeletal point type, match the each palm skeletal point to obtain a plurality of palm skeletal point sets; the palm skeletal point sets have corresponding geometric processing types; based on the skeletal point positions corresponding to each skeletal point in the palm skeletal point set, generate palm contour auxiliary lines that match the geometric processing type corresponding to the palm skeletal point set, and obtain palm contour auxiliary lines corresponding to each palm skeletal point set; based on the reference step size and skeletal point positions corresponding to the same palm skeletal point set, determine the palm contour points from the palm contour auxiliary lines corresponding to the palm skeletal point set, and obtain the contour point positions corresponding to the palm contour points; based on the contour point positions, generate the palm contour corresponding to the target palm image.

[0031] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the control instruction generation method when executing the computer program.

[0032] A computer-readable storage medium stores a computer program, which implements the steps of the control instruction generation method when executed by a processor.

[0033] A computer program product comprises a computer program, wherein when the computer program is executed by a processor, the steps of the control instruction generation method are implemented.

[0034] The control instruction generation method, device, computer equipment, storage medium and computer program product generate object control instructions based on the display information of the palm contour by displaying the palm contour determined based on the target palm image. The palm contour extraction process includes the following steps: obtaining the skeletal point information corresponding to each palm skeletal point in the target palm image; the skeletal point information includes the skeletal point position and the skeletal point type; matching each palm skeletal point based on the skeletal point type to obtain multiple palm skeletal point sets; the palm skeletal point sets have corresponding geometric processing types; based on the skeletal point positions corresponding to each skeletal point in the palm skeletal point set, generating palm contour auxiliary lines matching the geometric processing type corresponding to the palm skeletal point set, and obtaining palm contour auxiliary lines corresponding to each palm skeletal point set; based on the reference step size and skeletal point positions corresponding to the same palm skeletal point set, determining the palm contour points from the palm contour auxiliary lines corresponding to the palm skeletal point set, and obtaining the contour point positions corresponding to the palm contour points; generating the palm contour corresponding to the target palm image based on each contour point position. In this way, the palm contour is based on the skeleton point information corresponding to each palm skeleton point in the palm image, and the contour point position corresponding to the palm contour point is determined by geometric processing, and the palm contour is generated based on the contour point position corresponding to each palm contour point. The extraction of the palm contour not only does not require complex data processing of the pixel information of the palm image, but is also not easily disturbed by the image background, and has high accuracy. The accurate palm contour extracted also helps to ensure accurate display, and then helps to improve the accuracy of generating object control instructions. Automatically generating object control instructions based on the display information of the palm contour can also improve the efficiency of generating object control instructions. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A diagram showing an application environment of a palm contour extraction method and a control instruction generation method in one embodiment;

[0036] Figure 2 A schematic diagram of a palm contour extraction method in one embodiment;

[0037] Figure 3 is a schematic diagram of palm bone points in one embodiment;

[0038] Figure 4 A schematic diagram of the movement direction corresponding to the palm contour auxiliary line in one embodiment;

[0039] Figure 5 A schematic diagram of a contour point sorting mark and a palm contour in one embodiment;

[0040] Figure 6 A schematic diagram of a flow chart of a control instruction generating method in one embodiment;

[0041] Fig. 7A A schematic diagram of palm skeleton point recognition results in one embodiment;

[0042] Figure 7B is a schematic diagram of palm contour point recognition results in one embodiment;

[0043] Fig. 8A A schematic diagram of calculating the coordinates of the outer contour of a palm in one embodiment;

[0044] Figure 8B A schematic diagram of calculating the finger gap coordinates in one embodiment;

[0045] Figure 8C A schematic diagram of calculating the edge coordinates of a finger contour in one embodiment;

[0046] Fig.8D is a schematic diagram of a palm outline in one embodiment;

[0047] Fig. 9 A schematic diagram of palm-swipe payment in one embodiment;

[0048] Fig.10 is a structural block diagram of a palm contour extraction device in an embodiment;

[0049] Fig.11 is a structural block diagram of a palm contour extraction device in another embodiment;

[0050] Fig.12 is a structural block diagram of a control instruction generating device in one embodiment;

[0051] Fig.13 is an internal structure diagram of a computer device in one embodiment;

[0052] Fig.14 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] The palm contour extraction method and control instruction generation method provided in the embodiments of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other servers. The terminal 102 can be, but is not limited to, various desktop computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0055] Both the terminal and the server can be used independently to execute the palm contour extraction method and the control instruction generation method provided in the embodiments of the present application.

[0056] For example, the terminal obtains the skeletal point information corresponding to each palm skeletal point in the target palm image, the skeletal point information includes the skeletal point position and the skeletal point type, matches each palm skeletal point based on the skeletal point type, and obtains multiple palm skeletal point sets, and the palm skeletal point sets have corresponding geometric processing types. Based on the skeletal point positions corresponding to each skeletal point in the palm skeletal point set, the terminal generates palm contour auxiliary lines that match the geometric processing type corresponding to the palm skeletal point set, obtains palm contour auxiliary lines corresponding to each palm skeletal point set, and based on the reference step size and skeletal point positions corresponding to the same palm skeletal point set, determines the palm contour points from the palm contour auxiliary lines corresponding to the palm skeletal point set, and obtains the contour point positions corresponding to the palm contour points. The terminal generates the palm contour corresponding to the target palm image based on the positions of each contour point.

[0057] The terminal displays a palm contour determined based on the target palm image, and generates an object control instruction based on display information of the palm contour.

[0058] The terminal and the server can also be used together to execute the palm contour extraction method and the control instruction generation method provided in the embodiments of the present application.

[0059] For example, the server obtains the skeletal point information corresponding to each palm skeletal point in the target palm image from the terminal, the skeletal point information includes the skeletal point position and the skeletal point type, matches each palm skeletal point based on the skeletal point type, and obtains multiple palm skeletal point sets, and the palm skeletal point sets have corresponding geometric processing types. Based on the skeletal point positions corresponding to each skeletal point in the palm skeletal point set, the server generates palm contour auxiliary lines that match the geometric processing type corresponding to the palm skeletal point set, obtains palm contour auxiliary lines corresponding to each palm skeletal point set, and based on the reference step size and skeletal point positions corresponding to the same palm skeletal point set, determines the palm contour points from the palm contour auxiliary lines corresponding to the palm skeletal point set, and obtains the contour point positions corresponding to the palm contour points. The server generates the palm contour corresponding to the target palm image based on the positions of each contour point.

[0060] The terminal obtains a palm contour determined based on the target palm image corresponding to the resource transfer party from the server, and displays the palm contour, and generates an object control instruction based on the display information of the palm contour.

[0061] In one embodiment, Figure 2 As shown, a palm contour extraction method is provided, and the method is applied to a computer device as an example. The computer device can be the above-mentioned Figure 1 The terminal 102 or the server 104 in FIG. Figure 2 ,The palm contour extraction method includes the following steps:

[0062] Step S202, obtaining the skeleton point information corresponding to each palm skeleton point in the target palm image; the skeleton point information includes the skeleton point position and the skeleton point type.

[0063] The target palm image refers to the palm image from which the palm contour is to be extracted. The palm image refers to an image containing the palm. The palm image may contain only the palm, for example, the palm image is an image obtained by photographing the palm area. The palm image may also contain more body parts, for example, the palm image is obtained by photographing the half body of an object holding up the palm. The palm image may be obtained by real-time photography, for example, a real-time photograph of the palm is used as the palm image; a video frame containing the palm in a real-time video is used as the palm image. The palm image may also be obtained from pre-photographed data, for example, a palm photo stored in an album is used as the palm image; a video frame containing the palm in a pre-edited video is used as the palm image. In addition, the palm image may be obtained by photographing a real palm or a virtual palm.

[0064] Palm skeleton points refer to the skeleton points in the palm. The palm includes multiple palm skeleton points. By performing palm skeleton point recognition on the palm image, the skeleton point information corresponding to each palm skeleton point in the palm image can be obtained. The skeleton point information includes the skeleton point position and the skeleton point type. The skeleton point position is used to identify the position of the palm skeleton point in the palm image, and the skeleton point type is used to identify the type of the palm skeleton point.

[0065] Specifically, the computer device can obtain the target palm image locally or from other devices, perform palm skeleton point recognition on the target palm image in real time to obtain the skeleton point information corresponding to each palm skeleton point in the target palm image, or directly obtain the skeleton point information corresponding to each palm skeleton point in the target palm image that has been pre-recognized locally or from other devices. Furthermore, the computer device can perform palm contour recognition on the target palm image based on the skeleton point information corresponding to each palm skeleton point to obtain the palm contour in the target palm image.

[0066] Step S204, matching each palm skeleton point based on the skeleton point type to obtain multiple palm skeleton point sets; the palm skeleton point sets have corresponding geometric processing types.

[0067] Among them, the palm skeleton point set includes at least two palm skeleton points. Each palm skeleton point set has a corresponding geometric processing type. The geometric processing type is used to determine the geometric processing method corresponding to the palm skeleton point set. By performing geometric processing on the palm skeleton point set through the geometric processing method corresponding to the palm skeleton point set, the palm contour points corresponding to the palm skeleton point set can be obtained. It can be understood that the same palm skeleton point set can correspond to at least one geometric processing type.

[0068] Specifically, the computer device can match each palm skeleton point based on the skeleton point type, and form the successfully matched palm skeleton points into a palm skeleton point set, and finally obtain multiple palm skeleton point sets. Each palm skeleton point set has a corresponding geometric processing type.

[0069] In one embodiment, the computer device can obtain the bone point type matching information, which records the matching relationship between the bone point types. Based on the bone point type matching information, each palm bone point can be quickly matched based on the bone point type to quickly obtain each palm bone point set. In one embodiment, the bone point type matching information can also include the geometric processing type corresponding to each matching relationship, so that when the palm bone point set is determined based on the matching relationship, the geometric processing type corresponding to the palm bone point set can be synchronously determined.

[0070] In one embodiment, the geometric processing type corresponding to the palm skeletal point set can be determined based on the skeletal point type corresponding to each palm skeletal point in the palm skeletal point set. For example, at least one target palm skeletal point type corresponding to each geometric processing type is pre-set. If the palm skeletal point set includes the target palm skeletal type, the geometric processing type corresponding to the target palm skeletal point type is used as the geometric processing type corresponding to the palm skeletal point set. In one embodiment, the geometric processing type corresponding to the palm skeletal point set can be determined based on the skeletal point position corresponding to each palm skeletal point in the palm skeletal point set. For example, the palm part in the palm image is divided into multiple palm areas, each of which has a corresponding geometric processing type, and the geometric processing type corresponding to the palm area where the palm skeletal point set falls is used as the geometric processing type corresponding to the palm skeletal point set.

[0071] In one embodiment, there is a connection relationship between the palm skeletal points. When identifying the palm skeletal points, the connection relationship between the palm skeletal points and the skeletal point information of the palm skeletal points can be synchronously identified. The computer device can first perform preliminary matching on each palm skeletal point based on the connection relationship to obtain multiple first palm skeletal point sets, and then perform supplementary matching on each palm skeletal point based on the skeletal point type to obtain multiple second palm skeletal point sets, and finally, obtain a palm skeletal point set based on each first palm skeletal point set and the second palm skeletal point set.

[0072] For example, see Figure 3 , Figure 3 The dots in the figure represent the palm bone points, which include 21 palm bone points in total. Figure 3 The lines between the middle dots represent the connection relationship between the palm bone points. The computer device can group the adjacent palm bone points that are in a connection relationship into a first palm bone point set, for example, grouping dots 0 and 1 into a first palm bone point set, and grouping dots 1 and 2 into a first palm bone point set. The computer device can group the palm bone points corresponding to the preset bone point type into a second palm bone point set, for example, the bone point type corresponding to dots 13 and 17 is the preset bone point type, and dots 13 and 17 can be grouped into a second palm bone point set.

[0073] Step S206, based on the bone point positions corresponding to each bone point in the palm bone point set, generate palm contour auxiliary lines matching the geometric processing type corresponding to the palm bone point set, and obtain palm contour auxiliary lines corresponding to each palm bone point set.

[0074] Among them, the palm contour auxiliary line is a geometric line used to determine the palm contour points.

[0075] Specifically, after determining each palm bone point set, for any palm bone point set, the computer device can generate a palm contour auxiliary line that matches the geometric processing type corresponding to the palm bone point set based on the bone point position corresponding to each bone point in the palm bone point set. The computer device can determine the geometric processing method and the geometric processing algorithm based on the geometric processing type corresponding to the palm bone point set, perform data processing on the bone point position corresponding to each bone point in the palm bone point set through the geometric processing algorithm, and generate a palm contour auxiliary line corresponding to the palm bone point set. Finally, the computer device can obtain the palm contour auxiliary lines corresponding to each palm bone point set.

[0076] It can be understood that a palm skeleton point set can correspond to at least one palm contour auxiliary line.

[0077] Step S208, based on the reference step size and the skeleton point position corresponding to the same palm skeleton point set, palm contour points are determined from the palm contour auxiliary lines corresponding to the palm skeleton point set to obtain the contour point positions corresponding to the palm contour points.

[0078] The reference step length refers to the moving step length on the palm contour auxiliary line. Each palm skeleton point set has its own corresponding reference step length. The contour point position is used to identify the position of the palm contour point.

[0079] Specifically, for any palm skeleton point set, the computer device can determine the palm contour point from the palm contour auxiliary line corresponding to the palm skeleton point set based on the skeleton point position corresponding to each palm skeleton point in the palm skeleton point set and the reference step corresponding to the palm skeleton point set, thereby obtaining the contour point position corresponding to the palm contour point. The computer device can determine the moving starting point from the palm contour auxiliary line corresponding to the palm skeleton point set based on the skeleton point position corresponding to each palm skeleton point in the palm skeleton point set, start moving the reference step from the moving starting point on the palm contour auxiliary line corresponding to the palm skeleton point set, and use the moving end point obtained by moving the reference step as the palm contour point. Finally, the computer device can obtain multiple palm contour points and the contour point positions corresponding to each palm contour point.

[0080] In one embodiment, a palm image with a marked palm contour can be obtained as a reference palm image, and a universal reference step length is determined based on the reference palm image. Specifically, a palm contour auxiliary line corresponding to the reference palm image is determined based on the reference palm image, and the intersection between the palm contour auxiliary line and the marked palm contour is used as a reference contour point. The reference contour point is an accurate palm contour point, and an accurate moving step length is determined based on the distance between the moving starting point and the reference contour point, and the accurate moving step length is used as the reference step length. The computer device can obtain multiple reference palm images, and the same reference step length determined based on different reference palm images is counted to obtain a universal reference step length. For example, the average value of the reference step length can be calculated as the universal reference step length. When processing a palm image with an unknown palm contour later, the universal reference step length can be directly obtained to determine the palm contour point in the palm image, or the universal reference step length can be fine-tuned based on the personalized information of the palm image to obtain a target reference step length, and the palm contour point in the palm image can be determined based on the target reference step length. The personalized information of the palm image includes at least one data such as palm size information and palm depth information.

[0081] In one embodiment, the reference step increases as the palm size information corresponding to the target palm image increases. The palm size information is used to indicate the size of the palm in the target palm image. For a larger palm, more reference steps need to be moved on the palm contour auxiliary line to reach the accurate palm contour point. In one embodiment, the palm size information includes at least one of palm length information and palm width information.

[0082] In one embodiment, the reference step increases as the palm depth information corresponding to the target palm image decreases. The palm depth information is used to indicate the distance between the palm and the image acquisition device. The closer the distance, the smaller the palm depth information, and the farther the distance, the larger the palm depth information. It can be understood that the smaller the palm depth information corresponding to the target palm image, the closer the distance between the palm and the image acquisition device, and the larger the palm area presented in the target palm image. For a larger palm area, more reference steps need to be moved on the palm contour auxiliary line to reach the accurate palm contour point. In one embodiment, the target palm image can be a depth map, and each pixel value of the depth map represents the distance between a point in the shooting scene and the image acquisition device.

[0083] Step S210: generating a palm contour corresponding to the target palm image based on the positions of the contour points.

[0084] Specifically, after determining the contour point positions corresponding to the respective palm contour points, the computer device may connect the respective contour point positions, thereby obtaining the palm contour corresponding to the target palm image.

[0085] In the palm contour extraction method, the skeletal point information corresponding to each palm skeletal point in the target palm image is obtained; the skeletal point information includes the skeletal point position and the skeletal point type; based on the skeletal point type, each palm skeletal point is matched to obtain multiple palm skeletal point sets; the palm skeletal point set has a corresponding geometric processing type; based on the skeletal point position corresponding to each skeletal point in the palm skeletal point set, a palm contour auxiliary line matching the geometric processing type corresponding to the palm skeletal point set is generated to obtain the palm contour auxiliary lines corresponding to each palm skeletal point set; based on the reference step and skeletal point position corresponding to the same palm skeletal point set, the palm contour point is determined from the palm contour auxiliary line corresponding to the palm skeletal point set to obtain the contour point position corresponding to the palm contour point; based on the contour point position, the palm contour corresponding to the target palm image is generated. In this way, there is no need to perform complex data processing on the pixel information of the palm image, and based on the geometric processing of the skeletal point information corresponding to each palm skeletal point in the palm image, the contour point position corresponding to the palm contour point can be conveniently determined from the palm image, and then the palm contour is generated based on the contour point position corresponding to each palm contour point. The position and type of palm skeleton points are not easily affected by the image background. Extracting the palm contour based on the position and type of palm skeleton points in the palm image can effectively improve the accuracy of palm contour extraction.

[0086] In one embodiment, the bone point type is determined by the finger parameters and the interphalangeal joint parameters. Based on the bone point type, each palm bone point is matched to obtain multiple palm bone point sets, including:

[0087] Establish a matching relationship between adjacent palm bone points whose interdigital joint parameters are preset joint parameters to obtain a set of palm bone points corresponding to the first geometric processing type; establish a matching relationship between adjacent palm bone points corresponding to the same finger parameters to obtain a set of palm bone points corresponding to the second geometric processing type; use the palm bone point corresponding to the preset bone point type as the preset bone point, establish a matching relationship between the preset bone point and the palm bone point closest to the preset bone point, and obtain a set of palm bone points corresponding to the third geometric processing type.

[0088] The type of the bone point is determined by the finger parameter and the interphalangeal joint parameter. The finger parameter is used to indicate the finger to which the palm bone point belongs. Based on the finger parameter, the palm bone points located on different fingers can be distinguished. The interphalangeal joint parameter is used to indicate the joint to which the palm bone point belongs in the finger. Based on the interphalangeal joint parameter, different palm bone points located on the same finger can be distinguished.

[0089] The preset joint parameters are pre-set, specific inter-finger joint parameters. The preset bone point type is a preset, specific bone point type. Adjacent palm bone points refer to two palm bone points that are adjacent in position.

[0090] Specifically, when matching the palm bone points, the computer device can obtain the adjacent palm bone points whose interphalangeal joint parameters are preset joint parameters from each palm bone point, and establish a matching relationship between the adjacent palm bone points whose interphalangeal joint parameters are preset joint parameters, thereby obtaining each palm bone point set corresponding to the first geometric processing type. Figure 3 , the intraphalangeal joint parameters corresponding to points 13, 17, 9, and 5 are metacarpophalangeal joints, that is, points 13, 17, 9, and 5 are metacarpophalangeal joints corresponding to different fingers. If the preset joint parameters are metacarpophalangeal joints, a matching relationship is established between the adjacent points 13 and 17 to obtain a palm bone point set A1 corresponding to the first geometric processing type, a matching relationship is established between the adjacent points 17 and 9 to obtain a palm bone point set A2 corresponding to the first geometric processing type, and a matching relationship is established between the adjacent points 9 and 5 to obtain a palm bone point set A3 corresponding to the first geometric processing type.

[0091] When matching the palm skeleton points, the computer device can obtain adjacent palm skeleton points with the same finger parameters from each palm skeleton point, establish a matching relationship between adjacent palm skeleton points corresponding to the same finger parameters, and thus obtain a set of palm skeleton points corresponding to the second geometric processing type. Figure 3 , the finger parameters corresponding to points 1, 2, 3, and 4 are the thumb, that is, points 1, 2, 3, and 4 are different joints on the same finger, and a matching relationship is established between the adjacent points 1 and 2 to obtain a palm skeleton point set B1 corresponding to the second geometric processing type, and a matching relationship is established between the adjacent points 2 and 3 to obtain a palm skeleton point set B2 corresponding to the second geometric processing type, and a matching relationship is established between the adjacent points 3 and 4 to obtain a palm skeleton point set B3 corresponding to the second geometric processing type.

[0092] When matching the palm bone points, the computer device can obtain the palm bone points corresponding to the preset bone point type from each palm bone point as the preset bone point, obtain the palm bone point closest to the preset bone point, and establish a matching relationship between the preset bone point and the palm bone point closest to the preset bone point, thereby obtaining each palm bone point set corresponding to the third geometric processing type. Figure 3 If the bone point type corresponding to point 0 is the preset bone point type, then point 0 is the preset bone point, and point 1 is the palm bone point closest to point 0. A matching relationship is established between point 0 and point 1 to obtain a palm bone point set C corresponding to the third geometric processing type.

[0093] In the above embodiment, a matching relationship is established between adjacent palm bone points whose intraphalangeal joint parameters are preset joint parameters to obtain a set of palm bone points corresponding to the first geometric processing type; a matching relationship is established between adjacent palm bone points corresponding to the same finger parameters to obtain a set of palm bone points corresponding to the second geometric processing type; a palm bone point corresponding to a preset bone point type is used as a preset bone point, and a matching relationship is established between the preset bone point and the palm bone point closest to the preset bone point to obtain a set of palm bone points corresponding to the third geometric processing type. The bone point type is determined by the finger parameters and the intraphalangeal joint parameters, and a matching relationship can be quickly established through the finger parameters, the intraphalangeal joint parameters and the bone point type.

[0094] In one embodiment, the first geometry processing type is used to determine the palm contour points on the finger gaps and the palm outer contour. The second geometry processing type is used to determine the palm contour points on the finger contour edge. The third geometry processing type is used to determine the palm contour points on the finger contour edge and the palm outer contour.

[0095] In one embodiment, based on the skeletal point positions corresponding to each skeletal point in the palm skeletal point set, a palm contour auxiliary line matching the geometric processing type corresponding to the palm skeletal point set is generated, and the palm contour auxiliary lines corresponding to each palm skeletal point set are obtained, including:

[0096] Determine the current auxiliary line type based on the geometric processing type corresponding to the current palm bone point set; generate the current bone point line segment corresponding to the current palm bone point set based on the bone point position corresponding to each palm bone point in the current palm bone point set; based on the current auxiliary line type, perform geometric processing on the current bone point line segment to obtain the palm contour auxiliary line corresponding to the current auxiliary line type.

[0097] Among them, the auxiliary line type is used to determine the type of auxiliary line. The current auxiliary line type refers to the auxiliary line type of the palm contour auxiliary line corresponding to the current palm bone point set. The bone point line segment is a line segment composed of palm bone points. The current bone point line segment refers to the bone point line segment corresponding to the current palm bone point set. Geometry processing is used to generate a specific type of palm contour auxiliary line based on the bone point line segment.

[0098] The current palm bone point set can be any palm bone point set.

[0099] Specifically, when generating a palm contour auxiliary line, the computer device may first determine the auxiliary line type corresponding to the palm contour auxiliary line, and then generate the corresponding palm contour auxiliary line. The computer device may determine the current auxiliary line type based on the geometric processing type corresponding to the current palm bone point set, and each geometric processing type has a corresponding auxiliary line type. The computer device may generate the current bone point line segment corresponding to the current palm bone point set based on the bone point position corresponding to each palm bone point in the current palm bone point set. For example, two points determine a line segment, and two palm bone points may be obtained from the current palm bone point set, and the bone point positions corresponding to the two palm bone points are connected to generate the current bone point line segment. The present application does not limit the order of determining the current auxiliary line type and generating the current bone point line segment. Finally, the computer device may perform geometric processing on the current bone point line segment based on the current auxiliary line type, so as to obtain the palm contour auxiliary line corresponding to the current auxiliary line type. For example, if the current auxiliary line type is an extension line, the extension line of the current bone point line segment may be used as the palm contour auxiliary line.

[0100] In the above embodiment, the current auxiliary line type is determined based on the geometric processing type corresponding to the current palm bone point set; based on the bone point positions corresponding to each palm bone point in the current palm bone point set, the current bone point line segments corresponding to the current palm bone point set are generated; based on the current auxiliary line type, the current bone point line segments are geometrically processed to obtain the palm contour auxiliary lines corresponding to the current auxiliary line type. The auxiliary line type is first determined, and then the bone point line segments generated according to the palm bone points are geometrically processed based on the auxiliary line type, so that the corresponding palm contour auxiliary lines can be quickly generated.

[0101] In one embodiment, the geometry processing type includes any one of a first geometry processing type, a second geometry processing type, and a third geometry processing type, the auxiliary line type corresponding to the first geometry processing type includes at least one of a perpendicular bisector type and an extended line type, the auxiliary line type corresponding to the second geometry processing type includes a normal line type, and the auxiliary line type corresponding to the third geometry processing type includes at least one of a normal line type and an extended line type.

[0102] Among them, the geometric processing type includes any one of the first geometric processing type, the second geometric processing type and the third geometric processing type. The auxiliary line type corresponding to the first geometric processing type includes at least one of the perpendicular median type and the extended line type. The palm contour auxiliary line corresponding to the perpendicular median type is the perpendicular median. The perpendicular median refers to a straight line passing through the midpoint of a line segment and perpendicular to the line segment. The palm contour auxiliary line corresponding to the extended line type is the extended line, and the extended line refers to a straight line obtained by extending from a certain end point on the basis of a line segment. The auxiliary line type corresponding to the second geometric processing type includes the normal line type. The palm contour auxiliary line corresponding to the normal line type is the normal line. The normal of a line segment can also be called the perpendicular line of the line segment, and the line segment and the corresponding normal line are perpendicular to each other. The auxiliary line type corresponding to the third geometric processing type includes at least one of the normal line type and the extended line type.

[0103] In the above embodiment, different geometric processing types correspond to different auxiliary line types, and the same geometric processing type can correspond to at least one auxiliary line type, which can effectively increase the number of palm contour points and thus improve the accuracy of palm contour extraction.

[0104] In one embodiment, based on the current auxiliary line type, the current skeleton point line segment is geometrically processed to obtain a palm contour auxiliary line corresponding to the current auxiliary line type, including:

[0105] When the current auxiliary line type is the perpendicular median type, the perpendicular median corresponding to the current bone point segment is used as the palm contour auxiliary line; when the current auxiliary line type is the extension line type, the target bone point is determined from the various palm bone points corresponding to the current palm bone point set, and the extension line corresponding to the current bone point segment and starting from the target bone point is used as the palm contour auxiliary line; when the current auxiliary line type is the normal line type, the target bone point is determined from the various palm bone points corresponding to the current palm bone point set, and the normal of the current bone point segment at the target bone point is used as the palm contour auxiliary line.

[0106] Specifically, if the current auxiliary line type is a perpendicular median line type, the computer device may use the perpendicular median line corresponding to the current skeleton point line segment as the palm contour auxiliary line.

[0107] If the current auxiliary line type is an extension line type, the computer device can select a palm bone point from each palm bone point corresponding to the current palm bone point set as the target bone point, and use the extension line corresponding to the current bone point line segment and starting from the target bone point as the palm contour auxiliary line. When determining the target bone point, the computer device can use the palm bone point that is closer to the palm center as the target bone point, or the computer device can use the palm bone point of the specified bone point type in the current palm bone point set as the target bone point.

[0108] If the current auxiliary line type is a normal line type, the computer device can select a palm bone point from each palm bone point corresponding to the current palm bone point set as the target bone point, and use the normal of the current bone point line segment at the target bone point as the palm contour auxiliary line. When determining the target bone point, the computer device can use each palm bone point as the target bone point, and then generate multiple palm contour auxiliary lines. The computer device can also select a palm bone point as the target bone point, and then generate a palm contour auxiliary line. When selecting a palm bone point, it can be randomly selected, or a palm bone point closer to the fingertip can be selected.

[0109] In the above embodiment, when the current auxiliary line type is a perpendicular median type, the perpendicular median corresponding to the current bone point line segment is used as the palm contour auxiliary line; when the current auxiliary line type is an extension line type, the target bone point is determined from each palm bone point corresponding to the current palm bone point set, and the extension line corresponding to the current bone point line segment and starting from the target bone point is used as the palm contour auxiliary line; when the current auxiliary line type is a normal line type, the target bone point is determined from each palm bone point corresponding to the current palm bone point set, and the normal line of the current bone point line segment at the target bone point is used as the palm contour auxiliary line. For different auxiliary line types, different geometric methods can be used to quickly generate corresponding palm contour auxiliary lines.

[0110] In one embodiment, based on the reference step length and the skeletal point position corresponding to the same palm skeletal point set, palm contour points are determined from the palm contour auxiliary lines corresponding to the palm skeletal point set to obtain the contour point positions corresponding to the palm contour points, including:

[0111] Based on the bone point positions corresponding to each bone point in the current palm bone point set, generate the current bone point line segment corresponding to the current palm bone point set; determine the reference point from the current bone point line segment; on the current palm contour auxiliary line corresponding to the current palm bone point set, take the reference point as the starting point, move the reference step corresponding to the current palm contour auxiliary line in the moving direction corresponding to the current palm contour auxiliary line, and obtain the current palm contour point and the contour point position corresponding to the current palm contour point.

[0112] The current palm contour auxiliary line refers to the palm contour auxiliary line corresponding to the current palm skeleton point set.

[0113] Specifically, when determining the palm contour point, the computer device can determine the reference point from the skeleton point line segment, take the reference point as the starting point on the palm contour auxiliary line, move the step length according to the moving direction, and use the moving end point as the palm contour point. First, the computer device can generate the current skeleton point line segment corresponding to the current palm skeleton point set according to the skeleton point position corresponding to each skeleton point in the current palm skeleton point set. Then, the computer device can determine the reference point from the current skeleton point line segment, for example, any endpoint of the current skeleton point line segment can be used as the reference point, or the intersection between the current skeleton point line segment and the current palm contour auxiliary line can be used as the reference point. Then, taking the reference point as the starting point, according to the moving direction corresponding to the current palm contour auxiliary line, the computer device moves the reference step length corresponding to the current palm skeleton point set on the current palm contour auxiliary line, thereby determining the current palm contour point and the contour point position corresponding to the current palm contour point. Different types of palm contour auxiliary lines have their own corresponding moving directions, and the same type of palm contour auxiliary lines correspond to at least one moving direction.

[0114] In one embodiment, the reference step increases as the length of the line segment corresponding to the current skeleton point line segment increases. The length of the line segment corresponding to the current skeleton point line segment can reflect the personalized information of the palm from the side. The longer the line segment length, the larger the palm. For a larger palm, more reference steps need to be moved on the palm contour auxiliary line to reach the accurate palm contour point.

[0115] In the above embodiment, based on the skeletal point positions corresponding to each skeletal point in the current palm skeletal point set, a current skeletal point line segment corresponding to the current palm skeletal point set is generated; a reference point is determined from the current skeletal point line segment; on the current palm contour auxiliary line corresponding to the current palm skeletal point set, the reference point is used as a starting point, and the reference step corresponding to the current palm contour auxiliary line is moved in the moving direction corresponding to the current palm contour auxiliary line, to obtain the current palm contour point and the contour point position corresponding to the current palm contour point. In this way, on the palm contour auxiliary line, taking the reference point determined according to the palm skeletal point as a starting point, and moving the corresponding reference step in the moving direction corresponding to the palm contour auxiliary line, an accurate palm contour point can be quickly obtained.

[0116] In one embodiment, the current palm contour auxiliary line includes at least one of a perpendicular bisector, an extension line and a normal line. The moving direction corresponding to the perpendicular bisector is the opposite direction of the palm, the moving direction corresponding to the extension line is the extension direction of the extension line, and the moving direction corresponding to the normal line is the vertical direction of the current skeleton point line segment.

[0117] The current palm contour auxiliary line includes at least one of a perpendicular bisector, an extension line and a normal line.

[0118] The moving direction of the median line is opposite to the palm center, which is the direction away from the palm center. That is, on the median line type palm contour auxiliary line, it is necessary to move the reference step in the direction away from the palm center to determine the palm contour point. For example, reference Figure 4 In (a), the dotted line in (a) represents the perpendicular midline, the arrow represents the opposite direction of the palm, and the dot 15' represents the palm contour point determined by moving the reference step in the opposite direction of the palm on the perpendicular midline.

[0119] The moving direction of the extension line is the extension direction of the extension line. That is, on the palm contour auxiliary line of the extension line type, it is necessary to move the reference step in the extension direction of the extension line to determine the palm contour point. For example, reference Figure 4 In (b), the dotted line in (b) represents the extension line, the arrow represents the extension direction of the extension line, and the dot 36' represents the palm contour point determined by moving the reference step length on the extension line in the extension direction of the extension line.

[0120] The moving direction corresponding to the normal line is the vertical direction of the current bone point segment. That is, on the palm contour auxiliary line of the normal type, it is necessary to move the reference step in the vertical direction of the bone point segment to determine the palm contour point. For example, reference Figure 4 In (c), the dotted line in (c) represents the normal line, the arrow represents the vertical direction of the normal line, and the dots 3' and 6' represent the palm contour points determined by moving the reference step length in the vertical direction of the skeleton point line segment on the normal line.

[0121] In the above embodiment, the moving direction corresponding to the perpendicular bisector is the opposite direction of the palm, the moving direction corresponding to the extension line is the extension direction of the extension line, and the moving direction corresponding to the normal line is the vertical direction of the current skeleton point line segment. Different types of palm contour auxiliary lines correspond to different moving directions.

[0122] In one embodiment, each palm skeleton point has a corresponding skeleton point identifier. Generating a palm contour corresponding to the target palm image based on the position of each contour point includes:

[0123] Based on preset mapping information, the bone point identifiers corresponding to the palm bone points are mapped to obtain the contour point sorting identifiers corresponding to each palm contour point; the preset mapping information includes the identifier mapping relationship between the palm bone points in the palm bone point set and the palm contour points generated based on the palm bone point set; according to the contour point sorting identifiers, the contour point positions corresponding to each palm contour point are connected in order to obtain the palm contour.

[0124] The skeleton point identifier is used to uniquely identify the palm skeleton point, and the contour point sorting identifier is used to identify the arrangement order of the palm contour points.

[0125] The preset mapping information records the identification mapping relationship between the palm skeleton points in the palm skeleton point set and the palm contour points generated based on the palm skeleton point set. For example, the palm contour points generated based on palm skeleton point 0 and palm skeleton point 1 are marked as 1', and the palm contour points generated based on palm skeleton point 1 and palm skeleton point 2 are marked as 2' and 7'.

[0126] Specifically, when generating a palm contour based on the contour point position, each palm contour point can be connected in order to obtain the palm contour. When determining the sorting information of the palm contour point, the computer device can obtain preset mapping information, map the bone point identifiers corresponding to the palm bone points, generate the contour point identifiers corresponding to the palm contour points generated based on the palm bone points, and finally obtain the contour point sorting identifiers corresponding to each palm contour point. Furthermore, the computer device can connect the contour point positions corresponding to each palm contour point in order according to the contour point sorting identifiers, so as to obtain the correct palm contour.

[0127] For example, see Figure 5 , Figure 5 The black dots in the figure represent the palm bone points, and the white dots represent the palm contour points. Figure 5 (a) in FIG. 5 represents the target palm image, and palm contour points 1'-38' can be obtained based on palm skeleton points 0-20. Figure 5 (b) in the figure represents the palm contour, and the palm contour can be obtained by connecting the palm contour points 1'-38' in sequence.

[0128] In one embodiment, when the contour point positions are connected, they can be connected smoothly or in a straight line.

[0129] In the above embodiment, based on the preset mapping information, the bone point identifiers corresponding to the palm bone points are mapped to obtain the contour point sorting identifiers corresponding to each palm contour point; the preset mapping information includes the identifier mapping relationship between the palm bone points in the palm bone point set and the palm contour points generated based on the palm bone point set; according to the contour point sorting identifiers, the contour point positions corresponding to each palm contour point are connected in order to obtain the palm contour. In this way, the contour point sorting identifiers corresponding to the palm contour points can be quickly determined through the preset mapping information, so that the palm contour points can be connected in order to generate an accurate palm contour based on the contour point sorting identifiers.

[0130] In one embodiment, obtaining the bone point information corresponding to each palm bone point in the target palm image includes:

[0131] The target palm image is input into the target palm bone point recognition model to obtain the bone point information corresponding to each palm bone point.

[0132] The target palm skeleton point recognition model refers to a trained palm skeleton point recognition model. The input data of the palm skeleton point recognition model is a palm image, and the output data is the skeleton point information corresponding to each palm skeleton point in the palm image.

[0133] Specifically, the computer device can identify palm bone points based on a machine learning model. The computer device can obtain a pre-trained target palm bone point recognition model, input the target palm image into the target palm bone point recognition model, and through data processing of the model, the model finally outputs the bone point information corresponding to each palm bone point.

[0134] The training process of the target palm skeleton point recognition model includes the following steps: obtaining a training data set; the training data set includes multiple training palm images and training skeleton point information corresponding to each training palm image; determining the current palm image from the training data set; inputting the current palm image into the initial palm skeleton point recognition model to obtain the predicted skeleton point information corresponding to the current palm image; based on the difference between the predicted skeleton point information and the training skeleton point information corresponding to the current palm image, adjusting the model parameters corresponding to the initial palm skeleton point recognition model to obtain an intermediate skeleton point recognition model; taking the next training palm image as the current palm image, taking the intermediate skeleton point recognition model as the initial palm skeleton point recognition model, returning to the step of inputting the current palm image into the initial palm skeleton point recognition model, and executing until the convergence condition is met to obtain the target skeleton point recognition model.

[0135] The initial palm bone point recognition model refers to the palm bone point recognition model to be trained. The training palm image refers to a palm image with known bone point information. The current palm image refers to any palm image in the training data set. The training bone point information refers to accurate bone point information. The predicted bone point information refers to the bone point information predicted by the model.

[0136] Specifically, during model training, the computer device can obtain a training data set, and perform model training on the initial palm skeleton point recognition model based on the training data set to obtain a target palm skeleton point recognition model. The target training image in the training data set is used as input data of the palm skeleton point recognition model, and the training skeleton point information is used as expected output data of the palm skeleton point recognition model. The training goal of the model is to make the actual output data of the model and the expected output data as close as possible, so that the model can output predicted skeleton point information close to the training skeleton point information and correctly identify the palm skeleton points in the palm image.

[0137] The computer device can randomly select a training palm image from the training data set as the current palm image, input the current palm image into the initial palm skeleton point recognition model, and through the data processing of the model, the model outputs the predicted skeleton point information corresponding to the current palm image. Furthermore, the computer device generates loss information based on the difference between the predicted skeleton point information and the training skeleton point information corresponding to the current palm image, performs back propagation based on the loss information, and adjusts the model parameters of the initial palm skeleton point recognition model, thereby obtaining an intermediate palm skeleton point recognition model. The computer device obtains the next training palm image as a new training palm image, uses the intermediate palm skeleton point recognition model as a new initial palm skeleton point recognition model, and returns to the step of inputting the current palm image into the initial palm skeleton point recognition model to obtain the predicted skeleton point information corresponding to the current palm image, so as to perform model iterative training, and continuously adjusts the model parameters through multiple model iterative training until the second convergence condition is met, and the target palm skeleton point recognition model is obtained.

[0138] The second convergence condition may be at least one of the following conditions: the loss information is less than a preset threshold, the number of model iterations is greater than a preset number, etc. For example, if in a certain round of training, the loss information calculated based on the difference between the predicted bone point information and the training bone point information is less than a preset threshold, the adjustment of the model parameters is stopped, and the palm bone point recognition model obtained by the latest adjustment is used as the target palm bone point recognition model. If the number of model iterations after a certain round of training is greater than a preset number, the palm bone point recognition model obtained by the latest adjustment is used as the target palm bone point recognition model.

[0139] It can be understood that the current palm image determined in each round of training may include at least one training palm image.

[0140] In the above embodiment, palm skeleton points in a palm image are identified by using a target palm skeleton point recognition model, which can ensure recognition accuracy and recognition efficiency. An accurate target palm skeleton point recognition model can be trained through a supervised iterative training method.

[0141] In one embodiment, the palm outline is used to generate the palm animation.

[0142] Specifically, after the palm contour is generated, a palm animation may be generated based on the palm contour to display the palm animation.

[0143] In one embodiment, the computer device can collect palm images in real time, identify the palm contour in the palm image in real time, and display the dynamic palm contour to form a dynamic palm animation. In one embodiment, the computer device can set the display parameters of the palm contour to generate the palm animation. For example, the palm contour is displayed with dynamically changing gradient colors. In one embodiment, the computer device can set the display parameters of the closed area formed by the palm contour to generate the palm animation. For example, the closed area formed by the palm contour is displayed with dynamically changing rainbow colors.

[0144] In the above embodiment, the palm contour extracted from the palm image can be used to generate a palm animation to improve the fun and interactivity.

[0145] In one embodiment, Figure 6 As shown, a control instruction generation method is provided, and the method is applied to a terminal as an example. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the control instruction generation method includes the following steps:

[0146] Step S602: displaying the palm contour determined based on the target palm image.

[0147] Among them, the palm contour extraction process includes the following steps: obtaining the bone point information corresponding to each palm bone point in the target palm image; the bone point information includes the bone point position and the bone point type; based on the bone point type, matching each palm bone point to obtain multiple palm bone point sets; the palm bone point set has a corresponding geometric processing type; based on the bone point position corresponding to each bone point in the palm bone point set, generating palm contour auxiliary lines matching the geometric processing type corresponding to the palm bone point set, and obtaining palm contour auxiliary lines corresponding to each palm bone point set; based on the reference step size and bone point position corresponding to the same palm bone point set, determining the palm contour points from the palm contour auxiliary lines corresponding to the palm bone point set, and obtaining the contour point positions corresponding to the palm contour points; generating the palm contour corresponding to the target palm image based on each contour point position.

[0148] It can be understood that the specific process of extracting the palm contour can refer to the contents of the various embodiments of the aforementioned palm contour extraction method, which will not be repeated here.

[0149] Step S604: generating an object control instruction based on the displayed information of the palm contour.

[0150] The display information is used to reflect the display status of the palm contour, and includes at least one of display position, display size, display depth, display posture, etc.

[0151] Object control instructions refer to instructions for controlling and manipulating objects. The controlled object may be a virtual object, such as a virtual resource, a virtual character, a virtual control, etc. The controlled object may also be a physical object, such as an electronic device, a mechanical device, etc. The object control instruction may be a variety of control instructions. In one embodiment, the object control instruction may be an instruction to turn on or off one or some functions of the object, for example, the object control instruction may be a light-on instruction, a light-off instruction, a test start instruction, a test stop instruction, etc. In one embodiment, the object control instruction may be an instruction to control the movement of the object, for example, the object control instruction may be a virtual resource transfer instruction, a virtual character jump instruction, etc. In one embodiment, the object control instruction may be an instruction to control the adjustment of the attribute parameters of the object, for example, the object control instruction may be a brush color adjustment instruction, a window size adjustment instruction, etc.

[0152] Specifically, the terminal displays a palm contour determined based on the target palm image, and generates an object control instruction based on the display information of the palm contour. The terminal can autonomously execute the object control instruction and control the corresponding object through the object control instruction. The terminal can also send the object control instruction to other devices and instruct other devices to control the object through the object control instruction.

[0153] In one embodiment, if the display information of the palm contour meets the preset conditions, an object control instruction is generated; if the display information of the palm contour does not meet the preset conditions, a prompt information is generated. For example, if the position difference between the display position of the palm contour and the target display position is less than a preset threshold, an object control instruction is generated; or if the display size of the palm contour is greater than the preset size, an object control instruction is generated. Or if the display depth difference of the palm contour is less than the preset difference, an object control instruction is generated; and so on. For example, the closed area formed by the palm contour can be divided into at least two sub-areas, and the depth average values ​​corresponding to each sub-area are counted, and the data differences between each depth average values ​​are counted to obtain the display depth difference. If the display depth difference is less than the preset difference, it indicates that the depth values ​​corresponding to each pixel point in the palm are not much different, and the palm is placed correctly, then an object control instruction is generated. Among them, the prompt information can be a prompt that the current palm is placed incorrectly, or it can be a prompt to move the palm.

[0154] In one embodiment, an object control instruction corresponding to the displayed information can be generated based on the displayed information of the palm contour. For example, a corresponding object control instruction can be generated based on the displayed posture of the palm contour. For example, if the displayed posture of the palm contour is fingers open, a function start instruction is generated, and if the displayed posture corresponding to the palm contour is fingers close together, a function close instruction is generated.

[0155] In the control instruction generation method, the object control instruction is generated based on the display information of the palm contour by displaying the palm contour determined based on the target palm image. The palm contour extraction process includes the following steps: obtaining the skeletal point information corresponding to each palm skeletal point in the target palm image; the skeletal point information includes the skeletal point position and the skeletal point type; matching each palm skeletal point based on the skeletal point type to obtain multiple palm skeletal point sets; the palm skeletal point sets have corresponding geometric processing types; based on the skeletal point positions corresponding to each skeletal point in the palm skeletal point set, generating palm contour auxiliary lines matching the geometric processing type corresponding to the palm skeletal point set, and obtaining palm contour auxiliary lines corresponding to each palm skeletal point set; based on the reference step size and skeletal point positions corresponding to the same palm skeletal point set, determining the palm contour points from the palm contour auxiliary lines corresponding to the palm skeletal point set, and obtaining the contour point positions corresponding to the palm contour points; generating the palm contour corresponding to the target palm image based on each contour point position. In this way, the palm contour is based on the skeleton point information corresponding to each palm skeleton point in the palm image, and the contour point position corresponding to the palm contour point is determined by geometric processing, and the palm contour is generated based on the contour point position corresponding to each palm contour point. The extraction of the palm contour not only does not require complex data processing of the pixel information of the palm image, but is also not easily disturbed by the image background, and has high accuracy. The accurate palm contour extracted also helps to ensure accurate display, and then helps to improve the accuracy of generating object control instructions. Automatically generating object control instructions based on the display information of the palm contour can also improve the efficiency of generating object control instructions.

[0156] In one embodiment, generating an object control instruction based on the displayed information of the palm contour includes:

[0157] When the position difference between the display position of the palm outline and the target display position is less than a preset threshold, an object control instruction is generated; when the position difference is greater than or equal to the preset threshold, palm movement prompt information is generated based on the position difference; the palm movement prompt information is used to prompt the palm to move in order to reduce the position difference.

[0158] The target display position is a desired display position. The target display position can be set according to actual needs. The palm movement prompt information can be presented in at least one form of text, voice, picture, and video.

[0159] Specifically, the display information includes a display position. If the position difference between the display position of the palm outline and the target display position is less than a preset threshold, indicating that the palm outline is displayed at the expected position or the target position, the terminal triggers the generation of an object control instruction. If the position difference between the display position of the palm outline and the target display position is greater than or equal to a preset threshold, indicating that the palm outline is not displayed at the expected position or the target position, the terminal can generate palm movement prompt information based on the position difference, and the palm movement prompt information is used to prompt the palm to move to reduce the position difference so that the palm outline can finally be displayed at the expected position.

[0160] In one embodiment, the terminal may determine the display position of the palm outline based on the spatial position of the palm. For example, if the palm is located directly above the image acquisition device, the palm outline is displayed in the center of the screen, if the palm is located to the left of the image acquisition device, the palm outline is displayed on the left of the screen, and if the palm is located to the right of the image acquisition device, the palm outline is displayed on the right of the screen.

[0161] In the above embodiment, when the position difference between the display position of the palm contour and the target display position is less than a preset threshold, an object control instruction is generated; when the position difference is greater than or equal to the preset threshold, palm movement prompt information is generated based on the position difference; the palm movement prompt information is used to prompt the palm to move to reduce the position difference. In this way, the object control instruction is generated only when the position difference between the display position of the palm contour and the target display position is small, which can avoid blindly generating the object control instruction and ensure the effective and accurate triggering of the object control instruction. When the position difference between the display position of the palm contour and the target display position is large, the palm movement prompt information is generated to prompt the palm to move, and the palm movement prompt information can be used to achieve correct guidance, so that the palm contour can be displayed in the correct position, thereby triggering the object control instruction.

[0162] In one embodiment, the target palm image is a palm image corresponding to the virtual resource transferor, the object control instruction is a virtual resource transfer instruction, and the virtual resource transfer instruction is used to instruct to transfer virtual resources from a resource account corresponding to the virtual resource transferor to the target account.

[0163] The virtual resource transferor is the party that needs to transfer virtual resources from the resource account. The virtual resource receiver is the party that receives the virtual resources. A resource account refers to an account that can store virtual resources. Virtual resources refer to resources that exist in an electronic account and can be circulated, such as currency, virtual red envelopes, game coins, virtual items, etc. The target account refers to the account that needs to receive the transferred virtual resources. The target account can be an individual account or a collective account.

[0164] Specifically, in a virtual resource transfer scenario, a terminal or other terminal corresponding to the virtual resource transferor or virtual resource recipient can trigger generation of a virtual resource transfer instruction based on a palm image, and the virtual resource transfer instruction is used to instruct the transfer of virtual resources from a resource account corresponding to the resource transferor to a target account.

[0165] For example, the terminal collects the palm image corresponding to the virtual resource transferor as the target palm image, extracts the palm contour in the target palm image, and displays the palm contour determined based on the target palm image. When the position difference between the display position of the palm contour and the target display position is less than the preset threshold, the terminal verifies the identity information of the virtual resource transferor based on the target palm image. If the verification is successful, a virtual resource transfer instruction is generated, and the virtual resource transfer instruction is used to instruct the transfer of virtual resources from the resource account corresponding to the virtual resource transferor to the target account. If it fails, a verification failure prompt message is generated. When the position difference is greater than or equal to the preset threshold, the terminal generates palm movement prompt information based on the position difference, and the palm movement prompt information is used to prompt the palm to move to reduce the position difference. Subsequently, after the palm moves, if the new position difference determined by the terminal based on the new palm image is less than the preset threshold, the identity information of the virtual resource transferor is verified based on the new palm image. If the verification is successful, a virtual resource transfer instruction is generated.

[0166] In the above embodiment, the palm contour extracted from the palm image can be used to trigger the generation of the virtual resource transfer instruction, thereby expanding the triggering method of the virtual resource transfer instruction.

[0167] In a specific embodiment, the palm contour extraction method can be applied in a palm payment scenario. Specifically, the palm contour extraction method is applied in the application scenario as follows:

[0168] 1. Recognize the palm outline

[0169] 1-1. Identify palm bone points

[0170] Palm skeleton points in a palm image are identified based on a machine learning algorithm. The palm image is acquired through a camera, and the palm image is input into a trained machine learning model, and the model outputs the skeleton point information of the palm skeleton points in the palm image. In one embodiment, the machine learning model can be integrated into an SDK (Software Development Kit). The skeleton point information includes the position coordinates of the palm skeleton points. The position coordinates of the palm skeleton points can be mapped to the two-dimensional coordinates of the current operating system screen through the coordinate system in the SDK, and the palm contour is subsequently generated through the screen coordinates. The current operating system includes an Android system, a Windows system, etc.

[0171] Fig. 7A This is a schematic diagram of the palm bone point recognition results. Fig. 7A , and finally identified 21 palm skeleton points (palm skeleton point 0-palm skeleton point 20) in the palm image.

[0172] 1-2. Identify the palm contour points

[0173] Figure 7B This is a schematic diagram of the palm contour point recognition result. Figure 7B , 38 palm contour points (palm contour point 1'-palm contour point 38') are identified based on 21 palm skeleton points.

[0174] 1-2-1. Calculate the coordinates of the palm outer contour

[0175] The palm contour coordinates on the extension line are obtained through the extension line (also called the extension line) and the relative coefficient (also called the reference step length) of the two palm bone points. Fig. 8A On the extension line generated based on the palm skeleton point 0 and the palm skeleton point 1, a palm contour point 37' is determined, and the palm contour point 37' is a palm outer contour point.

[0176] The calculation process of the palm outer contour coordinates is as follows:

[0177] Palm bone point starting point coordinates: (Xa, Ya); Palm bone point end point coordinates: (Xb, Yb)

[0178] Palm contour point target point coordinates: (X, Y)

[0179] Target point coordinate X:

[0180] Target point coordinate Y:

[0181] refer to Figure 7B , palm contour point 37', palm contour point 36' and palm contour point 8' are palm outer contour points.

[0182] 1-2-2. Calculate the finger gap coordinates

[0183] The coordinates of the finger gap on the perpendicular line are obtained through the perpendicular line and relative coefficients of the two palm bone points. Figure 8B On the perpendicular bisector generated based on the palm skeleton point 13 and the palm skeleton point 17, a palm contour point 29' is determined, and the palm contour point 29' is a palm finger gap point.

[0184] The calculation process of the finger gap coordinates is as follows:

[0185] Palm bone point starting point coordinates: (Xa, Ya); Palm bone point end point coordinates: (Xb, Yb)

[0186] Midpoint coordinates: (Xmid, Ymid)

[0187] Xmid:

[0188] Ymid:

[0189] Palm contour point target point coordinates: (X, Y)

[0190] If the target point is in the clockwise direction:

[0191] Target point coordinate X:

[0192] Target point coordinate Y:

[0193] If the target point is in the counterclockwise direction:

[0194] Target point coordinate X:

[0195] Target point coordinate Y:

[0196] refer to Figure 7B The palm contour point 29', the palm contour point 22' and the palm contour point 15' are palm finger gap points.

[0197] Taking palm skeleton point 13 and palm skeleton point 17 as an example, if palm skeleton point 13 is the starting point and palm skeleton point 17 is the end point, palm contour point 29' is determined in the clockwise direction. If palm skeleton point 17 is the starting point and palm skeleton point 13 is the end point, palm contour point 29' is determined in the counterclockwise direction.

[0198] 1-2-3. Calculate the edge coordinates of the finger contour

[0199] For the vector AB generated by two palm bone points, on the normal vector of point A or point B, the coordinates of the edge of the finger contour on the normal vector are obtained according to the relative coefficient. For example, refer to Figure 8C On the normal vector generated based on the palm skeleton point 2 and the palm skeleton point 3, the palm contour point 3' and the palm contour point 6' are determined. The palm contour point 3' and the palm contour point 6' are the finger contour edge points.

[0200] The calculation process of the finger contour edge coordinates is as follows:

[0201] Palm bone point starting point coordinates: (Xa, Ya); Palm bone point end point coordinates: (Xb, Yb)

[0202] Palm contour point target point coordinates: (X, Y)

[0203] If the target point is in the clockwise direction:

[0204] Target point coordinate X:

[0205] Target point coordinate Y:

[0206] If the target point is in the counterclockwise direction:

[0207] Target point coordinate X:

[0208] Target point coordinate Y:

[0209] refer to Figure 7B , palm contour point 2' to palm contour point 7', palm contour point 9' to palm contour point 14', palm contour point 16' to palm contour point 21', palm contour point 23' to palm contour point 28', and palm contour point 30' to palm contour point 35' are finger contour edge points.

[0210] Wherein, k in the above formula is a relative coefficient.

[0211] 1-3. Generate palm outline

[0212] Connect each palm contour point to get the palm contour. Fig.8D For a schematic diagram of the palm outline, refer to Fig.8D , connect 38 palm contour points to get the palm contour.

[0213] 2. Generate palm swiping animation

[0214] In the palm payment scenario, after obtaining the palm outline, a palm swiping animation can be generated based on the palm outline to enhance the sense of interactivity and technology that palm swiping payment brings to users.

[0215] refer to Fig. 9 , Fig. 9 1002 in the figure indicates the palm outline displayed on the screen, and 1004 indicates the target display position on the screen. If the palm outline is not displayed at the target display position, the user is prompted to move the palm to the target display position. If the palm outline is displayed at the target display position, the user identity is verified based on the palm image, and payment is performed after the verification is passed.

[0216] In the above embodiment, palm contour points are determined based on palm skeleton points, and then the palm contour is generated. On the one hand, palm skeleton points are not easily affected by the image background, and the palm contour can be extracted from the image based on palm skeleton points to ensure the accuracy of palm contour extraction. On the other hand, using a geometric method to determine palm contour points can also ensure the efficiency of data processing, ensure recognition performance, and improve the efficiency of palm contour extraction.

[0217] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0218] Based on the same inventive concept, the embodiment of the present application also provides a palm contour extraction device for implementing the palm contour extraction method involved above, and also provides a control instruction generation device for implementing the control instruction generation method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more palm contour extraction device embodiments provided below can refer to the limitations on the palm contour extraction method above, and the specific limitations in one or more control instruction generation device embodiments provided below can refer to the limitations on the control instruction generation method above, which will not be repeated here.

[0219] In one embodiment, Fig.10 As shown, a palm contour extraction device is provided, comprising: a skeleton point information acquisition module 1002, a palm skeleton point matching module 1004, a palm contour auxiliary line determination module 1006, a palm contour point determination module 1008 and a palm contour generation module 1010, wherein:

[0220] The skeleton point information acquisition module 1002 is used to acquire the skeleton point information corresponding to each palm skeleton point in the target palm image; the skeleton point information includes the skeleton point position and the skeleton point type.

[0221] The palm skeleton point matching module 1004 is used to match each palm skeleton point based on the skeleton point type to obtain multiple palm skeleton point sets; the palm skeleton point sets have corresponding geometric processing types.

[0222] The palm contour auxiliary line determination module 1006 is used to generate palm contour auxiliary lines matching the geometric processing type corresponding to the palm bone point set based on the bone point positions corresponding to each bone point in the palm bone point set, and obtain palm contour auxiliary lines corresponding to each palm bone point set.

[0223] The palm contour point determination module 1008 is used to determine the palm contour points from the palm contour auxiliary lines corresponding to the palm skeleton point set based on the reference step size and skeleton point positions corresponding to the same palm skeleton point set, and obtain the contour point positions corresponding to the palm contour points.

[0224] The palm contour generating module 1010 is used to generate a palm contour corresponding to the target palm image based on the positions of each contour point.

[0225] The above-mentioned palm contour extraction device does not need to perform complex data processing on the pixel information of the palm image. It can conveniently determine the contour point position corresponding to the palm contour point from the palm image by performing geometric processing based on the skeleton point information corresponding to each palm skeleton point in the palm image, and then generate the palm contour based on the contour point position corresponding to each palm contour point. The position and type of the palm skeleton point are not easily affected by the image background. Extracting the palm contour based on the position and type of the palm skeleton point in the palm image can effectively improve the accuracy of palm contour extraction.

[0226] In one embodiment, the bone point type is determined by the finger parameters and the interphalangeal joint parameters. The palm bone point matching module is also used to establish a matching relationship between adjacent palm bone points whose interphalangeal joint parameters are preset joint parameters, to obtain a set of palm bone points corresponding to the first geometric processing type, to establish a matching relationship between adjacent palm bone points corresponding to the same finger parameters, to obtain a set of palm bone points corresponding to the second geometric processing type, to use the palm bone point corresponding to the preset bone point type as the preset bone point, to establish a matching relationship between the preset bone point and the palm bone point closest to the preset bone point, to obtain a set of palm bone points corresponding to the third geometric processing type.

[0227] In one embodiment, the palm contour auxiliary line determination module is also used to determine the current auxiliary line type based on the geometric processing type corresponding to the current palm bone point set, generate the current bone point line segment corresponding to the current palm bone point set based on the bone point position corresponding to each palm bone point in the current palm bone point set, and based on the current auxiliary line type, perform geometric processing on the current bone point line segment to obtain the palm contour auxiliary line corresponding to the current auxiliary line type.

[0228] In one embodiment, the geometry processing type includes any one of a first geometry processing type, a second geometry processing type, and a third geometry processing type, the auxiliary line type corresponding to the first geometry processing type includes at least one of a perpendicular bisector type and an extended line type, the auxiliary line type corresponding to the second geometry processing type includes a normal line type, and the auxiliary line type corresponding to the third geometry processing type includes at least one of a normal line type and an extended line type.

[0229] In one embodiment, the palm contour auxiliary line determination module is also used to, when the current auxiliary line type is a perpendicular median line type, use the perpendicular median line corresponding to the current bone point line segment as the palm contour auxiliary line; when the current auxiliary line type is an extension line type, determine the target bone point from each palm bone point corresponding to the current palm bone point set, and use the extension line corresponding to the current bone point line segment and starting from the target bone point as the palm contour auxiliary line; when the current auxiliary line type is a normal line type, determine the target bone point from each palm bone point corresponding to the current palm bone point set, and use the normal line of the current bone point line segment at the target bone point as the palm contour auxiliary line.

[0230] In one embodiment, the palm contour point determination module is also used to generate a current skeleton point line segment corresponding to the current palm skeleton point set based on the skeleton point position corresponding to each skeleton point in the current palm skeleton point set; determine a reference point from the current skeleton point line segment; on the current palm contour auxiliary line corresponding to the current palm skeleton point set, take the reference point as the starting point, and move the reference step corresponding to the current palm skeleton point set in the moving direction corresponding to the current palm contour auxiliary line to obtain the current palm contour point and the contour point position corresponding to the current palm contour point.

[0231] In one embodiment, the current palm contour auxiliary line includes at least one of a perpendicular bisector, an extension line and a normal line. The moving direction corresponding to the perpendicular bisector is the opposite direction of the palm, the moving direction corresponding to the extension line is the extension direction of the extension line, and the moving direction corresponding to the normal line is the vertical direction of the current skeleton point line segment.

[0232] In one embodiment, each palm skeleton point has a corresponding skeleton point identifier. The palm contour generation module is also used to map the skeleton point identifiers corresponding to the palm skeleton points based on preset mapping information to obtain contour point sorting identifiers corresponding to each palm contour point; the preset mapping information includes the identifier mapping relationship between the palm skeleton points in the palm skeleton point set and the palm contour points generated based on the palm skeleton point set; according to the contour point sorting identifiers, the contour point positions corresponding to each palm contour point are orderly connected to obtain the palm contour.

[0233] In one embodiment, the skeleton point information acquisition module is further used to input the target palm image into the target palm skeleton point recognition model to obtain the skeleton point information corresponding to each palm skeleton point.

[0234] refer to Fig.11 , the palm contour extraction device also includes:

[0235] Model training module 1001 is used to obtain a training data set; the training data set includes multiple training palm images and training bone point information corresponding to each training palm image; determine the current palm image from the training data set; input the current palm image into the initial palm bone point recognition model to obtain the predicted bone point information corresponding to the current palm image; based on the difference between the predicted bone point information corresponding to the current palm image and the training bone point information, adjust the model parameters corresponding to the initial palm bone point recognition model to obtain an intermediate bone point recognition model; use the next training palm image as the current palm image, use the intermediate bone point recognition model as the initial palm bone point recognition model, return to the step of inputting the current palm image into the initial palm bone point recognition model, and execute until the convergence condition is met to obtain the target bone point recognition model.

[0236] In one embodiment, the palm outline is used to generate the palm animation.

[0237] In one embodiment, Fig.12 As shown, a control instruction generating device is provided, including: a palm contour display module 1202 and a control instruction generating module 1204, wherein:

[0238] The palm contour display module 1202 is used to display the palm contour determined based on the target palm image.

[0239] The control instruction generating module 1204 is used to generate an object control instruction based on the displayed information of the palm contour.

[0240] The palm contour extraction process includes the following steps: obtaining the skeletal point information corresponding to each palm skeletal point in the target palm image; the skeletal point information includes the skeletal point position and the skeletal point type; based on the skeletal point type, matching each palm skeletal point to obtain multiple palm skeletal point sets; the palm skeletal point set has a corresponding geometric processing type; based on the skeletal point position corresponding to each skeletal point in the palm skeletal point set, generating palm contour auxiliary lines matching the geometric processing type corresponding to the palm skeletal point set, and obtaining palm contour auxiliary lines corresponding to each palm skeletal point set; based on the reference step size and skeletal point position corresponding to the same palm skeletal point set, determining the palm contour points from the palm contour auxiliary lines corresponding to the palm skeletal point set, and obtaining the contour point positions corresponding to the palm contour points; generating the palm contour corresponding to the target palm image based on each contour point position.

[0241] In the control instruction generating device, the palm contour is generated based on the contour point positions corresponding to the palm contour points by geometrically processing the skeleton point information corresponding to each palm skeleton point in the palm image, and the palm contour extraction not only does not require complex data processing of the pixel information of the palm image, but is also not easily disturbed by the image background, and has high accuracy. Extracting an accurate palm contour also helps to ensure accurate display, and thus helps to improve the accuracy of generating object control instructions. Automatically generating object control instructions based on the display information of the palm contour can also improve the efficiency of generating object control instructions.

[0242] In one embodiment, the control instruction generation module is also used to generate an object control instruction when the position difference between the display position of the palm outline and the target display position is less than a preset threshold; when the position difference is greater than or equal to the preset threshold, generate palm movement prompt information based on the position difference; the palm movement prompt information is used to prompt the palm to move to reduce the position difference.

[0243] In one embodiment, the target palm image is a palm image corresponding to the virtual resource transferor, the object control instruction is a virtual resource transfer instruction, and the virtual resource transfer instruction is used to instruct to transfer virtual resources from a resource account corresponding to the virtual resource transferor to the target account.

[0244] Each module in the palm contour extraction device and the control instruction generation device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0245] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.13As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as a target palm bone point recognition model, a reference step, a preset mapping relationship, etc. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a palm contour extraction method or a control instruction generation method is implemented.

[0246] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.14 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a palm contour extraction method or a control instruction generation method is implemented. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0247] Those skilled in the art will understand that Fig.13 , 14The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0248] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0249] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0250] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the above-mentioned method embodiments.

[0251] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0252] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0253] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0254] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A palm contour extraction method, characterized in that: The method comprises: Obtaining the skeleton point information corresponding to each palm skeleton point in the target palm image; the skeleton point information includes the skeleton point position and the skeleton point type; the skeleton point type is determined by the finger parameters and the interphalangeal joint parameters; Establishing a matching relationship between adjacent palm bone points whose interphalangeal joint parameters are preset joint parameters, and obtaining a set of palm bone points corresponding to the first geometric processing type; Establishing a matching relationship between adjacent palm skeleton points corresponding to the same finger parameter to obtain a set of palm skeleton points corresponding to the second geometric processing type; the adjacent palm skeleton points refer to two adjacent palm skeleton points; Taking the palm bone point corresponding to the preset bone point type as the preset bone point, establishing a matching relationship between the preset bone point and the palm bone point closest to the preset bone point, and obtaining a palm bone point set corresponding to the third geometric processing type; Based on the bone point positions corresponding to each bone point in the palm bone point set, a palm contour auxiliary line matching the geometric processing type corresponding to the palm bone point set is generated, and the palm contour auxiliary lines corresponding to each palm bone point set are obtained; Based on the target reference step length and the bone point position corresponding to the same palm bone point set, the palm contour point is determined from the palm contour auxiliary line corresponding to the palm bone point set, and the contour point position corresponding to the palm contour point is obtained; A palm contour corresponding to the target palm image is generated based on the positions of the contour points.

2. The method according to claim 1, characterized in that The method further comprises: Acquire a palm image with a marked palm contour as a reference palm image, and determine a universal reference step length based on the reference palm image; Based on the personalized information of the target palm image, the universal reference step is adjusted to obtain a target reference step; the personalized information includes at least one of palm size information and palm depth information.

3. The method according to claim 1, characterized in that The method of generating a palm contour auxiliary line matching a geometric processing type corresponding to the palm skeleton point set based on the skeleton point position corresponding to each skeleton point in the palm skeleton point set, and obtaining the palm contour auxiliary lines corresponding to each palm skeleton point set respectively, comprises: Determine the current auxiliary line type based on the geometry processing type corresponding to the current palm bone point set; Based on the skeleton point positions corresponding to the palm skeleton points in the current palm skeleton point set, generating a current skeleton point line segment corresponding to the current palm skeleton point set; Based on the current auxiliary line type, the current skeleton point line segment is geometrically processed to obtain a palm contour auxiliary line corresponding to the current auxiliary line type.

4. The method according to claim 3, characterized in that The geometric processing type includes any one of a first geometric processing type, a second geometric processing type and a third geometric processing type. The auxiliary line type corresponding to the first geometric processing type includes at least one of a perpendicular bisector type and an extended line type. The auxiliary line type corresponding to the second geometric processing type includes a normal line type. The auxiliary line type corresponding to the third geometric processing type includes at least one of a normal line type and an extended line type.

5. The method according to claim 3, characterized in that: The step of performing geometric processing on the current skeleton point line segment based on the current auxiliary line type to obtain a palm contour auxiliary line corresponding to the current auxiliary line type includes: When the current auxiliary line type is a perpendicular median line type, the perpendicular median line corresponding to the current skeleton point line segment is used as a palm contour auxiliary line; When the current auxiliary line type is an extension line type, a target bone point is determined from each palm bone point corresponding to the current palm bone point set, and an extension line corresponding to the current bone point line segment and starting from the target bone point is used as a palm contour auxiliary line; When the current auxiliary line type is a normal line type, a target bone point is determined from each palm bone point corresponding to the current palm bone point set, and a normal line of the current bone point line segment at the target bone point is used as a palm contour auxiliary line.

6. The method according to claim 1, characterized in that The method of determining palm contour points from palm contour auxiliary lines corresponding to the palm contour point set based on the target reference step length and the bone point positions corresponding to the same palm bone point set, and obtaining the contour point positions corresponding to the palm contour points, comprises: Based on the skeletal point positions corresponding to the skeletal points in the current palm skeletal point set, a current skeletal point line segment corresponding to the current palm skeletal point set is generated; Determine a reference point from the current skeleton point line segment; On the current palm contour auxiliary line corresponding to the current palm skeleton point set, with the reference point as the starting point, the target reference step corresponding to the current palm contour auxiliary line is moved in the moving direction corresponding to the current palm contour auxiliary line to obtain the current palm contour point and the contour point position corresponding to the current palm contour point.

7. The method according to claim 6, characterized in that The current palm contour auxiliary line includes at least one of a perpendicular bisector, an extension line and a normal line. The moving direction corresponding to the perpendicular bisector is the opposite direction of the palm, the moving direction corresponding to the extension line is the extension direction of the extension line, and the moving direction corresponding to the normal line is the vertical direction of the current skeleton point line segment.

8. The method according to claim 1, characterized in that Each palm skeleton point has a corresponding skeleton point identifier, and generating a palm contour corresponding to the target palm image based on the position of each contour point includes: Based on preset mapping information, the skeletal point identifiers corresponding to the palm skeletal points are mapped to obtain contour point sorting identifiers corresponding to each palm contour point; the preset mapping information includes an identifier mapping relationship between the palm skeletal points in the palm skeletal point set and the palm contour points generated based on the palm skeletal point set; According to the contour point sorting identifier, the contour point positions corresponding to the palm contour points are connected in order to obtain the palm contour.

9. The method according to claim 1, characterized in that: The step of obtaining the skeleton point information corresponding to each palm skeleton point in the target palm image includes: Inputting the target palm image into a target palm skeleton point recognition model to obtain skeleton point information corresponding to each palm skeleton point; The training process of the target palm skeleton point recognition model includes the following steps: Acquire a training data set; the training data set includes a plurality of training palm images and training skeleton point information corresponding to each training palm image; Determining a current palm image from the training data set; Input the current palm image into the initial palm bone point recognition model to obtain the predicted bone point information corresponding to the current palm image; Based on the difference between the predicted bone point information and the training bone point information corresponding to the current palm image, the model parameters corresponding to the initial palm bone point recognition model are adjusted to obtain an intermediate bone point recognition model; The next training palm image is used as the current palm image, the intermediate bone point recognition model is used as the initial palm bone point recognition model, and the step of inputting the current palm image into the initial palm bone point recognition model is returned to be executed until the convergence condition is met to obtain the target bone point recognition model.

10. The method according to any one of claims 1 to 9, characterized in that: The palm contour is used to generate palm animation.

11. A control instruction generation method, characterized in that: The method comprises: displaying a palm contour determined based on the target palm image; generating an object control instruction based on the displayed information of the palm contour; The palm contour extraction process comprises the following steps: Obtaining the skeleton point information corresponding to each palm skeleton point in the target palm image; the skeleton point information includes the skeleton point position and the skeleton point type; the skeleton point type is determined by the finger parameters and the interphalangeal joint parameters; Establishing a matching relationship between adjacent palm bone points whose interdigital joint parameters are preset joint parameters, and obtaining a set of palm bone points corresponding to the first geometric processing type; establishing a matching relationship between adjacent palm bone points corresponding to the same finger parameters, and obtaining a set of palm bone points corresponding to the second geometric processing type; the adjacent palm bone points refer to two palm bone points with adjacent positions; taking the palm bone point corresponding to the preset bone point type as the preset bone point, and establishing a matching relationship between the preset bone point and the palm bone point closest to the preset bone point, and obtaining a set of palm bone points corresponding to the third geometric processing type; Based on the bone point positions corresponding to each bone point in the palm bone point set, a palm contour auxiliary line matching the geometric processing type corresponding to the palm bone point set is generated, and the palm contour auxiliary lines corresponding to each palm bone point set are obtained; Based on the target reference step length and the bone point position corresponding to the same palm bone point set, the palm contour point is determined from the palm contour auxiliary line corresponding to the palm bone point set, and the contour point position corresponding to the palm contour point is obtained; A palm contour corresponding to the target palm image is generated based on the positions of the contour points.

12. The method according to claim 11, characterized in that The generating of the object control instruction based on the display information of the palm contour comprises: When the position difference between the display position of the palm contour and the target display position is less than a preset threshold, generating an object control instruction; When the position difference is greater than or equal to the preset threshold, palm movement prompt information is generated based on the position difference; the palm movement prompt information is used to prompt the palm to move to reduce the position difference.

13. The method according to claim 11 or 12, characterized in that: The target palm image is a palm image corresponding to the virtual resource transferor, the object control instruction is a virtual resource transfer instruction, and the virtual resource transfer instruction is used to instruct the transfer of virtual resources from the resource account corresponding to the virtual resource transferor to the target account.

14. A palm contour extraction device, characterized in that: The device comprises: A skeleton point information acquisition module is used to acquire skeleton point information corresponding to each palm skeleton point in the target palm image; the skeleton point information includes skeleton point position and skeleton point type; the skeleton point type is determined by finger parameters and interphalangeal joint parameters; A palm bone point matching module is used to establish a matching relationship between adjacent palm bone points whose interdigital joint parameters are preset joint parameters, and obtain a set of palm bone points corresponding to a first geometric processing type; establish a matching relationship between adjacent palm bone points corresponding to the same finger parameter, and obtain a set of palm bone points corresponding to a second geometric processing type; the adjacent palm bone points refer to two palm bone points with adjacent positions; use a palm bone point corresponding to a preset bone point type as a preset bone point, establish a matching relationship between the preset bone point and a palm bone point closest to the preset bone point, and obtain a set of palm bone points corresponding to a third geometric processing type; A palm contour auxiliary line determination module is used to generate a palm contour auxiliary line matching the geometric processing type corresponding to the palm bone point set based on the bone point position corresponding to each bone point in the palm bone point set, and obtain the palm contour auxiliary lines corresponding to each palm bone point set respectively; A palm contour point determination module is used to determine the palm contour points from the palm contour auxiliary lines corresponding to the palm skeleton point set based on the target reference step length and the skeleton point position corresponding to the same palm skeleton point set, and obtain the contour point position corresponding to the palm contour point; The palm contour generation module is used to generate the palm contour corresponding to the target palm image based on the positions of each contour point.

15. The device according to claim 14, characterized in that The device is also used for: Acquire a palm image with a marked palm contour as a reference palm image, and determine a universal reference step length based on the reference palm image; Based on the personalized information of the target palm image, the universal reference step is adjusted to obtain a target reference step; the personalized information includes at least one of palm size information and palm depth information.

16. The device according to claim 14, characterized in that The palm contour auxiliary line determination module is also used for: Determine the current auxiliary line type based on the geometry processing type corresponding to the current palm bone point set; Based on the skeleton point positions corresponding to the palm skeleton points in the current palm skeleton point set, generating a current skeleton point line segment corresponding to the current palm skeleton point set; Based on the current auxiliary line type, the current skeleton point line segment is geometrically processed to obtain a palm contour auxiliary line corresponding to the current auxiliary line type.

17. The device according to claim 16, characterized in that The geometric processing type includes any one of a first geometric processing type, a second geometric processing type and a third geometric processing type. The auxiliary line type corresponding to the first geometric processing type includes at least one of a perpendicular bisector type and an extended line type. The auxiliary line type corresponding to the second geometric processing type includes a normal line type. The auxiliary line type corresponding to the third geometric processing type includes at least one of a normal line type and an extended line type.

18. The device according to claim 16, characterized in that The palm contour auxiliary line determination module is also used for: When the current auxiliary line type is a perpendicular median line type, the perpendicular median line corresponding to the current skeleton point line segment is used as a palm contour auxiliary line; When the current auxiliary line type is an extension line type, a target bone point is determined from each palm bone point corresponding to the current palm bone point set, and an extension line corresponding to the current bone point line segment and starting from the target bone point is used as a palm contour auxiliary line; When the current auxiliary line type is a normal line type, a target bone point is determined from each palm bone point corresponding to the current palm bone point set, and a normal line of the current bone point line segment at the target bone point is used as a palm contour auxiliary line.

19. The device according to claim 14, characterized in that The palm contour point determination module is also used for: Based on the skeletal point positions corresponding to the skeletal points in the current palm skeletal point set, a current skeletal point line segment corresponding to the current palm skeletal point set is generated; Determine a reference point from the current skeleton point line segment; On the current palm contour auxiliary line corresponding to the current palm skeleton point set, with the reference point as the starting point, the target reference step corresponding to the current palm contour auxiliary line is moved in the moving direction corresponding to the current palm contour auxiliary line to obtain the current palm contour point and the contour point position corresponding to the current palm contour point.

20. The device according to claim 19, characterized in that The current palm contour auxiliary line includes at least one of a perpendicular bisector, an extension line and a normal line. The moving direction corresponding to the perpendicular bisector is the opposite direction of the palm, the moving direction corresponding to the extension line is the extension direction of the extension line, and the moving direction corresponding to the normal line is the vertical direction of the current skeleton point line segment.

21. The device according to claim 14, characterized in that Each palm skeleton point has a corresponding skeleton point identifier, and the palm contour generation module is further used for: Based on preset mapping information, the skeletal point identifiers corresponding to the palm skeletal points are mapped to obtain contour point sorting identifiers corresponding to each palm contour point; the preset mapping information includes an identifier mapping relationship between the palm skeletal points in the palm skeletal point set and the palm contour points generated based on the palm skeletal point set; According to the contour point sorting identifier, the contour point positions corresponding to the palm contour points are connected in order to obtain the palm contour.

22. The device according to claim 14, characterized in that The skeleton point information acquisition module is also used for: Inputting the target palm image into a target palm skeleton point recognition model to obtain skeleton point information corresponding to each palm skeleton point; The training process of the target palm skeleton point recognition model includes the following steps: Acquire a training data set; the training data set includes a plurality of training palm images and training skeleton point information corresponding to each training palm image; Determining a current palm image from the training data set; Input the current palm image into the initial palm bone point recognition model to obtain the predicted bone point information corresponding to the current palm image; Based on the difference between the predicted bone point information and the training bone point information corresponding to the current palm image, the model parameters corresponding to the initial palm bone point recognition model are adjusted to obtain an intermediate bone point recognition model; The next training palm image is used as the current palm image, the intermediate bone point recognition model is used as the initial palm bone point recognition model, and the step of inputting the current palm image into the initial palm bone point recognition model is returned to be executed until the convergence condition is met to obtain the target bone point recognition model.

23. The device according to any one of claims 14 to 22, characterized in that The palm contour is used to generate palm animation.

24. A control instruction generating device, characterized in that: The device comprises: A palm contour display module, used to display a palm contour determined based on a target palm image; A control instruction generating module, used for generating an object control instruction based on the displayed information of the palm contour; The palm contour extraction process comprises the following steps: Obtain the skeletal point information corresponding to each palm skeletal point in the target palm image; the skeletal point information includes the skeletal point position and the skeletal point type; the skeletal point type is determined by the finger parameters and the interphalangeal joint parameters; establish a matching relationship between adjacent palm skeletal points whose interphalangeal joint parameters are preset joint parameters, and obtain a set of palm skeletal points corresponding to the first geometric processing type; establish a matching relationship between adjacent palm skeletal points corresponding to the same finger parameters, and obtain a set of palm skeletal points corresponding to the second geometric processing type; the adjacent palm skeletal points refer to two palm skeletal points that are adjacent in position; use the palm skeletal point corresponding to the preset skeletal point type as the preset skeletal point, and match the preset skeletal point to the preset skeletal point type. A matching relationship is established between the palm skeleton point and the palm skeleton point that is closest to the preset skeleton point to obtain a palm skeleton point set corresponding to a third geometric processing type; based on the skeleton point positions corresponding to each skeleton point in the palm skeleton point set, palm contour auxiliary lines that match the geometric processing type corresponding to the palm skeleton point set are generated to obtain palm contour auxiliary lines corresponding to each palm skeleton point set; based on the target reference step and skeleton point positions corresponding to the same palm skeleton point set, palm contour points are determined from the palm contour auxiliary lines corresponding to the palm skeleton point set to obtain contour point positions corresponding to the palm contour points; based on each contour point position, a palm contour corresponding to the target palm image is generated.

25. The device according to claim 24, characterized in that The control instruction generation module is also used for: When the position difference between the display position of the palm contour and the target display position is less than a preset threshold, generating an object control instruction; When the position difference is greater than or equal to the preset threshold, palm movement prompt information is generated based on the position difference; the palm movement prompt information is used to prompt the palm to move to reduce the position difference.

26. The device according to claim 24 or 25, characterized in that The target palm image is a palm image corresponding to the virtual resource transferor, the object control instruction is a virtual resource transfer instruction, and the virtual resource transfer instruction is used to instruct the transfer of virtual resources from the resource account corresponding to the virtual resource transferor to the target account.

27. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 10 or 11 to 13 are implemented.

28. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 or 11 to 13 are implemented.

29. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 or 11 to 13 are implemented.

Images