Hand recognition method, device, electronic device and storage medium

Abstract

The embodiments of the present invention disclose a method, device, electronic device and storage medium for hand recognition. The hand recognition method may specifically include: obtaining an image to be recognized, and extracting low-level image features from the image to be recognized; performing deformation convolution processing on the low-level image features, and obtaining a palm point that matches the deformation processing result; performing standard convolution processing on the low-level image features, and obtaining hand description information that matches the standard processing result, the hand description information including: the offset between the palm point and the center point of the hand, and the width and height of the hand; identifying the hand area in the image to be recognized based on the palm point and the hand description information. The technical solution of the embodiment of the present invention can reduce the amount of calculation in the hand recognition process, simplify the hand recognition task, and thus improve the efficiency of hand recognition.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of artificial intelligence technology, and in particular to a method, device, electronic device, and storage medium for identifying a hand. Background Art

[0002] With the development of science and technology, the application of hand recognition in images is becoming more and more extensive, such as in virtual / augmented reality, human-computer interaction, motion recognition, assisted driving and other fields. Hand recognition, as an upstream task of gesture understanding, finger key point detection and other tasks, has also received much attention.

[0003] In different scenarios (such as standing, sitting, or holding an object), the distance between the human hand and the camera varies, resulting in different sizes of hands in the image, making hand recognition difficult to learn. In addition, there are cases where objects block part of the hand (such as holding an object, some fingers are invisible), which increases the difficulty of hand recognition. The shape of the hand changes with the change of gestures, and the relative position of each finger is not fixed, which increases the difficulty of learning. Hands are covered by skin and lack obvious distinguishing features. Therefore, when the hand is close to the face or close to the skin color, or when the fingers are close together, it will increase the difficulty of recognition.

[0004] Currently, the main hand recognition methods typically directly predict the hand rectangle, or predict the offset of a preset hand rectangle and select the most matching predicted hand rectangle as output. The predicted hand rectangle is generally the coordinates of the upper left corner or the center of the rectangle, as well as the width and height of the rectangle. In the process of implementing the present invention, the inventors discovered that when directly predicting the rectangle or the offset of the rectangle, the hand center point has no fixed position and cannot be clearly defined. The number of samples that need to be fitted becomes infinite, which means that many preset rectangles are required. This results in a large amount of computation during the hand recognition process. In addition, the selection of rectangle matching is required, which leads to a high complexity in the hand recognition process. Summary of the Invention

[0005] Embodiments of the present invention provide a hand recognition method, device, electronic device, and storage medium, which can reduce the amount of calculation in the hand recognition process, simplify the hand recognition task, and thus improve the hand recognition efficiency.

[0006] According to one aspect of the present invention, a method for identifying a hand is provided, comprising:

[0007] Obtain an image to be identified and extract low-level image features from the image to be identified;

[0008] Perform deformation convolution processing on low-level image features and obtain the palm center point that matches the deformation processing result;

[0009] Perform standard convolution processing on low-level image features and obtain hand description information that matches the standard processing results. The hand description information includes: the offset between the palm point and the center point of the hand, and the width and height of the hand;

[0010] Identify the hand area in the image to be identified based on the palm point and hand description information.

[0011] According to another aspect of the present invention, a method for identifying a hand is provided, comprising:

[0012] Acquire an image to be recognized and input the image to be recognized into a pre-trained hand recognition model; the hand recognition model includes: a first standard convolutional network, and a deformable convolutional branch network and a second standard convolutional branch network respectively connected to the first standard convolutional network;

[0013] Extract low-level image features from the image to be recognized through the first standard convolutional network, and input the low-level image features into the deformable convolutional branch network and the second standard convolutional branch network respectively;

[0014] Through the deformable convolution branch network, the low-level image features are deformed and convolved, and the palm center point that matches the deformation processing result is obtained;

[0015] Through the second standard convolution branch network, standard convolution processing is performed on the low-level image features, and hand description information that matches the standard processing results is obtained. The hand description information includes: the offset between the palm point and the center point of the hand, and the width and height of the hand;

[0016] According to the palm point and hand description information output by the hand recognition model, the hand area is identified in the image to be recognized.

[0017] According to another aspect of the present invention, there is provided a human hand recognition device, comprising:

[0018] A low-level image feature extraction module is used to obtain an image to be identified and extract low-level image features from the image to be identified;

[0019] The palm point acquisition module is used to perform deformation convolution processing on low-level image features and obtain the palm point that matches the deformation processing result;

[0020] The hand description information acquisition module is used to perform standard convolution processing on low-level image features and obtain hand description information that matches the standard processing results. The hand description information includes: the offset between the palm point and the center point of the hand, and the width and height of the hand;

[0021] The hand area recognition module is used to identify the hand area in the image to be recognized based on the palm point and hand description information.

[0022] According to another aspect of the present invention, there is provided a human hand recognition device, comprising:

[0023] An image processing module for obtaining an image to be recognized and inputting the image to be recognized into a pre-trained hand recognition model; the hand recognition model comprises: a first standard convolutional network, and a deformable convolutional branch network and a second standard convolutional branch network respectively connected to the first standard convolutional network;

[0024] The image processing module to be identified is used to extract low-level image features from the image to be identified through the first standard convolutional network, and input the low-level image features into the deformable convolutional branch network and the second standard convolutional branch network respectively;

[0025] A first low-level image feature processing module is used to perform deformation convolution processing on the low-level image features through a deformation convolution branch network, and obtain a palm point that matches the deformation processing result;

[0026] a second low-level image feature processing module, configured to perform standard convolution processing on the low-level image features through a second standard convolution branch network, and obtain hand description information that matches the standard processing result, the hand description information including: the offset between the palm point and the center point of the hand, and the width and height of the hand;

[0027] The hand region recognition module is used to recognize the hand region in the image to be recognized based on the palm point and hand description information output by the hand recognition model.

[0028] According to another aspect of the present invention, an electronic device is provided, comprising:

[0029] at least one processor; and

[0030] a memory communicatively connected to the at least one processor; wherein,

[0031] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the human hand recognition method described in any embodiment of the present invention.

[0032] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the human hand recognition method described in any embodiment of the present invention when executed.

[0033] The technical solution of the embodiment of the present invention obtains an image to be identified, extracts low-level image features from the image to be identified, performs deformation convolution processing on the low-level image features to obtain palm points that match the deformation processing results, and performs standard convolution processing on the low-level image features to obtain hand description information that matches the standard processing results, thereby identifying the hand area in the image to be identified based on the palm points and hand description information, solving the problems of large amount of computation and high complexity in the hand recognition process of existing hand recognition methods, reducing the amount of computation in the hand recognition process, simplifying the hand recognition task, and thus improving the efficiency of hand recognition.

[0034] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 This is a flow chart of a method for identifying a hand provided in the first embodiment of the present invention;

[0037] Figure 2 This is a flow chart of a method for identifying a hand provided in the second embodiment of the present invention;

[0038] Figure 3 This is a flow chart of a method for identifying a hand provided in the third embodiment of the present invention;

[0039] Figure 4 This is an example flow chart of a method for identifying a hand provided in the third embodiment of the present invention;

[0040] Figure 5 Schematic diagram of the structure of the deformable convolution module provided in the third embodiment of the present invention;

[0041] Figure 6 This is a schematic diagram of identifying a human hand area provided by the third embodiment of the present invention;

[0042] Figure 7 is a schematic diagram of a hand recognition device provided by a fourth embodiment of the present invention;

[0043] Figure 8 is a schematic diagram of a hand recognition device provided by Embodiment 5 of the present invention;

[0044] Figure 9 It is a structural diagram of an electronic device for implementing the human hand recognition method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0046] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0047] Example 1

[0048] Figure 1 This is a flowchart of a hand recognition method provided by the first embodiment of the present invention. This embodiment is applicable to reducing the amount of calculation in the hand recognition process and simplifying the hand recognition task. The method can be executed by a hand recognition device, which can be implemented by software and / or hardware and can generally be directly integrated into an electronic device that executes this method. The electronic device can be a terminal device or a server device. The embodiment of the present invention does not limit the type of electronic device that executes the hand recognition method. Specifically, Figure 1 As shown, the hand recognition method may specifically include the following steps:

[0049] S110 , obtaining an image to be recognized, and extracting low-level image features from the image to be recognized.

[0050] The image to be recognized can be any image requiring hand recognition. It is understood that the image to be recognized can include at least one human hand. Low-level image features can be low-level image features extracted from the image to be recognized, such as line features or angle features, and are not limited in this embodiment of the present invention.

[0051] In an embodiment of the present invention, an image to be identified is obtained, and low-level image features are extracted from the image to be identified. Optionally, before extracting the low-level image features from the image to be identified, image scaling processing may be performed on the image to be identified to scale the image to be identified to a preset image size corresponding to the image to be identified. After extracting the low-level image features from the image to be identified, image scaling processing may be performed on the low-level image features to scale the low-level image features to a preset image size corresponding to the low-level image features, thereby avoiding excessive computational effort.

[0052] S120: Perform deformation convolution processing on the low-level image features and obtain a palm center point that matches the deformation processing result.

[0053] The deformable convolution process may be a convolution process in which the convolution kernel action area is a rectangular area of ​​irregular convolution kernel size around a center point. The deformable process result may be an image feature result obtained after the deformable convolution process. It is understandable that due to the non-fixation of the rectangular area during the deformable convolution process, the deformable convolution process may affect the relative position of the features, so that the relative position of the features after the deformable convolution process is not completely aligned with the low-level image features.

[0054] In an embodiment of the present invention, after extracting low-level image features from the image to be recognized, deformation convolution processing may be further performed on the low-level image features to obtain a palm point that matches the deformation processing result.

[0055] The above technical solution can expand the receptive field by performing deformation convolution processing on low-level image features, thereby better identifying human hands and improving the effect of human hand recognition.

[0056] S130: Perform standard convolution processing on the low-level image features and obtain hand description information that matches the standard processing result. The hand description information includes: the offset between the palm point and the center point of the hand, and the width and height of the hand.

[0057] Among them, the standard convolution processing can be a convolution processing in which the convolution kernel action area is a rectangular area of ​​the standard convolution kernel size around the center point. The standard processing result can be the feature result obtained after the standard convolution processing operation. The human hand description information can be information that can describe the human hand. Specifically, the human hand description information can include the offset between the palm point and the center point of the human hand, and the width and height values ​​of the human hand. It can be understood that the offset between the palm point and the center point of the human hand can include the horizontal offset between the palm point and the center point of the human hand, and the vertical offset between the palm point and the center point of the human hand. The human hand width and height values ​​can include the height value of the human hand, and the width value of the human hand.

[0058] In an embodiment of the present invention, after extracting low-level image features from the image to be identified, standard convolution processing can be further performed on the low-level image features, and the offset between the palm point and the center point of the human hand that matches the standard processing result can be obtained, as well as the width and height values ​​of the human hand that match the standard processing result can be obtained.

[0059] S140: Identify a human hand region in the image to be identified based on the palm point and the human hand description information.

[0060] The hand region may be the region where the hand is located. It is understandable that if the image to be identified includes multiple hands, there may be multiple regions for identifying the hand.

[0061] In an embodiment of the present invention, after obtaining the palm point that matches the deformation processing result and obtaining the hand description information that matches the standard processing result, the hand area can be further identified in the image to be identified based on the palm point and the hand description information.

[0062] The technical solution of this embodiment obtains an image to be identified, extracts low-level image features from the image to be identified, performs deformation convolution processing on the low-level image features to obtain palm points that match the deformation processing results, and performs standard convolution processing on the low-level image features to obtain hand description information that matches the standard processing results, thereby identifying the hand area in the image to be identified based on the palm points and the hand description information, solving the problems of large amount of computation and high complexity in the hand recognition process of existing hand recognition methods, reducing the amount of computation in the hand recognition process, simplifying the hand recognition task, and thus improving the efficiency of hand recognition.

[0063] Example 2

[0064] Figure 2 This is a flowchart of a method for human hand recognition provided by the second embodiment of the present invention. This embodiment is a further refinement of the above technical solutions, and provides a variety of specific optional implementation methods for performing deformation convolution processing on low-level image features and obtaining palm points that match the deformation processing results, as well as performing standard convolution processing on low-level image features and obtaining human hand description information that matches the standard processing results. The technical solution in this embodiment can be combined with the various optional solutions in one or more of the above embodiments. Figure 2 As shown, the method may include the following steps:

[0065] S210: Acquire an image to be recognized, and extract low-level image features from the image to be recognized.

[0066] Optionally, obtaining the image to be identified may include: obtaining the image to be identified when it is detected that a single-person live broadcast scene or a single-person short video scene condition is met.

[0067] Specifically, when a single-person live broadcast scenario is detected, an image to be identified is obtained, and low-level image features are extracted from the image to be identified. Alternatively, when a single-person short video scenario is detected, an image to be identified is obtained, and low-level image features are extracted from the image to be identified.

[0068] S220 , extracting feature offsets from the low-level image features to obtain a horizontal offset matrix and a vertical offset matrix corresponding to each feature point in the low-level image features.

[0069] The feature offset extraction may include extracting the offset of each feature point in the low-level image features. The horizontal offset matrix may include a matrix composed of the horizontal offsets of each feature point in the low-level image features. The vertical offset matrix may include a matrix composed of the vertical offsets of each feature point in the low-level image features.

[0070] In an embodiment of the present invention, after obtaining the image to be identified and extracting low-level image features from the image to be identified, feature offsets can be further extracted from the low-level image features to obtain a horizontal offset matrix and a vertical offset matrix corresponding to each feature point in the low-level image features, so as to narrow the matrix elements in the horizontal offset matrix and the vertical offset matrix according to preset restriction rules.

[0071] S230 , performing a limit reduction process on the matrix elements in the horizontal offset matrix and the vertical offset matrix according to a preset limit rule, so as to constrain the offset range of each feature point in the low-level image features.

[0072] Among them, the preset restriction rules can be pre-set rules for limiting the matrix elements in the horizontal offset matrix and the vertical offset matrix, which can be used to limit the area of ​​the offset of each feature point, that is, they can be used to constrain the offset range of each feature point in the low-level image features. In an embodiment of the present invention, before the low-level image features are rearranged according to the horizontal offset matrix and the vertical offset matrix to obtain the rearranged image features, the matrix elements in the horizontal offset matrix and the vertical offset matrix can also be narrowed according to the preset restriction rules to constrain the offset range of each feature point in the low-level image features. Exemplarily, the tanh function can be used as an activation function to narrow the matrix elements in the horizontal offset matrix and the vertical offset matrix to constrain the offset range of each feature point in the low-level image features to be between [-1,1].

[0073] S240 : Rearrange the low-level image features according to the horizontal offset matrix and the vertical offset matrix to obtain rearranged image features.

[0074] The rearranged image features may be image features obtained by rearranging low-level image features.

[0075] In an embodiment of the present invention, after extracting the feature offset of the low-level image features and obtaining the horizontal offset matrix and the vertical offset matrix corresponding to each feature point in the low-level image features, the low-level image features can be further rearranged according to the horizontal offset matrix and the vertical offset matrix to obtain the rearranged image features, thereby performing standard convolution processing on the rearranged image features.

[0076] S250 , performing standard convolution processing on the rearranged image features to obtain a rearranged feature extraction result, and acquiring the palm center point according to the rearranged feature extraction result.

[0077] The rearranged feature extraction result may be an image feature result obtained by performing standard convolution processing on the rearranged image features.

[0078] In an embodiment of the present invention, after the low-level image features are rearranged according to the horizontal offset matrix and the vertical offset matrix to obtain the rearranged image features, the rearranged image features can be further subjected to standard convolution processing to obtain the rearranged feature extraction results, thereby obtaining the palm point based on the rearranged feature extraction results.

[0079] Optionally, obtaining the palm point based on the rearranged feature extraction result may include: inputting the rearranged feature extraction result into a palm point heat map generation network to obtain a left hand palm point heat map and a right hand palm point heat map; extracting the left hand palm point and the right hand palm point from the left hand palm point heat map and the right hand palm point heat map, respectively.

[0080] The palm point heatmap generation network may be a network for generating palm point heatmaps. For example, the palm point heatmap generation network may be a multi-layer convolutional network. The left palm point heatmap may be an image obtained by annotating the left palm points according to their level of attention. The right palm point heatmap may be an image obtained by annotating the right palm points according to their level of attention.

[0081] Specifically, after the low-level image features are rearranged according to the horizontal offset matrix and the vertical offset matrix to obtain the rearranged image features, the rearranged image features can be further subjected to standard convolution processing to obtain the rearranged feature extraction results, and the rearranged feature extraction results are input into the palm point heat map generation network to obtain the left palm point heat map and the right palm point heat map, thereby extracting the left palm point and the right palm point in the left palm point heat map and the right palm point heat map, respectively.

[0082] S260: Input the low-level image features into multiple standard convolutional layers one by one to obtain standard feature extraction results corresponding to each hand description information of the left and right hands.

[0083] The standard feature extraction result may be the result of extracting image features after inputting low-level image features into multiple standard convolutional layers.

[0084] In an embodiment of the present invention, after acquiring an image to be recognized and extracting low-level image features from the image to be recognized, the low-level image features can be further sequentially input into multiple standard convolution layers to obtain standard feature extraction results corresponding to each item of hand descriptive information for the left and right hands. For example, the low-level image features can be sequentially input into a standard convolution module comprising five downsampling steps to obtain standard feature extraction results corresponding to the offset between the palm of the left hand and the center of the hand, the standard feature extraction results corresponding to the width and height of the left hand, the standard feature extraction results corresponding to the offset between the palm of the right hand and the center of the hand, and the standard feature extraction results corresponding to the width and height of the right hand.

[0085] S270 , performing average pooling processing on each of the standard feature extraction results to obtain the human hand description information corresponding to the left and right hands respectively.

[0086] In an embodiment of the present invention, after the low-level image features are successively input into multiple standard convolutional layers to obtain the standard feature extraction results corresponding to each hand description information of the left and right hands, each standard feature extraction result can be further average pooled to obtain the hand description information corresponding to the left and right hands, respectively.

[0087] It should be noted that Figure 2 It is only a schematic diagram of one implementation method. There is no order relationship between steps S220-S250 and steps S260-S270. Steps S220-S250 can be implemented first, and then steps S260-S270. Steps S260-S270 can also be implemented first, and then steps S220-S250. They can also be implemented in parallel or one of them can be implemented selectively.

[0088] S280: Identify a human hand region in the image to be identified based on the palm point and the human hand description information.

[0089] The technical solution of this embodiment obtains an image to be recognized, extracts low-level image features from the image, performs feature offset extraction on the low-level image features, obtains horizontal and vertical offset matrices corresponding to each feature point in the low-level image features, then rearranges the low-level image features according to the horizontal and vertical offset matrices to obtain rearranged image features. Standard convolution is then performed on the rearranged image features to obtain rearranged feature extraction results, and palm points are obtained based on the rearranged feature extraction results. The low-level image features are then sequentially input into multiple standard convolutional layers to obtain standard feature extraction results corresponding to each hand description of the left and right hands. Average pooling is performed on each standard feature extraction result to obtain hand description information corresponding to each hand, respectively. Based on the palm points and hand description information, hand regions are identified in the image to be recognized. This solves the problems of high computational complexity and high computational complexity of existing hand recognition methods, reduces the computational load, simplifies the hand recognition task, and improves hand recognition efficiency.

[0090] Example 3

[0091] Figure 3 This is a flowchart of a hand recognition method provided by the third embodiment of the present invention. This embodiment can be applied to reduce the amount of calculation in the hand recognition process and simplify the hand recognition task. The method can be executed by a hand recognition device, which can be implemented by software and / or hardware and can generally be directly integrated into an electronic device that executes this method. The electronic device can be a terminal device or a server device. The embodiment of the present invention does not limit the type of electronic device that executes the hand recognition method. Specifically, Figure 3 As shown, the hand recognition method may specifically include the following steps:

[0092] S310. Acquire an image to be recognized, and input the image to be recognized into a pre-trained hand recognition model; the hand recognition model includes: a first standard convolutional network, and a deformable convolutional branch network and a second standard convolutional branch network respectively connected to the first standard convolutional network.

[0093] The first standard convolutional network can be any standard convolutional network. The deformable convolutional branch network can be a branch network composed of deformable convolutions. The second standard convolutional branch network can be a branch network composed of standard convolutions. It is understood that the input of the deformable convolutional branch network is the output of the first standard convolutional network. At the same time, the input of the second standard convolutional branch network is also the output of the first standard convolutional network.

[0094] In an embodiment of the present invention, an image to be recognized is obtained, and the image to be recognized is input into a pre-trained hand recognition model. Optionally, obtaining the image to be recognized may include: obtaining the image to be recognized when it is detected that the conditions of a single-person live broadcast scene or a single-person short video scene are met. Optionally, before extracting low-level image features from the image to be recognized, image scaling processing may be performed on the image to be recognized to scale the image to be recognized to a preset image size corresponding to the image to be recognized. After extracting low-level image features from the image to be recognized, image scaling processing may be performed on the low-level image features to scale the low-level image features to a preset image size corresponding to the low-level image features, thereby avoiding excessive computational complexity.

[0095] Optionally, when training the hand recognition model, the loss function used may include: a palm point heat map loss item, a hand width and height loss item, and an offset loss item between the palm point and the center point of the hand, wherein: in determining the palm point heat map loss item, the loss value between the palm point prediction value and the palm point annotation value and the palm point offset values ​​around the palm point annotation value is calculated respectively, and the minimum value of each loss value is determined as the loss value of the palm point heat map loss item; in the hand width and height loss item and the offset loss item between the palm point and the center point of the hand, a smoothing loss function is used.

[0096] Among them, the palm point heat map loss item can be a loss function in the process of determining the palm point heat map. The hand width and height loss item can be a loss function in the process of determining the hand width and height. The offset loss item between the palm point and the center point of the hand can be a loss function in the process of determining the offset between the palm point and the center point of the hand. The palm point prediction value can be the palm point coordinate value predicted by the hand recognition model. The palm point annotation value can be the palm point coordinate value obtained by annotation. The palm point offset values ​​around the palm point annotation value can be the palm point offset value to the lower right of the palm point annotation value, the palm point offset value to the lower left of the palm point annotation value, the palm point offset value to the upper right of the palm point annotation value, the palm point offset value to the upper left of the palm point annotation value, etc., and the embodiment of the present invention does not limit this.

[0097] Specifically, when training a hand recognition model, the loss function used may include: palm heat map loss term, hand width and height loss term, and offset loss term between the palm and hand center. Optionally, when training a hand recognition model, the loss function used may be calculated according to the following formula:

[0098] L=λ0L heatmap +λ1L hw +λ2L offset

[0099] Among them, L represents the loss function used when training the hand recognition model; L heatmap represents the palm point heat map loss term; λ0 represents the weight of the palm point heat map loss term; L hw represents the loss term of the width and height of the hand; λ1 represents the weight of the loss term of the width and height of the hand; L offset represents the offset loss term between the palm point and the center point of the hand; λ2 represents the weight of the offset loss term between the palm point and the center point of the hand.

[0100] Specifically, to determine the palm heatmap loss term, the loss between the predicted palm value, the palm annotation value, and the offset values ​​of each palm point around the annotation value is calculated. The minimum of these loss values ​​is used as the loss value for the palm heatmap loss term. A smoothing loss function is used for both the hand width and height loss term and the offset loss term between the palm point and the hand center.

[0101] Optionally, the palm point heat map loss term can be determined according to the following formula:

[0102]

[0103] Among them, H pred It can be the palm point prediction value; H gt It can be the palm point mark value; H gt ++ can be the palm point offset value to the lower right of the palm point mark value; H gt --It can be the palm point offset value to the upper left of the palm point mark value; H gt ± can be the palm point offset value to the upper right of the palm point mark value; It can be the palm point offset value to the lower left of the palm point label value.

[0104] Optionally, the hand width and height loss term can be determined according to the following formula:

[0105] L hw =SmoothL1(height pred ,height gt )+SmoothL1(width pred ,width gt )

[0106] Among them, height pred Indicates the predicted value of the hand height; height gt Indicates the hand height mark value; width pred Indicates the predicted value of the hand width; width gt Indicates the hand width dimension value.

[0107] Optionally, the offset loss term between the palm point and the center point of the hand can be determined according to the following formula:

[0108] L offset =SmoothL1(x pred ,x gt )+SmoothL1(y pred ,y gt )

[0109] Among them, x pred Indicates the horizontal predicted offset between the palm point and the center point of the hand; x gt Indicates the horizontal offset between the palm point and the center point of the hand; y pred Indicates the vertical predicted offset between the palm point and the center point of the hand; y gt Indicates the vertical offset between the palm point and the center point of the hand.

[0110] The above technical solution determines the loss function used when training the hand recognition model by using the palm heatmap loss term, the hand width and height loss term, and the offset loss term between the palm and the hand center. This can avoid excessive data errors in the hand recognition model due to inaccurate labeling values. By using the SmoothL1 loss function, the problem of non-smooth derivatives when the L1 loss is 0 can be avoided.

[0111] S320. Extract low-level image features from the image to be identified through the first standard convolutional network, and input the low-level image features into the deformable convolutional branch network and the second standard convolutional branch network respectively.

[0112] S330: Perform deformation convolution processing on the low-level image features through the deformation convolution branch network, and obtain the palm center point that matches the deformation processing result.

[0113] Optionally, deformable convolution processing is performed on low-level image features through a deformable convolution branch network, and a palm point matching the deformation processing result is obtained, which may include: extracting feature offsets on low-level image features through a deformable convolution branch network, and obtaining a horizontal offset matrix and a vertical offset matrix corresponding to each feature point in the low-level image features; rearranging the low-level image features according to the horizontal offset matrix and the vertical offset matrix through the deformable convolution branch network to obtain rearranged image features; performing standard convolution processing on the rearranged image features through the deformable convolution branch network to obtain rearranged feature extraction results, and obtaining the palm point based on the rearranged feature extraction results through the deformable convolution branch network.

[0114] Optionally, before rearranging the low-level image features according to the horizontal offset matrix and the vertical offset matrix through the deformable convolution branch network to obtain the rearranged image features, it can also include: limiting the matrix elements in the horizontal offset matrix and the vertical offset matrix according to preset restriction rules through the deformable convolution branch network to constrain the offset range of each feature point in the low-level image features.

[0115] Optionally, obtaining the palm point based on the rearranged feature extraction result through the deformable convolution branch network can include: inputting the rearranged feature extraction result into the palm point heat map generation network through the deformable convolution branch network to obtain the left palm point heat map and the right palm point heat map; extracting the left palm point and the right palm point from the left palm point heat map and the right palm point heat map respectively through the deformable convolution branch network.

[0116] S340. Perform standard convolution processing on the low-level image features through the second standard convolution branch network, and obtain hand description information that matches the standard processing results. The hand description information includes: the offset between the palm point and the center point of the hand, and the width and height of the hand.

[0117] Optionally, standard convolution processing is performed on the low-level image features through a second standard convolution branch network, and hand description information matching the standard processing results is obtained. This can include: inputting the low-level image features into multiple standard convolution layers in sequence through the second standard convolution branch network to obtain standard feature extraction results corresponding to each item of hand description information of the left and right hands; and performing average pooling processing on each standard feature extraction result through the second standard convolution branch network to obtain each hand description information corresponding to the left and right hands.

[0118] S350: Identify a hand area in the image to be identified based on the palm point and hand description information output by the hand recognition model.

[0119] In a specific example of an embodiment of the present invention, a single-person live broadcast scene or a single-person short video scene is used as a specific application scenario. Figure 4 This is an example flow chart of a method for identifying a hand provided by the third embodiment of the present invention. Figure 4 As shown, the method may specifically include the following contents:

[0120] (1) Obtain an image to be recognized, and input the image to be recognized into a pre-trained hand recognition model, so as to perform image scaling processing on the image to be recognized through the hand recognition model to obtain an image to be recognized with a uniform size of 192*192. Specifically, the hand recognition model may include: a first standard convolutional network, and a deformable convolutional branch network and a second standard convolutional branch network respectively connected to the first standard convolutional network. When training the hand recognition model, the loss function used includes: a palm point heat map loss term, a hand width and height loss term, and an offset loss term between the palm point and the center point of the hand.

[0121] (2) The image to be identified after image scaling processing is input into the ordinary convolution module (i.e., the first standard convolution network) to extract low-level image features from the image to be identified through the ordinary convolution module.

[0122] (3) The low-level image features are scaled to obtain low-level image features of a uniform size of 96*96, and the low-level image features after image scaling are input into the deformable convolution layer (i.e., the deformable convolution branch network) and the ordinary convolution module (i.e., the second standard convolution branch network) respectively.

[0123] (4) Perform deformable convolution processing on the low-level image features through the deformable convolution layer, and obtain the palm center point that matches the deformation processing result. Specifically, Figure 5 : is a structural diagram of a deformable convolution module provided in the third embodiment of the present invention, such as Figure 5 As shown, the first ordinary convolutional layer of the deformable convolutional layer extracts feature offsets from low-level image features to obtain feature offsets (i.e., the horizontal offset matrix and vertical offset matrix corresponding to each feature point in the low-level image features). The tanh activation function is then used to narrow the matrix elements in the horizontal and vertical offset matrices to constrain the offset range of each feature point in the low-level image features. The low-level image features are rearranged according to the horizontal and vertical offset matrices to obtain rearranged image features. The second ordinary convolutional layer of the deformable convolutional layer performs standard convolution on the rearranged image features to obtain rearranged feature extraction results. The rearranged feature extraction results are then input into the ordinary convolutional layer (i.e., the palm point heatmap generation network). Feature extraction and fusion are performed step by step through multiple convolutional layers to obtain left and right palm point heatmaps. The left and right palm points are extracted from the left and right palm point heatmaps, respectively.

[0124] In this technical solution, the deformation convolution process does not fix the operating area of ​​feature points, which can expand the receptive field. Rearranging low-level image features according to horizontal and vertical offset matrices can bring together scattered finger features, especially when grasping objects. The large receptive field enables the network to better detect the entire hand, making it easier to predict the palm point.

[0125] (5) Performing standard convolution processing on the low-level image features through the ordinary convolution module and obtaining hand description information that matches the standard processing results. Specifically, the low-level image features are successively input into five down-sampled standard convolution layers through the ordinary convolution module to obtain standard feature extraction results corresponding to each hand description information of the left and right hands, respectively. The feature map size of the standard feature extraction results is 3*3, and each standard feature extraction result is average pooled to obtain the hand description information corresponding to the left and right hands.

[0126] (6) Based on the palm point and hand description information output by the hand recognition model, the hand area is identified in the image to be recognized. Figure 6 is a schematic diagram of identifying a human hand area provided by the third embodiment of the present invention, such as Figure 6 As shown, based on the palm point, hand width and height values, and the offset between the palm point and the hand center point output by the hand recognition model, the hand area can be identified in the image to be recognized.

[0127] The above technical solution changes the existing hand recognition algorithm process. It does not directly predict the rectangular box or the offset of the rectangular box, which can simplify the hand recognition task and improve the hand recognition effect. It can better solve the multi-scale problem caused by the different sizes of hands in the image and reduce the amount of calculation. Through the deformable convolution layer, it can solve the non-rigid deformation problem caused by the hand shape changing with the gesture and the relative position of the fingers being not fixed.

[0128] The technical solution of this embodiment is to obtain an image to be recognized, input the image to be recognized into a pre-trained hand recognition model, and extract low-level image features from the image to be recognized through the first standard convolutional network in the hand recognition model. The low-level image features are respectively input into the deformable convolutional branch network and the second standard convolutional branch network. The deformable convolutional branch network in the hand recognition model performs deformable convolution processing on the low-level image features to obtain a palm point that matches the deformable processing result, and the second standard convolutional branch network in the hand recognition model performs standard convolution processing on the low-level image features to obtain hand description information that matches the standard processing result. Thus, based on the palm point and hand description information output by the hand recognition model, the hand area is identified in the image to be recognized. This solves the problems of large computational complexity and high complexity in the hand recognition process of existing hand recognition methods, can reduce the computational complexity in the hand recognition process, simplify the hand recognition task, and thus improve the efficiency of hand recognition.

[0129] Example 4

[0130] Figure 7 Schematic diagram of a hand recognition device provided by the fourth embodiment of the present invention. Figure 7 As shown, the device includes: a low-level image feature extraction module 710, a palm point acquisition module 720, a human hand description information acquisition module 730 and a human hand region recognition module 740, wherein:

[0131] A low-level image feature extraction module 710 is used to obtain an image to be identified and extract low-level image features from the image to be identified;

[0132] The palm point acquisition module 720 is used to perform deformation convolution processing on low-level image features and obtain the palm point that matches the deformation processing result;

[0133] The hand description information acquisition module 730 is used to perform standard convolution processing on low-level image features and obtain hand description information that matches the standard processing results. The hand description information includes: the offset between the palm point and the center point of the hand, and the width and height of the hand;

[0134] The hand region recognition module 740 is used to recognize the hand region in the image to be recognized based on the palm point and the hand description information.

[0135] The technical solution of this embodiment obtains an image to be identified, extracts low-level image features from the image to be identified, performs deformation convolution processing on the low-level image features to obtain palm points that match the deformation processing results, and performs standard convolution processing on the low-level image features to obtain hand description information that matches the standard processing results, thereby identifying the hand area in the image to be identified based on the palm points and the hand description information, solving the problems of large amount of computation and high complexity in the hand recognition process of existing hand recognition methods, reducing the amount of computation in the hand recognition process, simplifying the hand recognition task, and thus improving the efficiency of hand recognition.

[0136] Optionally, the palm point acquisition module 720 can be specifically used to: extract feature offsets of low-level image features to obtain horizontal offset matrices and vertical offset matrices corresponding to each feature point in the low-level image features; rearrange the low-level image features according to the horizontal offset matrix and the vertical offset matrix to obtain rearranged image features; perform standard convolution processing on the rearranged image features to obtain rearranged feature extraction results, and obtain the palm point based on the rearranged feature extraction results.

[0137] Optionally, the palm point acquisition module 720 may be further configured to: perform a limit processing on the matrix elements in the horizontal offset matrix and the vertical offset matrix according to a preset limit rule, so as to constrain the offset range of each feature point in the low-level image features.

[0138] Optionally, the palm point acquisition module 720 can be further used to: input the rearranged feature extraction results into the palm point heat map generation network to obtain the left hand palm point heat map and the right hand palm point heat map; and extract the left hand palm point and the right hand palm point from the left hand palm point heat map and the right hand palm point heat map, respectively.

[0139] Optionally, the hand description information acquisition module 730 can be specifically used to: input low-level image features into multiple standard convolutional layers in sequence to obtain standard feature extraction results corresponding to each hand description information of the left and right hands, respectively; perform average pooling processing on each standard feature extraction result to obtain each hand description information corresponding to the left and right hands, respectively.

[0140] Optionally, the low-level image feature extraction module 710 may be specifically used to obtain an image to be identified when it is detected that the conditions of a single-person live broadcast scene or a single-person short video scene are met.

[0141] The hand recognition device provided in the embodiment of the present invention can execute the hand recognition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0142] Example 5

[0143] Figure 8is a schematic diagram of a hand recognition device provided by the fifth embodiment of the present invention, such as Figure 8 As shown, the apparatus includes: an image processing module to be identified 810, an image processing module to be identified 820, a first low-level image feature processing module 830, a second low-level image feature processing module 840, and a human hand region recognition module 850, wherein:

[0144] The image processing module 810 is used to obtain an image to be recognized and input the image to be recognized into a pre-trained hand recognition model; the hand recognition model includes: a first standard convolutional network, and a deformable convolutional branch network and a second standard convolutional branch network respectively connected to the first standard convolutional network;

[0145] The image processing module 820 is used to extract low-level image features from the image to be identified through the first standard convolutional network, and input the low-level image features into the deformable convolutional branch network and the second standard convolutional branch network respectively;

[0146] A first low-level image feature processing module 830 is configured to perform deformation convolution processing on the low-level image features through a deformation convolution branch network, and obtain a palm point that matches the deformation processing result;

[0147] A second low-level image feature processing module 840 is configured to perform standard convolution processing on the low-level image features using a second standard convolution branch network, and obtain hand description information that matches the standard processing result. The hand description information includes: the offset between the palm point and the center point of the hand, and the width and height of the hand;

[0148] The hand region recognition module 850 is configured to recognize the hand region in the image to be recognized based on the palm point and hand description information output by the hand recognition model.

[0149] The technical solution of this embodiment is to obtain an image to be recognized, input the image to be recognized into a pre-trained hand recognition model, and extract low-level image features from the image to be recognized through the first standard convolutional network in the hand recognition model. The low-level image features are respectively input into the deformable convolutional branch network and the second standard convolutional branch network. The deformable convolutional branch network in the hand recognition model performs deformable convolution processing on the low-level image features to obtain a palm point that matches the deformable processing result, and the second standard convolutional branch network in the hand recognition model performs standard convolution processing on the low-level image features to obtain hand description information that matches the standard processing result. Thus, based on the palm point and hand description information output by the hand recognition model, the hand area is identified in the image to be recognized. This solves the problems of large computational complexity and high complexity in the hand recognition process of existing hand recognition methods, can reduce the computational complexity in the hand recognition process, simplify the hand recognition task, and thus improve the efficiency of hand recognition.

[0150] Optionally, when training a hand recognition model, the loss function used may include: a palm point heat map loss item, a hand width and height loss item, and an offset loss item between the palm point and the center point of the hand, wherein: in determining the palm point heat map loss item, the loss values ​​between the palm point prediction value and the palm point annotation value and the palm point offset values ​​around the palm point annotation value are calculated respectively, and the minimum value of each loss value is determined as the loss value of the palm point heat map loss item; in the hand width and height loss item and the offset loss item between the palm point and the center point of the hand, a smoothing loss function is used.

[0151] The hand recognition device provided in the embodiment of the present invention can execute the hand recognition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0152] Example 6

[0153] Figure 9 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0154] like Figure 9As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0155] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0156] The processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the hand recognition method.

[0157] In some embodiments, the hand recognition method may be implemented as a computer program that is tangibly contained in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the hand recognition method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the hand recognition method in any other appropriate manner (e.g., by means of firmware).

[0158] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0159] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0160] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0161] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0162] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0163] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0164] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0165] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for identifying a hand, characterized in that: include: Obtain an image to be identified and extract low-level image features from the image to be identified; Extracting feature offsets of low-level image features to obtain horizontal offset matrices and vertical offset matrices corresponding to respective feature points in the low-level image features; Rearranging the low-level image features according to the horizontal offset matrix and the vertical offset matrix to obtain rearranged image features; Performing standard convolution processing on the rearranged image features to obtain a rearranged feature extraction result, and obtaining a palm point based on the rearranged feature extraction result; The low-level image features are successively input into multiple standard convolutional layers to obtain the standard feature extraction results corresponding to each hand description information of the left and right hands; Performing average pooling processing on the standard feature extraction results to obtain hand description information corresponding to the left and right hands respectively, the hand description information including: the offset between the palm point and the center point of the hand, and the width and height of the hand; Identify the hand area in the image to be identified based on the palm point and hand description information.

2. The method according to claim 1, characterized in that Before rearranging the low-level image features according to the horizontal offset matrix and the vertical offset matrix to obtain rearranged image features, the method further includes: According to preset restriction rules, the matrix elements in the horizontal offset matrix and the vertical offset matrix are narrowed down to constrain the offset range of each feature point in the low-level image features.

3. The method according to claim 1, characterized in that Acquiring the palm point according to the rearranged feature extraction result includes: Inputting the rearranged feature extraction result into the palm point heat map generation network to obtain the left palm point heat map and the right palm point heat map; In the left palm center point heat map and the right palm center point heat map, the left palm center point and the right palm center point are extracted respectively.

4. The method according to claim 1, wherein Get the image to be recognized, including: When it is detected that the conditions of a single-person live broadcast scene or a single-person short video scene are met, the image to be identified is obtained.

5. A method for identifying a hand, characterized in that: include: Acquire an image to be recognized, and input the image to be recognized into a pre-trained hand recognition model; The hand recognition model includes: a first standard convolutional network, and a deformable convolutional branch network and a second standard convolutional branch network respectively connected to the first standard convolutional network; Extract low-level image features from the image to be recognized through the first standard convolutional network, and input the low-level image features into the deformable convolutional branch network and the second standard convolutional branch network respectively; Extracting feature offsets from low-level image features using a deformable convolutional branch network to obtain a horizontal offset matrix and a vertical offset matrix corresponding to each feature point in the low-level image features; rearranging the low-level image features according to the horizontal offset matrix and the vertical offset matrix to obtain rearranged image features; performing standard convolution processing on the rearranged image features to obtain rearranged feature extraction results, and obtaining a palm point based on the rearranged feature extraction results; The low-level image features are sequentially input into multiple standard convolutional layers through a second standard convolutional branch network to obtain standard feature extraction results corresponding to each hand description information of the left and right hands. Each of the standard feature extraction results is average pooled to obtain the hand description information corresponding to the left and right hands, respectively. The hand description information includes: the offset between the palm point and the center point of the hand, and the width and height of the hand; According to the palm point and hand description information output by the hand recognition model, the hand area is identified in the image to be recognized.

6. The method according to claim 5, characterized in that When training the hand recognition model, the loss functions used include: palm heat map loss, hand width and height loss, and offset loss between the palm and hand center, where: In determining the palm point heat map loss item, respectively calculating the loss values ​​between the palm point prediction value, the palm point annotation value, and the palm point offset values ​​around the palm point annotation value, and determining the minimum value of the loss values ​​as the loss value of the palm point heat map loss item; A smooth loss function is used in both the hand width and height loss item and the offset loss item between the palm point and the hand center point.

7. A hand recognition device, characterized in that: include: A low-level image feature extraction module is used to obtain an image to be identified and extract low-level image features from the image to be identified; a palm point acquisition module, configured to extract feature offsets from low-level image features, obtain a horizontal offset matrix and a vertical offset matrix corresponding to each feature point in the low-level image features, and rearrange the low-level image features according to the horizontal offset matrix and the vertical offset matrix to obtain rearranged image features; Performing standard convolution processing on the rearranged image features to obtain a rearranged feature extraction result, and obtaining a palm point based on the rearranged feature extraction result; The hand description information acquisition module is used to input low-level image features into multiple standard convolutional layers in sequence to obtain standard feature extraction results corresponding to each hand description information of the left and right hands; Performing average pooling processing on the standard feature extraction results to obtain hand description information corresponding to the left and right hands respectively, the hand description information including: the offset between the palm point and the center point of the hand, and the width and height of the hand; The hand area recognition module is used to identify the hand area in the image to be recognized based on the palm point and hand description information.

8. A hand recognition device, characterized in that: include: An image processing module for obtaining an image to be recognized and inputting the image to be recognized into a pre-trained hand recognition model; The hand recognition model includes: a first standard convolutional network, and a deformable convolutional branch network and a second standard convolutional branch network respectively connected to the first standard convolutional network; The image processing module to be identified is used to extract low-level image features from the image to be identified through the first standard convolutional network, and input the low-level image features into the deformable convolutional branch network and the second standard convolutional branch network respectively; a first low-level image feature processing module, configured to extract feature offsets from low-level image features using a deformable convolutional branch network to obtain a horizontal offset matrix and a vertical offset matrix corresponding to each feature point in the low-level image features; rearrange the low-level image features according to the horizontal offset matrix and the vertical offset matrix to obtain rearranged image features; perform standard convolution processing on the rearranged image features to obtain rearranged feature extraction results, and obtain a palm point based on the rearranged feature extraction results; A second low-level image feature processing module is configured to sequentially input the low-level image features into multiple standard convolutional layers via a second standard convolutional branch network to obtain standard feature extraction results corresponding to each hand description information of the left and right hands; perform average pooling processing on each of the standard feature extraction results to obtain hand description information corresponding to the left and right hands, respectively. The hand description information includes: an offset between the palm point and the center point of the hand, and a width and height value of the hand; The hand region recognition module is used to recognize the hand region in the image to be recognized based on the palm point and hand description information output by the hand recognition model.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the human hand recognition method described in any one of claims 1-4, or execute the human hand recognition method described in any one of claims 5-6.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the human hand recognition method according to any one of claims 1 to 4, or to implement the human hand recognition method according to any one of claims 5 to 6 when executed.

Images