A hand recognition method and device based on a multi-view camera and a storage medium

Abstract

The application discloses a hand recognition method and device based on a multi-view camera and a storage medium. The method comprises the following steps: collecting a plurality of hand images of a hand to be recognized; extracting a hand mesh and a hand mask of the hand image; converting each hand image into a corresponding feature map; extracting a feature value of each hand feature point in the corresponding hand mesh in the feature map, and splicing to obtain a hand low-order feature corresponding to each feature map; fusing the hand low-order feature of each feature map with the corresponding hand mask to obtain a hand low-order feature fused with each hand feature point, determining a final hand feature of the hand to be recognized according to the hand low-order feature fused with the hand feature point, determining hand visibility of the hand to be recognized according to all hand masks, and recognizing the hand to be recognized according to the hand visibility and the final hand feature. The application can improve the convenience of hand recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of biometric identification, and in particular to a hand recognition method and device based on a multi-view camera and a storage medium. BACKGROUND

[0002] Hand recognition technology is based on human hands, and uses the characteristics of hands to achieve identity recognition. These features mainly include hand veins, etc. Although the structure of human hands is relatively simple, it has strong diversity and polysemy, and the hand features remain relatively stable after adulthood, which is a good biometric feature for identity recognition.

[0003] Existing hand recognition technology is greatly affected by hand posture, and hands of different angles and gestures are difficult to recognize each other. In recognition, the user's hand posture needs to be strictly constrained, and the user needs to pose a prescribed hand posture (such as opening the palm) to perform recognition. That is, the existing hand recognition technology is easily affected by the hand posture, and the user needs to deliberately cooperate with the prescribed hand posture to complete the hand recognition, which has great limitations and is not convenient. SUMMARY

[0004] The embodiments of the present application provide a hand recognition method and device based on a multi-view camera and a storage medium, which can reduce the influence of hand posture in existing hand recognition technology and improve the convenience of hand recognition.

[0005] An embodiment of the present application provides a hand recognition method based on a multi-view camera, comprising:

[0006] Obtaining a plurality of hand images of a hand to be recognized collected by a multi-view camera; wherein each hand image is collected by a camera in the multi-view camera;

[0007] For each hand image, extracting a hand Mesh of the hand image and a hand Mask for representing the visibility of each hand feature point in the hand Mesh;

[0008] Converting each hand image into a corresponding feature map; for each feature map, extracting the feature value of each hand feature point in the corresponding hand Mesh in the feature map, and splicing to obtain a hand low-order feature corresponding to each feature map;

[0009] Fusing the hand low-order feature of each feature map with the corresponding hand Mask to obtain a hand low-order feature after fusion of each hand feature point, and determining the final hand feature of the hand to be recognized according to the hand low-order feature after fusion of the hand feature point;

[0010] Determining the hand visibility of the hand to be recognized according to all hand Masks;

[0011] According to the hand visibility of the hand to be identified and the final hand features, the hand to be identified is identified.

[0012] Further, the hand Mesh extracted from the hand image and the hand Mask used to represent the visibility of each hand feature point in the hand Mesh include:

[0013] Detect a hand frame in the hand image, and extract a hand local image located in the hand frame according to the hand frame;

[0014] According to the hand local image, a hand Mesh and a hand Mask are identified; wherein the hand Mask is a Boolean vector, when a hand feature point in the hand Mesh is a visible hand feature point, the corresponding value of the hand feature point in the hand Mask is 1, and when a hand feature point in the hand Mesh is an invisible hand feature point, the corresponding value of the hand feature point in the hand Mask is 0.

[0015] Further, each hand image is converted into a corresponding feature map; for each feature map, the feature value of each hand feature point in the corresponding hand Mesh in the feature map is extracted, and spliced to obtain a low-order feature corresponding to each feature map, including:

[0016] The hand image is converted into a corresponding feature map through a preset neural network;

[0017] For each feature map, the transverse coordinate value and the longitudinal coordinate value of each hand feature point in the hand Mesh are used as indexes to extract the feature value of each hand feature point in the corresponding hand Mesh in the feature map;

[0018] The feature values of all hand feature points in the hand Mesh in the feature map are spliced to obtain the hand low-order feature corresponding to each feature map.

[0019] Further, the fused hand low-order feature is determined by the following formula:

[0020]

[0021] wherein, E merge is the fused hand low-order feature of the jth hand feature point; w i is the weight coefficient corresponding to the jth hand feature point in the ith feature map; E i is the corresponding feature value of the jth hand feature point in the ith feature map; eps is a preset value to prevent the denominator from being zero; n is the total number of feature maps; k is the total number of hand feature points; Mask i[j] is the value of the jth hand feature point in the hand mask corresponding to the ith feature map;d i [j] is the depth value of the jth hand feature point in the hand mesh corresponding to the ith feature map.

[0022] Further, the hand visibility is determined by the following formula:

[0023]

[0024] wherein V1[j] is the hand visibility of the jth hand feature point in the hand to be identified.

[0025] Further, the hand to be identified is identified according to the hand visibility of the hand to be identified and the final hand feature, comprising:

[0026] calculating the similarity score of the hand visibility of the hand to be identified with the hand visibility in the pre-stored data, to obtain a first similarity score;

[0027] calculating the similarity score of the final hand feature of the hand to be identified with the final hand feature in the pre-stored data, to obtain a second similarity score;

[0028] when it is determined that the first similarity score and the second similarity score are both greater than the corresponding preset threshold, it is determined that the hand to be identified and the hand corresponding to the pre-stored data are the same hand.

[0029] Further, the first similarity score is calculated by the following formula:

[0030]

[0031] wherein S v is the first similarity score; V2[j] is the hand visibility of the jth hand feature point in the hand corresponding to the pre-stored data.

[0032] Further, the second similarity score is calculated by the following formula:

[0033]

[0034] wherein S f is the second similarity score; F1 is the final hand feature of the hand to be identified; F1 is the final hand feature of the hand corresponding to the pre-stored data.

[0035] On the basis of the above method embodiment, the present application correspondingly provides a device embodiment;

[0036] An embodiment of the present application provides a hand recognition device based on a multi-view camera, comprising an image acquisition module, a hand detection module, a feature extraction module, a feature fusion module, a visibility determination module and a comparison module.

[0037] The image acquisition module is configured to acquire a plurality of hand images of a hand to be recognized acquired by the multi-view camera; each hand image is acquired by one camera in the multi-view camera.

[0038] The hand detection module is configured to extract, for each hand image, a hand mesh of the hand image and a hand mask for representing the visibility of each hand feature point in the hand mesh.

[0039] The feature extraction module is configured to convert each hand image into a corresponding feature map; for each feature map, the feature extraction module extracts the feature value of each hand feature point in the corresponding hand mesh in the feature map and splices to obtain a hand low-order feature corresponding to each feature map.

[0040] The feature fusion module is configured to fuse the hand low-order feature of each feature map with the corresponding hand mask to obtain a hand low-order feature after fusion of each hand feature point, and determine a final hand feature of the hand to be recognized according to the hand low-order feature after fusion of the hand feature point.

[0041] The visibility determination module is configured to determine the hand visibility of the hand to be recognized according to all hand masks.

[0042] The comparison module is configured to recognize the hand to be recognized according to the hand visibility and the final hand feature of the hand to be recognized.

[0043] On the basis of the above-mentioned method embodiment, the present application correspondingly provides a storage medium embodiment.

[0044] An embodiment of the present application provides a storage medium, which comprises a stored computer program, wherein the computer program controls a device where the storage medium is located to execute the hand recognition method based on the multi-view camera according to any one of the embodiments of the present application when the computer program is running.

[0045] By implementing the embodiments of the present application, the following beneficial effects are achieved:

[0046] This invention provides a method, apparatus, and storage medium for hand recognition based on a multi-view camera. The method first acquires several hand images of the hand to be recognized. For each hand image, a hand mesh and a hand mask representing the visibility of each hand feature point in the hand mesh are extracted. Each hand image is converted into a corresponding feature map. For each feature map, the feature values ​​of each hand feature point in the corresponding hand mesh are extracted and concatenated to obtain the low-order hand features corresponding to each feature map. The low-order hand features of each feature map are fused with the corresponding hand mask to obtain the fused low-order hand features for each hand feature point. Based on the fused low-order hand features, the final hand features of the hand to be recognized are determined. The hand visibility of the hand to be recognized is determined based on all hand masks. Finally, the hand to be recognized is identified based on the hand visibility and the final hand features. Compared with existing technologies, this invention extracts three-dimensional coordinates for each hand image's extracted hand mesh vertices, and the feature values ​​in the feature map can be mapped to the mesh vertices through projection relationships. When hand pose changes, the three-dimensional coordinates of the mesh vertices change, but this application deforms and aligns the features, transforming mesh vertices of different poses into a normalized space. This allows features from different poses to be compared in the same coordinate system. In this case, feature differences are determined only by hand identity (such as hand veins), not pose, thus reducing the influence of hand pose. This allows users to recognize hands using any hand pose, improving the convenience of hand recognition. Attached Figure Description

[0047] Figure 1 This is a flowchart illustrating a hand recognition method based on a multi-view camera, provided in an embodiment of the present invention.

[0048] Figure 2 This is a schematic diagram illustrating the principle of a hand recognition method based on a multi-view camera, provided by an embodiment of the present invention.

[0049] Figure 3 This is a schematic diagram illustrating the principle of feature extraction in a hand recognition method based on a multi-view camera, provided by an embodiment of the present invention.

[0050] Figure 4 This is a schematic diagram of a hand recognition device based on a multi-view camera provided in an embodiment of the present invention. Detailed Implementation

[0051] With reference to the drawings and embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0052] As shown in the figure, Figure 1 An embodiment of the present application provides a hand recognition method based on a multi-view camera, at least comprising the following steps:

[0053] Step S1: acquiring a plurality of hand images of a hand to be recognized collected by a multi-view camera; wherein each hand image is collected by a camera in the multi-view camera.

[0054] In the present application, the hand is placed in the working range of the multi-view device, and a plurality of hand images {P1, P2...Pn} are obtained by shooting a hand static image, wherein n is the number of cameras of the multi-view device (which can also be the total number of subsequent feature maps);

[0055] Referring to Figure 2 , in an embodiment of the present application, four infrared cameras (i.e. four infrared cameras) are arranged in the multi-view device (i.e. the above multi-view camera), and the four infrared cameras are arranged in four different directions of up, down, left and right, each camera is 90 degrees apart, and the parameters of each camera are consistent. The user places the hand to be recognized in the collection area of the multi-view camera, and the four infrared cameras shoot the hand to be recognized respectively to obtain four hand images P1, P2, P3 and P4. Of course, the number of infrared cameras only needs to meet two or more, and the specific number can not be limited. Compared with the general visible light camera, the infrared camera can capture more rich hand vein information, and can also reduce the interference of handprint, stain and other information. Through the four infrared cameras arranged in different directions, the information of palm veins, finger veins and back veins can be collected, and the accuracy is higher than that of single palm vein and finger vein algorithm, and it is difficult to recognize due to insufficient local vein information. In addition, because the multi-view images of different angles of the hand are used at the same time, the hand has three-dimensional information in the recognition feature, and the two-dimensional photo obtained by general printing cannot be attacked successfully, and has strong anti-fake ability.

[0056] Step S2: for each hand image, extracting a hand Mesh of the hand image and a hand Mask for representing the visibility of each hand feature point in the hand Mesh.

[0057] In a preferred embodiment, the hand Mesh extracted from the hand image and the hand Mask used to represent the visibility of each hand feature point in the hand Mesh include:

[0058] detecting a hand frame in the hand image and extracting a hand local image located in the hand frame according to the hand frame;

[0059] identifying a hand Mesh and a hand Mask according to the hand local image; wherein the hand Mask is a Boolean vector, when a hand feature point in the hand Mesh is a visible hand feature point, the value corresponding to the hand feature point in the hand Mask is 1, and when a hand feature point in the hand Mesh is an invisible hand feature point, the value corresponding to the hand feature point in the hand Mask is 0.

[0060] Specifically, according to the image {P1, P2...Pn}, the hand Mesh and the hand Mask are {Mesh1, Mesh2...Meshn} and {Mask1, Mask2...Maskn}.

[0061] Illustratively, in the present application, the form of the hand Mesh is the 3D coordinates of k points, k is determined by the hand Mesh model, and in the present embodiment, the MANO model is used, so k = 778; the hand Mask corresponds to the visibility of the k points in the hand Mesh (its form is a k-dimensional Boolean vector, and the value is 1 when visible and 0 when invisible).

[0062] Specifically, the hand frame in the hand image is first detected using the target detection algorithm disclosed by YOLO, the hand local image is extracted according to the hand frame, and the hand Mesh and the hand Mask are further detected using the MANO hand Mesh detection algorithm.

[0063] Step S3: converting each hand image into a corresponding feature map; for each feature map, extracting the feature value of each hand feature point in the corresponding hand Mesh in the feature map, and splicing to obtain the hand low-order feature corresponding to each feature map.

[0064] In a preferred embodiment, the hand Mesh extracted from the hand image and the hand Mask used to represent the visibility of each hand feature point in the hand Mesh include:

[0065] converting the hand image into a corresponding feature map through a preset neural network;

[0066] For each feature map, the feature values ​​of each hand feature point in the corresponding hand mesh are extracted using the horizontal and vertical coordinates of each hand feature point as an index.

[0067] The feature values ​​of all hand feature points in the hand mesh are concatenated in the feature map to obtain the low-order hand features corresponding to each feature map.

[0068] Specifically, by extracting features from {P1,P2...Pn} and {Mesh1,Mesh2...Meshn}, the low-level hand features {E1,E2...En} of the hand to be identified can be obtained.

[0069] like Figure 3 As shown, taking an input hand image as an example: for a hand image P and its corresponding hand mesh, extract the low-level hand features E. Here, E is the low-level hand feature corresponding to the mesh, a tensor of dimension (k, s), where s is a preset value of the algorithm; in this embodiment, k = 778 and s = 64. The specific steps are as follows:

[0070] The hand image P is transformed into a feature map E-Map using a neural network with a U-net structure. The dimensions of P are (640, 400, 3), and the dimensions of E-Map are (640, 400, 64).

[0071] In the E-Map, deformation alignment is performed based on the mesh coordinates to obtain the low-order hand feature E. The deformation alignment operation is as follows: a. For the coordinates (uj, vj, dj) of the j-th point in the mesh, the feature value ej of the index (uj, vj) in the E-Map, where ej is an s-dimensional vector; b. Concatenate {e1, e2... ek} together to obtain E. uj is the horizontal coordinate value of the j-th hand feature point, vj is the vertical coordinate value of the j-th hand feature point, and dj represents the depth value of the j-th hand feature point in 3D space, reflecting the distance of the j-th hand feature point relative to the camera (a larger value usually indicates that the vertex is farther from the camera).

[0072] To illustrate, the low-level hand features obtained from the above four hand images P1, P2, P3 and P4 after the above feature extraction are E1, E2, E3 and E4, respectively.

[0073] In this embodiment of the invention, by deforming and aligning the features, the Mesh vertices of different poses are transformed into a normalized space, so that features of different poses can be compared in the same coordinate system. At this time, the feature differences are determined only by the hand identity (such as hand veins) rather than the pose, thereby reducing the influence of hand pose. This allows users to identify hands using any hand pose, improving the convenience of hand recognition.

[0074] Step S4, fusing the hand low-order feature of each feature map with the corresponding hand Mask to obtain the hand low-order feature after fusing each hand feature point, and determining the final hand feature of the hand to be recognized according to the hand low-order feature after fusing the hand feature point.

[0075] In a preferred embodiment, the fused hand low-order feature is determined by the following formula:

[0076]

[0077] Wherein, Emerge[j] is the jth hand low-order feature after fusing the hand feature point; w[i][j] is the weight coefficient corresponding to the jth hand feature point in the ith feature map; E[i][j] is the feature value corresponding to the jth hand feature point in the ith feature map; eps is a preset value to prevent the denominator from being zero; n is the total number of feature maps; k is the total number of hand feature points; Mask[i][j] is the value of the jth hand feature point in the hand Mask corresponding to the ith feature map; d[i][j] is the depth value of the jth hand feature point in the hand Mesh corresponding to the ith feature map. merge [j] for the jth hand low-order feature after fusing the hand feature point; w i [j] is the weight coefficient corresponding to the jth hand feature point in the ith feature map; E i [j] is the feature value corresponding to the jth hand feature point in the ith feature map; eps is a preset value to prevent the denominator from being zero; n is the total number of feature maps; k is the total number of hand feature points; Mask i [j] is the value of the jth hand feature point in the hand Mask corresponding to the ith feature map; d i [j] is the depth value of the jth hand feature point in the hand Mesh corresponding to the ith feature map.

[0078] Illustratively, multi-view fusion is performed on {E1, E2... En} and {Mask1, Mask2... Maskn} to obtain the fused hand feature F1 of the hand to be recognized; F1 is a t-dimensional unit vector, and t is a preset value of the algorithm, k = 778, t = 64 in this embodiment.

[0079] The specific steps are as follows:

[0080] First, the weight {w1, w2... wn} is calculated; the weight of each hand feature point in the corresponding feature map is calculated based on the depth value of the hand feature point in the hand Mesh (the closer to the camera, the greater the weight of the vertex, and the weight of the invisible point is 0), n = 4, k = 778, eps = 10-5 to prevent the denominator from being 0:

[0081]

[0082] The jth hand low-order feature Emerge[j] after fusing the hand feature point is fused, and the formula is:

[0083]

[0084] The hand low-order features of each hand feature point are input into a pre-trained neural network GCN (Graph Convolutional Network) that can accept graph structure input, to obtain hand features F1 of the hand to be recognized.

[0085] In step S5, hand visibility of the hand to be recognized is determined according to all hand masks.

[0086] In a preferred embodiment, the hand visibility is determined by the following formula:

[0087]

[0088] wherein V1[j] is the hand visibility of the jth hand feature point in the hand to be recognized.

[0089] For example, n = 4 and k = 778, and the hand visibility of the jth hand feature point in the hand to be recognized is:

[0090] In step S6, the hand to be recognized is recognized according to the hand visibility and the final hand features of the hand to be recognized.

[0091] In a preferred embodiment, the hand to be recognized is recognized according to the hand visibility and the final hand features of the hand to be recognized, including:

[0092] A similarity score of the hand visibility of the hand to be recognized and the hand visibility in the pre-stored data is calculated to obtain a first similarity score;

[0093] A similarity score of the final hand features of the hand to be recognized and the final hand features in the pre-stored data is calculated to obtain a second similarity score;

[0094] When the first similarity score and the second similarity score are both greater than a corresponding preset threshold, it is determined that the hand to be recognized and the hand corresponding to the pre-stored data are the same hand.

[0095] In a preferred embodiment, the first similarity score is calculated by the following formula:

[0096]

[0097] wherein S v is the first similarity score, and V2[j] is the hand visibility of the jth hand feature point in the hand corresponding to the pre-stored data.

[0098] In a preferred embodiment, the second similarity score is calculated by the following formula:

[0099]

[0100] wherein, S f is the second similarity score; F1 is the final hand feature of the hand to be identified; and F2 is the final hand feature of the hand corresponding to the pre-stored data.

[0101] Specifically, in the embodiment, the threshold value corresponding to the first similarity is T v ; the threshold value corresponding to the first similarity is T f ; preferably, T v = 0.8; T f = 0.5;

[0102] When S v ≤ T v, it is not determined whether the hand to be identified and the hand corresponding to the pre-stored data are the same hand of the same person;

[0103] If S v > T v and S f > T f, it is determined that the hand to be identified and the hand corresponding to the pre-stored data are the same hand of the same person;

[0104] If S v > T v and S f ≤ T f, it is determined that the hand to be identified and the hand corresponding to the pre-stored data are not the same hand of the same person.

[0105] As shown in FIG. Figure 4 , an embodiment of the present application provides a hand recognition device based on a multi-view camera, comprising: an image acquisition module, a hand detection module, a feature extraction module, a feature fusion module, a visibility determination module, and a comparison module;

[0106] The image acquisition module is configured to acquire a plurality of hand images of a hand to be identified acquired by a multi-view camera; wherein each hand image is acquired by a camera in the multi-view camera.

[0107] The hand detection module is configured to, for each hand image, extract a hand Mesh of the hand image and a hand Mask for representing the visibility of each hand feature point in the hand Mesh.

[0108] The feature extraction module is configured to convert each hand image into a corresponding feature map; for each feature map, extract the feature value of each hand feature point in the corresponding hand Mesh in the feature map, and splice to obtain the low-order hand feature corresponding to each feature map.

[0109] The feature fusion module is configured to fuse the low-order hand feature of each feature map with the corresponding hand Mask to obtain the low-order hand feature of each hand feature point after fusion, and determine the final hand feature of the hand to be identified according to the low-order hand feature of the hand feature point after fusion.

[0110] The visibility determination module is configured to determine hand visibility of the hand to be recognized according to all hand masks.

[0111] The comparison module is configured to recognize the hand to be recognized according to the hand visibility of the hand to be recognized and the final hand feature.

[0112] It should be noted that the device embodiments described above correspond to the method embodiments of the present application, and can implement the hand recognition method based on the multi-view camera described in any one of the method embodiments of the present application. In addition, the device described above is only illustrative, and the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. According to actual needs, part or all of the modules can be selected to achieve the purpose of the embodiment. In addition, the connection between the modules in the device embodiment provided by the present application indicates that there is a communication connection between them, which can be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement it without creative labor.

[0113] On the basis of the above-mentioned method embodiments, the present application correspondingly provides a terminal device embodiment.

[0114] An embodiment of the present application provides a terminal device including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and the processor implements the hand recognition method based on the multi-view camera according to any one of the embodiments of the present application when executing the computer program.

[0115] The terminal device can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like.

[0116] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor and the like, which is the control center of the terminal device and connects all parts of the terminal device through various interfaces and lines.

[0117] The memory can be used to store the computer program, and the processor realizes various functions of the terminal device by running or executing the computer program stored in the memory and calling data stored in the memory. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory device.

[0118] The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms, etc.

[0119] On the basis of the above-mentioned method embodiment, the application correspondingly provides a storage medium embodiment.

[0120] An embodiment of the application provides a storage medium, which includes a stored computer program, wherein the computer program controls a device where the storage medium is located to perform the hand recognition method of the multi-view camera according to any one of the embodiments of the application when the computer program is running.

[0121] The storage medium is a computer readable medium, which can include any entity or device capable of carrying the computer program code, a recording medium, a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal and a software distribution medium, etc.

[0122] The above is the preferred embodiment of the present application, it should be pointed out that, for those skilled in the art, without departing from the principles of the present application, can also make a number of improvements and refinements, these improvements and refinements are also considered to be within the scope of the present application.

Claims

1. A method for hand recognition based on a multi-view camera, characterized in that, The method comprises the following steps: acquiring a plurality of hand images of a hand to be identified, wherein each hand image is captured by a camera of a multi-view camera; for each hand image, extracting a hand mesh of the hand image and a hand mask for representing the visibility of each hand feature point in the hand mesh; converting each hand image into a corresponding feature map; for each feature map, extracting the feature value of each hand feature point in the corresponding hand mesh in the feature map, and splicing to obtain the low-order feature of the hand corresponding to each feature map; fusing the low-order feature of each hand in the feature map with the corresponding hand mask to obtain the low-order feature of each hand feature point after fusion, and determining the final hand feature of the hand to be identified according to the low-order feature of each hand feature point after fusion; determining the hand visibility of the hand to be identified according to all hand masks; identifying the hand to be identified according to the hand visibility and the final hand feature of the hand to be identified; the extraction of the hand mesh of the hand image and the hand mask for representing the visibility of each hand feature point in the hand mesh comprises: detecting a hand frame in the hand image, and extracting a hand local image located in the hand frame according to the hand frame; identifying the hand mesh and the hand mask according to the hand local image; wherein the hand mask is a Boolean vector, and when the hand feature point in the hand mesh is a visible hand feature point, the value corresponding to the hand feature point in the hand mask is 1, and when the hand feature point in the hand mesh is an invisible hand feature point, the value corresponding to the hand feature point in the hand mask is 0; the conversion of each hand image into a corresponding feature map; for each feature map, extracting the feature value of each hand feature point in the corresponding hand mesh in the feature map, and splicing to obtain the low-order feature of the hand corresponding to each feature map, comprises: converting the hand image into a corresponding feature map through a preset neural network; for each feature map, taking the horizontal coordinate value and the vertical coordinate value of each hand feature point in the hand mesh as the index, extracting the feature value of each hand feature point in the corresponding hand mesh in the feature map; splicing the feature values of all hand feature points in the hand mesh in the feature map to obtain the low-order feature of the hand corresponding to each feature map; the fused low-order feature is determined by the following formula: ； ； wherein, is the low-order hand feature of the jth hand feature point after fusion; is the weight coefficient corresponding to the jth hand feature point in the ith feature map; is the feature value corresponding to the jth hand feature point in the ith feature map; eps is a preset value to prevent the denominator from being zero; n is the total number of feature maps; k is the total number of hand feature points; is the value of the jth hand feature point in the hand Mask corresponding to the ith feature map; is the depth value of the jth hand feature point in the hand Mesh corresponding to the ith feature map. 2.The multi-camera based hand recognition method of claim 1, wherein, the hand visibility is determined by the following formula: ； wherein, is the hand visibility for the jth hand feature point in the hand to be recognized. 3.The multi-camera based hand recognition method of claim 2, wherein, identifying the hand to be identified according to the hand visibility and the final hand feature of the hand to be identified comprises: calculating the similarity score of the hand visibility of the hand to be identified and the hand visibility in the pre-stored data to obtain a first similarity score; calculating the similarity score of the final hand feature of the hand to be identified and the final hand feature in the pre-stored data to obtain a second similarity score; when it is determined that the first similarity score and the second similarity score are both greater than a corresponding preset threshold, it is determined that the hand to be identified and the hand corresponding to the pre-stored data are the same hand. 4.The multi-camera based hand recognition method of claim 3, wherein, The first similarity score is calculated by the following formula: ； wherein, is the first similarity score; is the hand visibility of the jth hand feature point in the hand corresponding to the pre-stored data. 5.The multi-camera based hand recognition method of claim 4, wherein, The second similarity score is calculated by the following formula: ； wherein, is a second similarity score; is a final hand feature of the hand to be recognized; is a final hand feature of the hand corresponding to the pre-stored data.

6. A multi-camera based hand recognition apparatus, comprising: The method comprises the following steps: An image acquisition module, a hand detection module, a feature extraction module, a feature fusion module, a visibility determination module, and a comparison module; The image acquisition module is configured to acquire a plurality of hand images of a hand to be identified captured by a multi-camera; each hand image is captured by a camera in the multi-camera; The hand detection module is configured to, for each hand image, extract a hand mesh of the hand image and a hand mask representing the visibility of each hand feature point in the hand mesh; The feature extraction module is configured to convert each hand image into a corresponding feature map; for each feature map, extract the feature value of each hand feature point in the corresponding hand mesh in the feature map, and splice the feature values to obtain a hand low-order feature corresponding to each feature map; The feature fusion module is configured to fuse the hand low-order feature of each feature map with the corresponding hand mask to obtain a hand low-order feature after fusion of each hand feature point, and determine a final hand feature of the hand to be identified according to the hand low-order feature after fusion of the hand feature point; The visibility determination module is configured to determine the hand visibility of the hand to be identified according to all hand masks; The comparison module is configured to identify the hand to be identified according to the hand visibility and the final hand feature of the hand to be identified; The hand detection module is configured to, for each hand image, extract a hand mesh of the hand image and a hand mask representing the visibility of each hand feature point in the hand mesh; Detect a hand box in the hand image, and extract a hand local image located in the hand box according to the hand box; According to the hand local image, identify the hand mesh and the hand mask; wherein the hand mask is a Boolean vector, when the hand feature point in the hand mesh is a visible hand feature point, the value corresponding to the hand feature point in the hand mask is 1, and when the hand feature point in the hand mesh is an invisible hand feature point, the value corresponding to the hand feature point in the hand mask is 0; The feature extraction module is configured to convert each hand image into a corresponding feature map; for each feature map, extract the feature value of each hand feature point in the corresponding hand mesh in the feature map, and splice the feature values to obtain a hand low-order feature corresponding to each feature map; Convert the hand image into a corresponding feature map through a preset neural network; For each feature map, take the horizontal coordinate value and the vertical coordinate value of each hand feature point in the hand mesh as an index to extract the feature value of each hand feature point in the corresponding hand mesh in the feature map; Splice the feature values of all hand feature points in the hand mesh in the feature map to obtain the hand low-order feature corresponding to each feature map; The hand low-order feature after fusion is determined by the following formula: ； ； wherein, is the low-order hand feature of the jth hand feature point after fusion; is the weight coefficient corresponding to the jth hand feature point in the ith feature map; is the feature value corresponding to the jth hand feature point in the ith feature map; eps is a preset value to prevent the denominator from being zero; n is the total number of feature maps; k is the total number of hand feature points; is the value of the jth hand feature point in the hand Mask corresponding to the ith feature map; is the depth value of the jth hand feature point in the hand Mesh corresponding to the ith feature map.

7. A storage medium, characterized by The storage medium comprises a stored computer program, wherein the computer program controls a device where the storage medium is located to perform the hand recognition method for multi-view cameras as claimed in any one of claims 1 to 6 when the computer program is running.

Images