Gesture detection method and device, electronic device, and storage medium

Abstract

The present disclosure relates to the technical field of information processing, and particularly relates to a gesture detection method and device, electronic equipment and storage medium, the detection method comprising: acquiring a to-be-detected image, and inputting the to-be-detected image into a trained detection model to obtain hand information corresponding to a target hand in the to-be-detected image; determining, according to the hand detection frame, a region image corresponding to the hand detection frame in the to-be-detected image; determining position information corresponding to each hand key point in the region image in the plurality of hand key points; and determining, according to a preset position relationship between a plurality of preset hand key points in a preset gesture, and a plurality of position information corresponding to the plurality of preset hand key points in the plurality of hand key points, a gesture detection result corresponding to the target hand in the to-be-detected image. In the embodiment of the present disclosure, a plurality of information in the hand information can be obtained through the same detection, which is beneficial to improving the detection efficiency.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of information processing, and particularly relates to a gesture detection method and device, electronic equipment and storage medium. BACKGROUND

[0002] Gesture detection can be applied in various scenarios. A user can realize human-computer interaction through a specific gesture, so as to realize touchless electronic device adjustment functions. For example, a user can adjust the brightness of indoor light through a specific gesture, or trigger the shutter of a camera through a specific gesture. Human-computer interaction through gestures has the advantages of reducing hardware costs and improving operation convenience. Therefore, how to better realize gesture detection is a technical problem that developers urgently need to solve. SUMMARY

[0003] The present disclosure provides a gesture detection technical solution.

[0004] According to an aspect of the present disclosure, a gesture detection method is provided. The method comprises: acquiring a to-be-detected image, and inputting the to-be-detected image into a trained detection model to obtain hand information corresponding to a target hand in the to-be-detected image; wherein the hand information comprises a hand detection box and a plurality of hand key points; wherein the plurality of hand key points are obtained through the same detection; determining a region image corresponding to the hand detection box in the to-be-detected image according to the hand detection box; determining position information corresponding to each hand key point in the region image; and determining a gesture detection result corresponding to the target hand in the to-be-detected image according to a preset position relationship between a plurality of preset hand key points in a preset gesture and a plurality of position information corresponding to the plurality of preset hand key points in the plurality of hand key points.

[0005] In a possible implementation, the determining of the gesture detection result corresponding to the target hand in the to-be-detected image according to the preset position relationship between the plurality of preset hand key points in the preset gesture and the plurality of position information corresponding to the plurality of preset hand key points in the plurality of hand key points comprises: adjusting the position information corresponding to each hand key point according to a preset hand orientation; and determining the gesture detection result corresponding to the target hand in the to-be-detected image according to the preset position relationship between the plurality of preset hand key points in the preset gesture and the plurality of position information corresponding to the plurality of preset hand key points in the plurality of hand key points after the adjustment.

[0006] In a possible implementation, the adjusting the position information corresponding to each hand key point according to the preset hand orientation includes: determining an angle difference between a line connecting two first key points in the plurality of hand key points and a line connecting the two first key points in the preset hand orientation, and taking the angle difference as a rotation angle; determining a circumscribed rectangle corresponding to the plurality of hand key points, and taking a center of the circumscribed rectangle as a rotation center; determining a mapping matrix according to the rotation center and the rotation angle; and mapping and adjusting the position information of each hand key point according to the mapping matrix.

[0007] In a possible implementation, the adjusting the position information corresponding to each hand key point according to the preset hand orientation further includes: determining a circumscribed square corresponding to the plurality of hand key points after the mapping adjustment, and performing normalization adjustment on the position information of the plurality of hand key points after the mapping adjustment according to the circumscribed square.

[0008] In a possible implementation, the inputting the to-be-detected image into the trained detection model to obtain hand information corresponding to a target hand in the to-be-detected image includes: inputting the to-be-detected image into the trained detection model to obtain hand information corresponding to each hand in the to-be-detected image and a confidence of the corresponding hand information; and taking a hand with the highest confidence in the plurality of hands, hand information corresponding to the hand, and the hand information corresponding to the target hand as the target hand.

[0009] In a possible implementation, the to-be-detected image is a plurality of to-be-detected images, and the inputting the to-be-detected image into the trained detection model to obtain hand information corresponding to a target hand in the to-be-detected image includes: inputting the plurality of to-be-detected images into the trained detection model in sequence to obtain hand information corresponding to each hand in the to-be-detected image; determining a target hand in each to-be-detected image in the plurality of to-be-detected images; and determining hand information corresponding to the target hand in each to-be-detected image in the plurality of to-be-detected images.

[0010] In one possible implementation, the plurality of images to be detected are arranged sequentially. Determining the target hand in each of the plurality of images to be detected includes: determining multiple hands in the first image of the plurality of images to be detected; generating hand information and confidence scores for each hand among the plurality of hands; selecting the hand with the highest confidence score among the plurality of hands as the target hand and determining the target position information corresponding to the target hand; according to the sequential order of the plurality of images to be detected, taking the plurality of images after the first image of the plurality of images to be detected as the current image and performing the following steps until each image corresponds to a target hand: determining multiple hands in the current image and the position information corresponding to each hand among the plurality of hands; selecting the hand with the smallest difference between its position information and the target position information as the target hand in the current image and updating the target position information according to the position information of the hand.

[0011] In one possible implementation, the image to be detected is a plurality of images to be detected, and the preset gesture is a plurality of preset gestures. The step of determining the gesture detection result corresponding to the target hand in the image to be detected based on the preset positional relationship between the plurality of preset hand key points in the preset gesture and the plurality of positional information corresponding to the plurality of preset hand key points in the plurality of hand key points includes: for each of the plurality of images to be detected, determining the probability that the target hand in the image to be each preset gesture based on the preset positional relationship between the plurality of preset hand key points in each of the plurality of preset gestures and the plurality of positional information corresponding to the plurality of preset hand key points in the image to be detected; if the probability that the target hand in the image to be detected is a first preset gesture is greater than a preset probability, taking the first preset gesture corresponding to the highest probability as the gesture detection sub-result of the image to be detected; and taking the gesture detection sub-result with the largest number among the plurality of gesture detection sub-results corresponding to the plurality of images to be detected as the gesture detection result corresponding to the target hand in the plurality of images to be detected.

[0012] According to an aspect of the present disclosure, a training method of a detection model is provided. The training method comprises: obtaining a training sample, the training sample comprising: a plurality of training images, and a hand key point corresponding to each of the plurality of training images; inputting the training sample into a pre-trained model to obtain a first hand detection box corresponding to each of the training images; determining a second hand detection box corresponding to each of the training images according to the hand key point corresponding to each of the training images; determining a third hand detection box corresponding to each of the training images according to an intersection-over-union between the first hand detection box corresponding to each of the training images and the second hand detection box corresponding to each of the training images; and training the detection model according to the training sample and the third hand detection box corresponding to each of the training images to obtain a trained detection model.

[0013] In a possible implementation, the determining the second hand detection box corresponding to each of the training images according to the hand key point corresponding to each of the training images comprises: determining a bounding rectangle box of the hand key point corresponding to each of the training images; and expanding the bounding rectangle box corresponding to each of the training images, and taking the expanded bounding rectangle box corresponding to each of the training images as the second hand detection box corresponding to each of the training images.

[0014] In a possible implementation, the determining the third hand detection box corresponding to each of the training images according to the intersection-over-union between the first hand detection box corresponding to each of the training images and the second hand detection box corresponding to each of the training images comprises at least one of: in a case where the intersection-over-union between the first hand detection box corresponding to each of the training images and the second hand detection box corresponding to each of the training images is less than a preset threshold, taking the second hand detection box corresponding to each of the training images as the third hand detection box corresponding to each of the training images; and in a case where the intersection-over-union between the first hand detection box corresponding to each of the training images and the second hand detection box corresponding to each of the training images is greater than or equal to the preset threshold, taking a union set of the first hand detection box corresponding to each of the training images and the second hand detection box corresponding to each of the training images as the third hand detection box corresponding to each of the training images.

[0015] In a possible implementation, the detection model comprises: a backbone network, a hand key point detection head connected to the backbone network, and a hand detection head connected to the backbone network; wherein the backbone network is configured to extract a hand feature in an image; the hand key point detection head is configured to determine a plurality of hand key points according to the hand feature; and the hand detection head is configured to determine a hand detection box according to the hand feature.

[0016] In a possible implementation, the hand information further includes a hand attribute, the hand attribute being used to indicate whether the target hand in the to-be-detected image is a left hand or a right hand; the training sample further includes the hand attribute corresponding to each training image; and the detection model further includes a hand attribute detection network connected to the backbone network, the hand attribute detection network being used to determine the hand attribute according to the hand feature output by the backbone network.

[0017] In a possible implementation, the training of the detection model according to the training sample and the third hand detection frame corresponding to each training image to obtain the trained detection model includes: inputting the plurality of training images in the training sample into the detection model to obtain the hand key point prediction result corresponding to each training image and the hand detection frame prediction result corresponding to each training image; determining a first loss according to a difference between the hand key point prediction result corresponding to each training image and the hand key point; determining a second loss according to a difference between the hand detection frame prediction result corresponding to each training image and the third hand detection frame; determining an overall loss of the detection model according to the first loss, a first weight corresponding to the first loss, the second loss, and a second weight corresponding to the second loss; and adjusting the model parameter of the detection model according to the overall loss of the detection model.

[0018] In a possible implementation, the first weight is equal to the second weight.

[0019] According to an aspect of the present disclosure, a hand gesture detection apparatus is provided, which includes: a hand information determination module configured to obtain a to-be-detected image, and input the to-be-detected image into a trained detection model to obtain hand information corresponding to a target hand in the to-be-detected image; wherein the hand information includes a hand detection frame and a plurality of hand key points; the plurality of information in the hand information is obtained through the same detection; a region image determination module configured to determine a region image corresponding to the hand detection frame in the to-be-detected image according to the hand detection frame; a position information determination module configured to determine position information corresponding to each hand key point in the region image in the plurality of hand key points; and a detection result determination module configured to determine a hand gesture detection result corresponding to the target hand in the to-be-detected image according to a preset position relationship between a plurality of preset hand key points in a preset hand gesture and a plurality of position information corresponding to the plurality of preset hand key points in the plurality of hand key points.

[0020] According to an aspect of the present disclosure, a training device of a detection model is provided, the training device comprising: a training sample acquisition module configured to acquire a training sample, the training sample comprising: a plurality of training images, and a hand key point corresponding to each of the plurality of training images; a first hand detection box generation module configured to input the training sample into a pre-trained model to obtain a first hand detection box corresponding to each of the plurality of training images; a second hand detection box generation module configured to determine a second hand detection box corresponding to each of the plurality of training images according to the hand key point corresponding to each of the plurality of training images; a third hand detection box generation module configured to determine a third hand detection box corresponding to each of the plurality of training images according to an intersection over union between the first hand detection box corresponding to each of the plurality of training images and the second hand detection box corresponding to each of the plurality of training images; and a model training module configured to train the detection model according to the training sample and the third hand detection box corresponding to each of the plurality of training images to obtain a trained detection model.

[0021] According to an aspect of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to perform the gesture detection method or the training method of the detection model.

[0022] According to an aspect of the present disclosure, a computer-readable storage medium is provided, having stored thereon computer program instructions, which, when executed by a processor, implement the gesture detection method or the training method of the detection model.

[0023] In the embodiments of the present disclosure, a to-be-detected image can be acquired, and the to-be-detected image is input into a trained detection model to obtain hand information corresponding to a target hand in the to-be-detected image, wherein a plurality of information in the hand information is obtained through the same detection. Then, a region image corresponding to the hand detection box in the to-be-detected image is determined according to the hand detection box. The position information of each hand key point in the plurality of hand key points in the region image is determined. Finally, a gesture detection result corresponding to the target hand in the to-be-detected image is determined according to a preset position relationship between a plurality of preset hand key points in a preset gesture and a plurality of position information corresponding to the plurality of preset hand key points in the plurality of hand key points. The embodiments of the present disclosure can realize end-to-end hand information generation, and a plurality of information in the hand information can be obtained through the same detection, which is beneficial to improving the detection efficiency.

[0024] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, but not limiting the present disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0025] The accompanying drawings, which are incorporated herein and constitute part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, further serve to explain the principles of the present disclosure.

[0026] Figure 1 A flow chart of a gesture detection method is shown according to an embodiment of the present disclosure.

[0027] Figure 2 A reference schematic diagram of a gesture detection method is shown according to an embodiment of the present disclosure.

[0028] Figure 3 A reference schematic diagram of a hand key point of a preset gesture is shown according to an embodiment of the present disclosure.

[0029] Figure 4 A flow chart of a training method of a detection model is shown according to an embodiment of the present disclosure.

[0030] Figure 5 A block diagram of a gesture detection apparatus is shown according to an embodiment of the present disclosure.

[0031] Figure 6 A block diagram of a training apparatus of a detection model is shown according to an embodiment of the present disclosure.

[0032] Figure 7 A block diagram of an electronic device is shown according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] Various exemplary embodiments, features and aspects of the present disclosure will be explained in detail below with reference to the accompanying drawings. Like reference numerals in the drawings denote like elements or elements having a similar function. Although various aspects of embodiments are illustrated in the drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

[0034] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations.

[0035] The term "and / or", merely describes association relationship of associated objects, and means that three relationships can exist, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the term "at least one" herein means any one of multiple or any combination of at least two of multiple, for example, at least one of A, B and C includes any one or more elements selected from the set consisting of A, B and C.

[0036] In addition, for better illustrating the present disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that the present disclosure can be practiced without certain specific details. In some instances, well-known methods, structures, elements, and circuits have not been described in detail in order to avoid obscuring the subject matter.

[0037] In the related art, the technical solutions of gesture detection include: 1. Gesture detection based on wearable devices, users can collect finger key point positions through gloves or other sensor devices, and then classify the collected data to detect various gestures. Although this approach has high reliability, it is difficult to free the user's hands, the user experience is poor, and the price is expensive, making it difficult to popularize to common application scenarios. 2. Through the method of target detection + image classification, first do target detection on the input image to obtain the ROI region of the palm, and then directly perform classification task on the ROI region to detect the corresponding gesture type. This makes end-to-end gesture detection possible, but there is a lack of training samples, and there is currently no public data set for direct gesture detection and classification. In addition, due to the high degree of freedom of the hand, there are differences between people of different angles, different ages, and different genders, so even if a large number of training samples are collected and labeled, the expansibility is poor, and it is difficult to popularize to common application scenarios.

[0038] Therefore, the present disclosure provides a gesture detection method, which can obtain a to-be-detected image, input the to-be-detected image into a trained detection model, and obtain hand information corresponding to a target hand in the to-be-detected image, wherein a plurality of information in the hand information is obtained through the same detection. Then, according to the hand detection frame, a region image corresponding to the hand detection frame in the to-be-detected image is determined. Then, the position information of each hand key point in the plurality of hand key points in the region image is determined. Finally, according to the preset position relationship between a plurality of preset hand key points in a preset gesture, and the plurality of position information corresponding to the plurality of preset hand key points in the plurality of hand key points, a gesture detection result corresponding to the target hand in the to-be-detected image is determined. The present disclosure can realize end-to-end hand information generation, and a plurality of information in the hand information can be obtained through the same detection, which is beneficial to improving the detection efficiency.

[0039] Reference is made to Figure 1 , Figure 1 A flowchart of a gesture detection method according to an embodiment of the present disclosure is shown, combined with Figure 1The detection method comprises the following steps: S100, acquiring a to-be-detected image, and inputting the to-be-detected image into the trained detection model to obtain hand information corresponding to a target hand in the to-be-detected image. The hand information comprises a hand detection box and a plurality of hand key points, and the plurality of pieces of information in the hand information are obtained through the same detection. For example, the to-be-detected image can be acquired by an image acquisition device in the related art. The specific format of the to-be-detected image is not limited in the embodiment of the present disclosure, for example, the to-be-detected image can be in a format such as JPG or PNG in the related art. In an example, the to-be-detected image can also be an image in a real-time video acquired by the image acquisition device, which is not limited in the embodiment of the present disclosure. The hand detection box (or palm detection box) is used to represent a region with a hand. The definition of the hand key points can refer to the related art, for example, the hand key points can be divided into 21 joint points, for example, the hand key points comprise finger key points and key points corresponding to the connection between the finger bones. It should be understood that the hand key points can be added or deleted according to the actual needs of the developer, which is not limited in the embodiment of the present disclosure. The plurality of pieces of information in the hand information are obtained through the same detection, which means that the hand detection box, the plurality of hand key points and the hand information described below are generated in parallel, rather than in series. In the related art, the generation of the hand detection box is performed first, and then the generation of the plurality of hand key points is performed, and the overall efficiency is low. The trained detection model can generate the hand information in an end-to-end manner at one time, which is beneficial to improving the detection efficiency of the detection model and the overall efficiency of gesture detection. For example, the training sample used in the training process of the detection model is a combination of an image and a label (for example, the hand detection box and the hand key points described above), and after the detection model is trained, the to-be-detected image can be directly inputted to obtain the label. In an example, each training image in the training sample comprises at least one hand to improve the training efficiency, and the to-be-detected image can comprise a hand or not comprise a hand, in other words, the specific content of the to-be-detected image is not limited. Considering the actual application scenario, there are images with or without hands in a video. If the processing cost is considered, a plurality of to-be-detected images in a video can be directly inputted into the detection model. In an example, the continuous video can also be preprocessed, which is not limited in the embodiment of the present disclosure.

[0040] In a possible implementation, the inputting the to-be-detected image into the trained detection model in step S100 to obtain hand information corresponding to a target hand in the to-be-detected image includes: inputting the to-be-detected image into the trained detection model to obtain hand information corresponding to each hand in the to-be-detected image and confidence of the corresponding hand information. For example, the confidence can be directly output by the detection model described above, and a specific generation process can refer to related technologies, which will not be described herein again. Finally, a hand with the highest confidence among the multiple hands and hand information corresponding to the hand are taken as a target hand and hand information corresponding to the target hand. In this way, a determination manner of a target hand is provided, a hand with the highest confidence can be taken as the target hand, that is, a gesture of a hand with a higher detection accuracy is preferentially detected, a probability that an irrelevant hand is first detected can be reduced, and the detection expectation of a user is better met. In combination with an actual application scenario, an image of a hand with a higher confidence is often clearer, and when a user uses the image, a hand of the user is usually the clearest hand in the image, rather than a more blurred hand in a distance. Therefore, the target hand determined by the embodiments of the present disclosure is better in line with the detection expectation of the user.

[0041] In a possible implementation, the image to be detected is a plurality of images to be detected, and the inputting of the image to be detected into the trained detection model in step S100 to obtain the hand information corresponding to the target hand in the image to be detected includes: sequentially inputting the plurality of images to be detected into the trained detection model to obtain the hand information corresponding to each hand in the image to be detected. The target hand in each image to be detected in the plurality of images to be detected is determined. Then, the hand information corresponding to the target hand in each image to be detected in the plurality of images to be detected is determined. In an example, the target hand in each image to be detected in the plurality of images to be detected can be determined in the same manner as above, or can be flexibly set according to other needs of the developer. In another example, the plurality of images to be detected are arranged in time sequence, and the determination of the target hand in each image to be detected in the plurality of images to be detected includes: determining a plurality of hands in a first image in the plurality of images to be detected. Then, the hand information corresponding to each hand in the plurality of hands and the confidence of the corresponding hand information are generated. The hand with the highest confidence in the plurality of hands is taken as the target hand, and the target position information corresponding to the target hand is determined. Then, according to the time sequence of the plurality of images to be detected, a plurality of images after the first image in the plurality of images to be detected are sequentially taken as current images, and the following steps are performed until each image has a target hand: determining a plurality of hands in the current image, the position information corresponding to each hand in the plurality of hands, taking the hand with the smallest distance between the position information and the target position information as the target hand in the current image, and updating the target position information according to the position information of the hand. For example, the target position information can be updated to be equal to the position information of the hand, or the target position information can be updated to be the average of the position information of the hand and the target position information, which is not limited in the embodiments of the present disclosure. The embodiments of the present disclosure can track the same target hand between different images to be detected by determining the target hand between the plurality of images to be detected, so as to adapt to more application scenarios.

[0042] Continuing with Figure 1 In step S200, the region image corresponding to the hand detection frame in the image to be detected is determined according to the hand detection frame. For example, the hand detection frame corresponds to a hand region in the image to be detected. Here, the hand detection frame is taken as a rectangular detection frame, and the hand detection frame can be represented by the coordinates of the four vertices of the rectangle (which can be represented based on the image coordinate system of the image to be detected). The vertex coordinates of the hand detection frame can be taken as the vertex coordinates of the region image. In an example, the area of the region image can be slightly larger than the area of the hand detection frame, and the specific size relationship is not limited in the embodiments of the present disclosure.

[0043] In step S300, the position information of each hand key point in the region image is determined. For example, the position information can be represented as the coordinates of each hand key point in the image coordinate system of the region image. For example, the coordinates of the top-left corner of the region image can be (0, 0). In some examples, the position information of the hand key point can also be represented by a data form that can represent a position in the related art, such as a distance from a reference point in the image coordinate system of the region image, an angle, and the like, which is not limited in the embodiments of the present disclosure.

[0044] In step S400, the gesture detection result corresponding to the target hand in the to-be-detected image is determined according to the preset position relationship between the plurality of preset hand key points in the preset gesture, and the plurality of position information corresponding to the plurality of preset hand key points in the plurality of hand key points. For example, when the preset position relationship is similar or identical to the plurality of position relationships, the gesture detection result corresponding to the target hand is the preset gesture corresponding to the preset position relationship. The embodiments of the present disclosure provide a preset gesture for reference. For reference, see Figure 2 , Figure 3 , Figure 2 FIG. 7 shows a reference schematic diagram of a gesture detection method according to an embodiment of the present disclosure, in which key points 0-20 are shown. Figure 3 FIG. 8 shows a reference schematic diagram of hand key points of a preset gesture according to an embodiment of the present disclosure, in which key points 0-20 are shown. For reference, see Figure 2 , 3As shown, the developer first sets the positional relationship between the hand key points according to the specific gesture content to obtain a preset positional relationship. In combination with the "OK" gesture, the following measurements are performed: 1. Geometric measurement is performed on the little finger to obtain a straight line passing through key points 17 and 18, a straight line passing through key points 18 and 20, and an included angle θ1 between the two straight lines. 2. Geometric measurement is performed on the ring finger to obtain a straight line passing through key points 13 and 14, a straight line passing through key points 14 and 16, and an included angle θ2 between the two straight lines. 3. Geometric measurement is performed on the middle finger to obtain a straight line passing through key points 9 and 10, a straight line passing through key points 10 and 12, and an included angle θ3 between the two straight lines. 4. Geometric measurement is performed on the index finger and the thumb to obtain a distance d1 between key point 8 and key point 3, and a distance d2 between key point 8 and key point 3. According to the threshold value set by the developer, for example, the angle difference threshold value between any two included angles and the key point distance threshold value for the "OK" gesture are set to perform comparison. For the "OK" gesture, if θ1, θ2, and θ3 are all less than 15 degrees (15 degrees is used as the angle difference threshold value), and any one of d1 and d2 is less than 0.1 (0.1 is used as the key point distance threshold value), it is determined that the gesture detection result corresponding to the target hand in the target image is the "OK" gesture. The preset positional relationship and the preset hand key points can be determined according to actual requirements, which are not limited in the embodiments of the present disclosure.

[0045] In a possible implementation, the step S400 can include: adjusting the position information corresponding to each hand key point according to a preset hand orientation. The gesture detection result corresponding to the target hand in the image to be detected is determined according to the preset positional relationship between the plurality of preset hand key points in the preset gesture and the plurality of position information corresponding to the plurality of preset hand key points in the plurality of adjusted hand key points. Exemplarily, the preset hand orientation is used to represent the rotation direction of the palm center as the rotation center (or any reference point of the palm as the rotation center) around the vertical axis after the palm is upward. By setting the preset hand orientation, the hand in different postures can be rotated to a position corresponding to the preset hand orientation, so as to reduce the probability of precision reduction caused by different rotation directions of the palm in subsequent gesture detection. The embodiments of the present disclosure do not limit the specific hand orientation, for example, the developer can connect the middle finger root node and the wrist root node in the hand key points, and the hand orientation parallel to the y-axis in the image coordinate system of the region image or the image to be detected is set as the preset hand orientation (the middle finger root node can be higher or lower than the wrist root node in the image coordinate system). Other connections between hand key points can also be used as the connection, which is not limited in the embodiments of the present disclosure. The specific manner of the adjustment can also include translation, rotation, etc., which is not limited in the embodiments of the present disclosure. The developer can set according to actual requirements.

[0046] In one possible implementation, adjusting the position information corresponding to each hand key point according to a preset hand orientation may include: determining the angle difference between the line connecting two first key points among the plurality of hand key points and the line connecting two first key points in the preset hand orientation, and using this angle as the rotation angle. For example, the root node of the middle finger and the root node of the wrist among the hand key points can be used as the two first key points, and connected to obtain a first line. Then, the root node of the middle finger and the root node of the wrist in the preset hand orientation are connected to obtain a second line. Comparing the angle difference between the first and second lines yields the rotation angle. Then, the circumscribed rectangle corresponding to the plurality of hand key points is determined, and the center of the circumscribed rectangle is used as the rotation center. Then, a mapping matrix is ​​determined based on the rotation center and the rotation angle, and the position information of each hand key point is adjusted according to the mapping matrix. For example, using the image coordinate system of the region image described above as a reference, if the coordinates of the 21 hand key points are (x1, y1), (x2, y2)...(x21, y21), then the coordinates of the upper left corner and the lower right corner of the bounding rectangle of the 21 hand key points can be expressed as: (min(x1, x2...x21), min(y1, y2...y21)) and (max(x1, x2...x21), max(y1, y2...y21)), respectively. Then the x-coordinate of the rotation center... It can be represented as = (min(x1, x2…x21) + max(x1, x2…x21)) / 2, where the ordinate of the rotation center is... It can be represented as = (min(y1, y2…y21) + max(y1, y2…y21)) / 2. Considering the rotation conditions in the image coordinate system of related technologies, the rotation center needs to be translated from the origin to... Then rotate all the hand key points around the translated rotation center by θ degrees (θ is the rotation angle mentioned above), and finally translate all the rotated hand key points. The mapping matrix M can then be obtained, and the mapping matrix M can be expressed as: After applying the mapping matrix M to all hand keypoints, the coordinates of all hand keypoints after rotation correction can be obtained. Combined with... Figure 2 , Figure 2The coordinate system in the figure is the image coordinate system of the region image described above. In this example, the left palm is taken as an example, the hand can include 21 hand key points, and the 21 hand key points are adjusted in rotation based on the line between key point 9 (i.e., the middle finger root node described above) and key point 0 (i.e., the wrist root node described above). The coordinates of key point 9 (x1, y1) and the coordinates of key point 0 (x2, y2) are exemplarily labeled in the figure, and the rotation angle θ is determined based on the line between key point 9 and key point 0 being parallel to the y-axis (when parallel, it can be regarded as a positive palm). The embodiment of the present disclosure can reduce the influence of hand rotation on gesture detection by adjusting and correcting the hand key points in rotation, and is beneficial to improve the generation accuracy of the gesture detection result.

[0047] In a possible implementation, the adjusting the position information of each hand key point according to the preset hand orientation further includes: determining an inscribed square corresponding to the plurality of hand key points after the mapping adjustment, and adjusting the position information of the plurality of hand key points after the mapping adjustment according to the inscribed square. In an example, the inscribed rectangle of the plurality of hand key points after the mapping adjustment can be determined first, and then the long side of the inscribed rectangle is taken as one side of the inscribed square, and the length of the short side of the inscribed rectangle is expanded to the length of the long side of the inscribed rectangle, so that the inscribed square is obtained. In another example, the center of the inscribed rectangle can also be taken as the center of the inscribed square, and the length of the long side of the inscribed rectangle is taken as the length of the side of the inscribed square, which is not limited in the embodiment of the present disclosure. Then, the position information of the plurality of hand key points after the mapping adjustment is subtracted from the coordinates of the top left corner of the inscribed square, and then the horizontal and vertical coordinates are divided by the length of the side of the inscribed square, so that the normalization adjustment is performed. It should be understood that the specific numerical values described above can also be adjusted according to actual conditions, for example, the horizontal and vertical coordinates can be divided by half of the length of the side of the inscribed square, which is not limited in the embodiment of the present disclosure. The embodiment of the present disclosure can reduce the influence of hand translation and hand size on gesture detection by normalizing the plurality of hand key points, which is beneficial to improve the generation accuracy of the gesture detection result.

[0048] In a possible implementation, the to-be-detected image is a plurality of to-be-detected images, and the preset gesture is a plurality of preset gestures. Step S400 includes: for each to-be-detected image in the plurality of to-be-detected images, determining a probability that the target hand in the to-be-detected image is each preset gesture according to a plurality of preset position relationships between a plurality of preset hand key points of each preset gesture, and a plurality of position information corresponding to the plurality of preset hand key points in the plurality of hand key points in the to-be-detected image. In a case where the probability that the target hand in the to-be-detected image is a first preset gesture is greater than a preset probability, a first preset gesture corresponding to the maximum probability is taken as a gesture detection sub-result of the to-be-detected image. A gesture detection result corresponding to the target hand in the plurality of to-be-detected images is taken as a gesture detection result corresponding to the target hand in the plurality of to-be-detected images. For example, a probability that the target hand is a preset gesture can be obtained by using a softmax function in the related art. For example, if the target gesture detection is preset gesture 1 and the detection is not preset gesture 2, the gesture detection sub-result corresponding to the target gesture is (1, 0, 1) (i.e., the target gesture is preset gesture 1, not preset gesture 2, and preset gesture 3), and the probability obtained by using the softmax function is (0.5, 0, 0.5) (i.e., the probability that the target gesture is preset gesture 1 is 0.5, the probability that the target gesture is preset gesture 2 is 0, and the probability that the target gesture is preset gesture 3 is 0.5). The preset probability can be set to 0.5, and the gesture detection sub-result is discarded. In other words, in this case, as long as the target hand satisfies the position relationship of the hand key points of at least two preset gestures, the gesture detection sub-result is considered to be inaccurate and is discarded. It should be understood that the gesture detection sub-result can also indicate that the target hand is a plurality of gestures. In the above example, the preset probability can be set to 0.4, and the gesture detection sub-result is used to indicate that the target hand is preset gesture 1 or preset gesture 3. In one example, in a case where there are at least two gesture detection sub-results in the plurality of gesture detection sub-results corresponding to the plurality of to-be-detected images, a prompt information is generated. The prompt information is used to prompt a user to select a preset gesture corresponding to a gesture detection result, or prompt a developer to replace preset gestures corresponding to at least two gesture detection results, or prompt the developer to reset the position relationship of the preset gestures corresponding to the at least two gesture detection results. The present embodiment is not limited herein. The present embodiment can generate a gesture detection result of the target hand through a plurality of gesture detection sub-results, which is beneficial to improve the generation accuracy of the gesture detection result.

[0049] Referring to Figure 4 as shown, Figure 4 A flowchart of a training method of a detection model is shown, which is combined withFigure 4 The embodiment of the present disclosure further provides a training method of a detection model, which comprises the following steps: S10, obtaining a training sample, wherein the training sample comprises a plurality of training images and hand key points corresponding to each of the training images. In an example, the training images can include a hand, and the embodiment of the present disclosure is not limited in this regard, and a training sample set in the related art can be referred to.

[0050] S20, inputting the training sample into a preset model trained to obtain a first hand detection frame corresponding to each of the training images. In an example, the model structure of the preset model can be more complex than that of the detection model, so as to improve the generation accuracy of the first hand detection frame. The preset model can be trained by using a palm detection related training sample in the related art, and a specific training process is not limited in the embodiment of the present disclosure, and can be determined by a developer according to actual requirements.

[0051] S30, determining a second hand detection frame corresponding to each of the training images according to the hand key points corresponding to each of the training images. In an example, a circumscribed rectangle of the hand key points corresponding to each of the training images can be determined, and then the circumscribed rectangle is directly taken as the second hand detection frame corresponding to each of the training images. In another example, a circumscribed rectangle of the hand key points corresponding to each of the training images can be determined, the circumscribed rectangle corresponding to each of the training images is expanded (for example, by a preset multiple, and a specific value can be determined by a developer), and the expanded circumscribed rectangle corresponding to each of the training images is taken as the second hand detection frame corresponding to each of the training images. In combination with an actual application scenario, the position of the hand key points is usually inside the contour of the hand, so the embodiment of the present disclosure can improve the probability that a complete hand is included in the second hand detection frame by expanding the circumscribed rectangle, and then improve the gesture detection accuracy.

[0052] Step S40, determining the third hand detection box corresponding to each training image according to the intersection over union between the first hand detection box corresponding to each training image and the second hand detection box corresponding to each training image. Exemplarily, the intersection over union can represent the intersection rate between the first hand detection box and the second hand detection box. In a possible implementation, in a case where the intersection over union between the first hand detection box corresponding to each training image and the second hand detection box corresponding to each training image is less than a preset threshold, the second hand detection box corresponding to each training image is taken as the third hand detection box corresponding to each training image. In a case where the intersection over union between the first hand detection box corresponding to each training image and the second hand detection box corresponding to each training image is greater than or equal to the preset threshold, the union between the first hand detection box corresponding to each training image and the second hand detection box corresponding to each training image is taken as the third hand detection box corresponding to each training image. In an example, in a case where the intersection over union is less than a preset difference value than the preset threshold, the second hand detection box can also be expanded, and the expanded second hand detection box is taken as the third hand detection box corresponding to each training image. The specific values of the preset threshold and the preset difference value are not limited in the embodiments of the present disclosure, and can be set according to actual needs by a developer.

[0053] Step S50, training the detection model according to the training sample and the third hand detection box corresponding to each training image to obtain a trained detection model. The specific training manner of the detection model can be determined according to actual needs by a developer. In a possible implementation, the detection model includes a backbone network, a hand key point detection head connected with the backbone network, and a hand detection head connected with the backbone network. The backbone network is used to extract hand features in an image. The hand key point detection head is used to determine a plurality of hand key points according to the hand features. The hand detection head is used to determine a hand detection box according to the hand features. In the embodiments of the present disclosure, the detection heads of the detection model share the same backbone network, which can reduce the complexity of the model and improve the detection efficiency of the gesture. In addition, the hand key point detection head and the hand detection head in the detection model provided by the embodiments of the present disclosure are arranged in parallel, so that the detection efficiencies of the two are not affected by each other. In combination with an example, after a to-be-detected image obtains a feature vector through the backbone network, the to-be-detected image is input into a plurality of detection heads in parallel, so that a plurality of information in hand information can be obtained through the same detection instead of a plurality of detections in series.

[0054] In a possible implementation, step S50 can include: inputting a plurality of training images in the training samples into the detection model to obtain a hand key point prediction result corresponding to each training image and a hand detection frame prediction result corresponding to each training image. Then, a first loss is determined according to a difference between the hand key point prediction result corresponding to each training image and the hand key point. For example, the first loss can be determined by a loss function, and the size of the first loss is positively correlated with the difference. In an example, the loss function corresponding to the first loss can be a wing_loss function, a Mean Squared Error function, an L1 loss function, an L2 loss function, and the like in the related art. Then, a second loss is determined according to a difference between the hand detection frame prediction result corresponding to each training image and the third hand detection frame. For example, the second loss can be determined by a loss function, and the second loss is positively correlated with the difference. In an example, the loss function corresponding to the second loss can be an L1 loss function, an L2 loss function, and the like in the related art. Then, an overall loss of the detection model is determined according to the first loss, a first weight corresponding to the first loss, the second loss, and a second weight corresponding to the second loss. For example, the overall loss can be a sum of the first loss and the second loss after being multiplied by the respective weights. In another example, other losses can also be introduced. In other words, the overall loss can be composed of at least the first loss and the second loss. Finally, the model parameters of the detection model are adjusted according to the overall loss of the detection model. For example, the detection model takes reducing the overall loss as much as possible as a training target, and the adjustment of the specific model parameters can be automatically performed by the detection model, which can refer to a back propagation mechanism in the related art. In an example, the first weight is equal to the second weight. The disclosure can set the first weight and the second weight to be equal, so that the detection model does not tend to be the same as the detection result of a certain detection head in the back propagation process, which is beneficial to improving the overall detection accuracy of the detection head.

[0055] In a possible implementation, the hand information further includes a hand attribute, and the hand attribute is used to indicate whether the target hand in the to-be-detected image is a left hand or a right hand. The training sample further includes the hand attribute corresponding to each training image. For example, in the training process of the detection model, the training sample with the hand attribute in the related art can be directly used, so that the trained detection model can output the hand information including the hand detection frame, the plurality of hand key points, and the hand attribute at one time. For example, the hand detection frame, the hand key point, and the hand attribute can share the same backbone network, so as to avoid repeated calculation. The hand key point is a 2D point, which can include a horizontal coordinate and a vertical coordinate. When the number of hand key points is 21, the output dimension can be 42. The above backbone network can be selected according to the inference efficiency of different models on an end-side device, for example, yolov5s (a target detection network) can be selected. In an example, to improve the generalization performance of the detection model, the training data can be subjected to data enhancement. The training sample can be subjected to random noise addition, random scaling, random rotation, and the like according to the influence of illumination, distance, noise, rotation angle, and the like in the training sample, so as to improve the generalization ability of the trained detection model. For example, the detection model further includes a hand attribute detection network connected with the backbone network. The hand attribute detection network is used to determine the hand attribute according to the hand feature output by the backbone network.

[0056] The training method provided by the embodiments of the present disclosure can make the input of the detection model include the training image, the hand key point corresponding to the image, and the third hand detection frame (and the hand attribute in an example) corresponding to the image, so as to train the backbone network and the plurality of detection heads. In this way, the trained detection model can output the hand key point, the hand detection frame (and the hand attribute in an example) corresponding to the image at one time when the to-be-detected image is input. Since the detection heads are parallel, the detection efficiency of the detection heads does not affect each other, and the detection efficiency of the gesture can be improved.

[0057] The embodiment of the present disclosure provides an actual application scenario for reference. A hand key point detection and hand attribute detection regression branch can be added to the network structure of the detection model based on the related art yolov5 network, so that the detection model can obtain the hand detection frame, the hand key point and the hand attribute through one-time input. The three branches share a backbone network yolov5s. In an example, the training sample used by the detection model can be handpose_datasets_v2 (a data set) in the related art. The data set includes picture data labeled with hand key points and hand attributes. The data set includes 380,000 images, so the automatic labeling strategy is adopted in the embodiment of the present disclosure. The automatic labeling strategy is as follows: an open source palm detection data set is obtained, a preset model of a hand detection task with high precision is trained by using a large model yolov5L of yolov5, and is used to predict the hand detection frame in the data set. The prediction result is recorded as bbx_det. The open source palm detection data set includes: to-be-trained images and palm detection frames corresponding to the to-be-trained images. The developer can set it according to the actual situation, for example: different data sets usually correspond to different collection environments. A data set similar to the collection environment of the training sample of the detection model can be selected to train the preset model to improve the detection precision. The hand key points labeled in handpose_datasets_v2 are used to calculate the circumscribed rectangle corresponding to the hand key points. Considering that the circumscribed rectangle may not completely cover the hand, the width and height of the circumscribed rectangle are expanded to 1.1 times of the original, recorded as bbx_rect_expand. The intersection over union of bbx_det and bbx_rect_expand corresponding to each image is calculated. If the intersection over union is less than a preset threshold, bbx_rect_expand is taken as the true value of the palm detection frame of the image. If the intersection over union is greater than or equal to the preset threshold, the union of bbx_det and bbx_rect_expand is taken as the true value of the palm detection frame of the image. In an example, the preset threshold can be set to 0.7. To improve the generalization performance of the detection model, data enhancement can also be performed on the training data. The training sample can be randomly added noise, randomly scaled, randomly rotated and the like considering the influence of light, distance, noise and rotation angle in the training sample. The training of the detection model can adopt the multi-task training manner in the related art. The regression branch of the hand detection frame and the hand attribute can use the general loss function in yolov5 in the related art. The hand key point regression branch can use wing_loss (a loss function) in the related art as the loss function. The regression loss of the hand key point is multiplied by a weight factor to balance the hand detection loss and the key point regression loss. The sum of the two losses is used as the final loss. The weight factor can be set to 0.5.The weights of the network parameters of the detection model can be initialized with the pre-training weights of yolov5s in the related art. The training samples can also be normalized. The pixel values of the training images are first divided by 255, and then the mean (0.485, 0.456, 0.406) and variance (0.229, 0.224, 0.225) of a plurality of training images are calculated. The mean and variance can be directly obtained from the training samples according to the preset values, that is, the handpose_datasets_v2 corresponds to the specific values of the mean and variance. For example, the pixel value of the image before normalization is X1, and the pixel value of the image after normalization is X2, then X2 = (X1 / 255-mean) / variance. It should be understood that other normalization formulas can also be used, which are not limited in the present disclosure. The training set, test set and validation set are divided according to the ratio of 8:1:1, the SGD (Stochastic Gradient Descent) optimizer is used, and the acceleration parameter (which can refer to the momentum in the related art) is set to 0.93. The learning rate can be preheated before actual training. The initial learning rate is set to 0.0001, the preheating number of rounds is 3, the overall training number of rounds is set to 200, the batch size is set to 128, and the early stopping strategy is used to reduce the overfitting probability. In the gesture detection stage, the present disclosure also performs palm rotation correction to reduce the influence of the overall rotation of the hand on the gesture type. First, the coordinates of the 21 hand key points are corrected to make the hand as a whole into a "zero gesture" (relative to the standard front image of the hand). Since the fingers have high degrees of freedom and the palm has low degrees of freedom, the coordinates of the middle finger root node and the wrist root node are selected to calculate the rotation angle of the palm, and then the center point of the hand key point's bounding rectangle is used as the rotation center to rotate all the hand key points to obtain the "zero gesture" hand key point coordinates. The clockwise rotation angle is positive, and the counterclockwise rotation angle is negative. After rotation correction, the hand key point coordinates are normalized. The bounding square of the 21 hand key points is calculated. The coordinates of the 21 hand key points are first subtracted from the coordinates of the top left corner of the bounding square, and then the coordinates of the 21 hand key points are divided by the side length of the bounding square to eliminate the influence of translation and palm size on the gesture detection result. The side length s can be expressed as: , the coordinates of the top left corner of the bounding square can be expressed as .

[0058] The embodiments of the present disclosure can also post-process the detection result of the gesture, for example: after starting gesture detection, a target hand is selected from the first detected hand detection frame (for example, the hand detection frame with the highest confidence can be selected as the target hand), when processing the subsequent frame, if multiple hands are detected, the hand closest to the target hand of the last frame is selected as the target hand, and the center point coordinates of the target hand are updated, so as to realize palm tracking. Illustratively, the gesture detection result described above can be expressed as a queue, the queue length can be set to 16, and the gesture detection sub-result of each frame target hand is sequentially stored in the queue. The gesture detection sub-result is first processed by a softmax function to obtain the probability of the target hand being each preset gesture, and if the probability is greater than 0.5, the preset gesture is stored in the queue. Finally, the number of each preset gesture in the queue is counted, and the preset gesture with the largest number is taken as the final detection result of the target hand.

[0059] The embodiments of the present disclosure can adopt a lightweight model architecture to further automatically process the disclosed data set to adapt to the generation task of processing hand detection frames, hand attributes and hand key points. In addition, the embodiments of the present disclosure set the position relationship between the preset hand key points of the preset gestures to realize the detection of the target gesture, so that the gesture detection data does not need to be specially collected, and the gesture detection can be efficiently and accurately realized. If a new preset gesture needs to be added in the future, the detection model does not need to be retrained, and the position relationship between the hand key points of the new preset gesture can be imported, and the overall flexibility is relatively high. In addition, the embodiments of the present disclosure also process the position information of the hand key points through rotation correction, normalization and the like, which can improve the accuracy of the detection result.

[0060] The related technologies, open source data sets and the like mentioned in the embodiments of the present disclosure have been authorized by the relevant personnel.

[0061] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form combined embodiments without violating the principle logic. Limited by the length, the present disclosure will not be repeated. Those skilled in the art can understand that in the above-mentioned method of the specific embodiment, the specific execution order of each step should be determined according to its function and possible internal logic.

[0062] In addition, the present disclosure also provides an electronic device, a computer readable storage medium and a program, which can be used to realize any one of the switching methods of the operating system provided by the present disclosure. The corresponding technical solutions and descriptions are described in the method part and will not be repeated.

[0063] Referring to Figure 5 , as shown in Figure 5 , a block diagram of a gesture detection device according to an embodiment of the present disclosure is shown, which is combined with Figure 5The detection device 100 comprises: a hand information determination module 110, configured to acquire a to-be-detected image, input the to-be-detected image into a trained detection model, and obtain hand information corresponding to a target hand in the to-be-detected image; wherein the hand information comprises a hand detection frame and a plurality of hand key points; wherein the plurality of hand key points are obtained through the same detection; a region image determination module 120, configured to determine a region image corresponding to the hand detection frame in the to-be-detected image according to the hand detection frame; a position information determination module 130, configured to determine position information corresponding to each hand key point in the region image; and a detection result determination module 140, configured to determine a gesture detection result corresponding to the target hand in the to-be-detected image according to a preset position relationship between a plurality of preset hand key points in a preset gesture and a plurality of position information corresponding to the plurality of preset hand key points in the plurality of hand key points.

[0064] In a possible implementation, the determination of the gesture detection result corresponding to the target hand in the to-be-detected image according to the preset position relationship between the plurality of preset hand key points in the preset gesture and the plurality of position information corresponding to the plurality of preset hand key points in the plurality of hand key points comprises: adjusting the position information corresponding to each hand key point according to a preset hand orientation; and determining the gesture detection result corresponding to the target hand in the to-be-detected image according to the preset position relationship between the plurality of preset hand key points in the preset gesture and the plurality of position information corresponding to the plurality of preset hand key points in the plurality of hand key points after the adjustment.

[0065] In a possible implementation, the adjustment of the position information corresponding to each hand key point according to the preset hand orientation comprises: determining an angle difference between a line connecting two first key points in the plurality of hand key points and a line connecting the two first key points in the preset hand orientation, and taking the angle difference as a rotation angle; determining an external rectangle corresponding to the plurality of hand key points, and taking a center of the external rectangle as a rotation center; determining a mapping matrix according to the rotation center and the rotation angle; and mapping and adjusting the position information of each hand key point according to the mapping matrix.

[0066] In a possible implementation, the adjustment of the position information corresponding to each hand key point according to the preset hand orientation further comprises: determining an external square corresponding to the plurality of hand key points after the mapping adjustment, and performing normalization adjustment on the position information of the plurality of hand key points after the mapping adjustment according to the external square.

[0067] In a possible implementation, the inputting the to-be-detected image into the trained detection model to obtain hand information corresponding to a target hand in the to-be-detected image comprises: inputting the to-be-detected image into the trained detection model to obtain hand information corresponding to each hand in the to-be-detected image and a confidence of the corresponding hand information; taking a hand with the highest confidence in the multiple hands and hand information corresponding to the hand as a target hand and the hand information corresponding to the target hand.

[0068] In a possible implementation, the to-be-detected image is multiple to-be-detected images, and the inputting the to-be-detected image into the trained detection model to obtain hand information corresponding to a target hand in the to-be-detected image comprises: inputting the multiple to-be-detected images into the trained detection model in sequence to obtain hand information corresponding to each hand in the to-be-detected image; determining a target hand in each to-be-detected image in the multiple to-be-detected images; and determining hand information corresponding to the target hand in each to-be-detected image in the multiple to-be-detected images.

[0069] In a possible implementation, the multiple to-be-detected images are arranged in time sequence, and the determining a target hand in each to-be-detected image in the multiple to-be-detected images comprises: determining multiple hands in a first image in the multiple to-be-detected images; generating hand information corresponding to each hand in the multiple hands and a confidence of the corresponding hand information; taking a hand with the highest confidence in the multiple hands as a target hand and determining target position information corresponding to the target hand; and according to the time sequence of the multiple to-be-detected images, taking multiple images after the first image in the multiple to-be-detected images as current images in sequence, performing the following steps until each image corresponds to a target hand: determining multiple hands in the current image and position information corresponding to each hand in the multiple hands; taking a hand with minimum distance between position information and the target position information as a target hand in the current image, and updating the target position information according to position information of the hand.

[0070] In a possible implementation, the to-be-detected image is a plurality of to-be-detected images, and the preset gesture is a plurality of preset gestures. The determining, according to a preset positional relationship between a plurality of preset hand key points in each preset gesture and a plurality of position information corresponding to the plurality of preset hand key points in the plurality of hand key points in the to-be-detected image, of a gesture detection result corresponding to the target hand in the to-be-detected image, includes: for each to-be-detected image in the plurality of to-be-detected images, determining a probability that the target hand in the to-be-detected image is each preset gesture according to a preset positional relationship between a plurality of preset hand key points in each preset gesture and a plurality of position information corresponding to the plurality of preset hand key points in the plurality of hand key points in the to-be-detected image; in a case where the probability that the target hand in the to-be-detected image is a first preset gesture is greater than a preset probability, taking a first preset gesture corresponding to the maximum probability as a gesture detection sub-result of the to-be-detected image; and taking a gesture detection sub-result with a maximum quantity from a plurality of gesture detection sub-results corresponding to the plurality of to-be-detected images as the gesture detection result corresponding to the target hand in the plurality of to-be-detected images.

[0071] Referring to Figure 6 as shown, Figure 6 A block diagram of a training device of a detection model is shown, according to an embodiment of the present disclosure. In combination with Figure 6 , the training device 200 includes: a training sample acquisition module 210 configured to acquire a training sample, the training sample including: a plurality of training images, and hand key points corresponding to each training image in the plurality of training images; a first hand detection box generation module 220 configured to input the training sample into a preset model trained to obtain a first hand detection box corresponding to each training image; a second hand detection box generation module 230 configured to determine a second hand detection box corresponding to each training image according to the hand key points corresponding to each training image; a third hand detection box generation module 240 configured to determine a third hand detection box corresponding to each training image according to an intersection-over-union between the first hand detection box corresponding to each training image and the second hand detection box corresponding to each training image; and a model training module 250 configured to train the detection model according to the training sample and the third hand detection box corresponding to each training image to obtain a trained detection model.

[0072] In a possible implementation, the determining, according to the hand key points corresponding to each training image, of the second hand detection box corresponding to each training image includes: determining a circumscribed rectangular box of the hand key points corresponding to each training image; and expanding the circumscribed rectangular box corresponding to each training image, and taking the expanded circumscribed rectangular box corresponding to each training image as the second hand detection box corresponding to each training image.

[0073] In a possible implementation, the determining the third hand detection box corresponding to each training image according to the intersection-over-union between the first hand detection box corresponding to each training image and the second hand detection box corresponding to each training image comprises at least one of the following: in a case where the intersection-over-union between the first hand detection box corresponding to each training image and the second hand detection box corresponding to each training image is less than a preset threshold, taking the second hand detection box corresponding to each training image as the third hand detection box corresponding to each training image; in a case where the intersection-over-union between the first hand detection box corresponding to each training image and the second hand detection box corresponding to each training image is greater than or equal to the preset threshold, taking the union of the first hand detection box corresponding to each training image and the second hand detection box corresponding to each training image as the third hand detection box corresponding to each training image.

[0074] In a possible implementation, the detection model comprises: a backbone network, a hand key point detection head connected to the backbone network, and a hand detection head connected to the backbone network; the backbone network is configured to extract hand features in an image; the hand key point detection head is configured to determine a plurality of hand key points according to the hand features; and the hand detection head is configured to determine a hand detection box according to the hand features.

[0075] In a possible implementation, the hand information further comprises a hand attribute, the hand attribute being used to indicate whether a target hand in a to-be-detected image is a left hand or a right hand; the training sample further comprises a hand attribute corresponding to each training image; and the detection model further comprises a hand attribute detection network connected to the backbone network, the hand attribute detection network being configured to determine a hand attribute according to hand features output by the backbone network.

[0076] In a possible implementation, the training the detection model according to the training sample and the third hand detection box corresponding to each training image to obtain a trained detection model comprises: inputting a plurality of training images in the training sample into the detection model to obtain a hand key point prediction result corresponding to each training image and a hand detection box prediction result corresponding to each training image; determining a first loss according to a difference between the hand key point prediction result corresponding to each training image and a hand key point; determining a second loss according to a difference between the hand detection box prediction result corresponding to each training image and the third hand detection box; determining an overall loss of the detection model according to the first loss, a first weight corresponding to the first loss, the second loss, and a second weight corresponding to the second loss; and adjusting a model parameter of the detection model according to the overall loss of the detection model.

[0077] In a possible implementation, the first weight is equal to the second weight.

[0078] The method has specific technical correlation with the internal structure of the computer system, and can solve the technical problem of how to improve the hardware operation efficiency or execution effect (including reducing data storage, reducing data transmission, improving hardware processing speed, etc.), so as to obtain the technical effect of improving the internal performance of the computer system in accordance with the natural law.

[0079] In some embodiments, the apparatus provided by the embodiments of the present disclosure has functions or includes modules that can be used to execute the methods described in the above method embodiments, and the specific implementation can refer to the description of the above method embodiments. For brevity, it will not be repeated here.

[0080] The embodiments of the present disclosure also provide a computer-readable storage medium having computer program instructions stored therein, and the computer program instructions are executed by a processor to implement the above method. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0081] The embodiments of the present disclosure also provide an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.

[0082] The embodiments of the present disclosure also provide a computer program product, including computer readable code or a non-volatile computer readable storage medium carrying computer readable code, when the computer readable code is run in the processor of the electronic device, the processor in the electronic device executes the above method.

[0083] The electronic device can be provided as a terminal device, a server or other forms of devices.

[0084] Referring to Figure 7 shown, Figure 7 A block diagram of an electronic device 1900 is shown, which is provided according to an embodiment of the present disclosure. For example, the electronic device 1900 can be provided as a server or a terminal device. Referring to Figure 7 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932, for storing instructions executable by the processing component 1922, such as an application program. The application program stored in the memory 1932 can include one or more than one module each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above method.

[0085] The electronic device 1900 can further include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output interface 1958. The electronic device 1900 can operate based on an operating system stored in the memory 1932, such as Microsoft Windows Server TM ), Apple's graphical user interface-based operating system (Mac OS X TM ), a multi-user multi-processing computer operating system (Unix TM ), a free and open-source Unix-like operating system (Linux TM ), an open-source Unix-like operating system (FreeBSD TM ), or the like.

[0086] In an exemplary embodiment, there is also provided a non-transitory computer readable storage medium, such as the memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above-described method.

[0087] The present disclosure can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present disclosure.

[0088] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or punched tape, a

[0089] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0090] Computer readable program instructions for carrying out operations of the present disclosure can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, for example, through the Internet using an Internet Service Provider. In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0091] The computer readable program instructions can also be loaded onto a computing / processing device, other programmable data processing apparatus, or other device to cause a series of operations to be performed on the computing / processing device, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computing / processing device, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0092] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer readable storage medium having no data signals on it. The instructions can be executed by one or more processors of a computer or other programmable data processing apparatus to produce a computer implemented process such that the instructions, which execute via the one or more processors of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions that execute on the computer or other programmable apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0093] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer readable storage medium having no data signals on it. The instructions can be executed by one or more processors of a computer or other programmable data processing apparatus to produce a computer implemented process such that the instructions, which execute via the one or more processors of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions that execute on the computer or other programmable apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0094] The flow and block diagrams in the drawings show architectural, functional, and operational aspects of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow and block diagrams can represent a module, a segment, or a portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may

[0095] The computer program product can be embodied by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) or the like.

[0096] The above description of various embodiments is directed to highlight the differences between various embodiments, and the same or similar parts can be referred to each other, and for brevity, will not be repeated here.

[0097] Those skilled in the art can understand that, in the above method of the specific embodiment, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined according to its function and possible internal logic.

[0098] If the technical solution of the present application involves personal information, the product applying the technical solution of the present application has been explicitly informed of the personal information processing rules before processing the personal information, and has obtained the personal independent consent. If the technical solution of the present application involves sensitive personal information, the product applying the technical solution of the present application has obtained the personal independent consent before processing the sensitive personal information, and at the same time meets the requirement of "explicit consent". For example, at the personal information collection device such as camera, a clear and prominent mark is set to inform that the personal information collection range has been entered and the personal information will be collected. If the person voluntarily enters the collection range, it is regarded as agreeing to collect the personal information. Or, on the device for processing personal information, the personal information processing rules are informed by using obvious marks / information, and the personal authorization is obtained by means of pop-up information or asking the person to upload his / her personal information. The personal information processing rules can include personal information processor, personal information processing purpose, processing method, and personal information type, etc.

[0099] The above has described various embodiments of the present disclosure, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes are obvious to those skilled in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical application, or improvement of technology in the market, or to enable other ordinary skilled persons in the art to understand the embodiments disclosed herein.

Claims

1. A method for training a detection model, characterized in that, The training method comprises: obtaining a training sample, the training sample comprising: a plurality of training images, and a hand key point corresponding to each of the plurality of training images; inputting the training sample into a pre-trained model to obtain a first hand detection frame corresponding to each of the training images; determining a second hand detection frame corresponding to each of the training images according to the hand key point corresponding to each of the training images; determining a third hand detection frame corresponding to each of the training images according to an intersection-over-union ratio between the first hand detection frame and the second hand detection frame corresponding to each of the training images; training the detection model according to the training sample and the third hand detection frame corresponding to each of the training images to obtain a trained detection model; The determining of the third hand detection frame corresponding to each of the training images according to the intersection-over-union ratio between the first hand detection frame and the second hand detection frame corresponding to each of the training images comprises at least one of the following: in a case where the intersection-over-union ratio between the first hand detection frame and the second hand detection frame corresponding to each of the training images is less than a preset threshold, taking the second hand detection frame corresponding to each of the training images as the third hand detection frame corresponding to each of the training images; in a case where the intersection-over-union ratio between the first hand detection frame and the second hand detection frame corresponding to each of the training images is greater than or equal to the preset threshold, taking a union set of the first hand detection frame and the second hand detection frame corresponding to each of the training images as the third hand detection frame corresponding to each of the training images.

2. The training method of claim 1, wherein, The determining of the second hand detection frame corresponding to each of the training images according to the hand key point corresponding to each of the training images comprises: determining a circumscribed rectangular frame of the hand key point corresponding to each of the training images; enlarging the circumscribed rectangular frame corresponding to each of the training images, and taking the enlarged circumscribed rectangular frame corresponding to each of the training images as the second hand detection frame corresponding to each of the training images.

3. The training method of claim 1, wherein The detection model comprises: a backbone network, a hand key point detection head connected to the backbone network, and a hand detection head connected to the backbone network; wherein the backbone network is used to extract hand features in an image; the hand key point detection head is used to determine a plurality of hand key points according to the hand features; and the hand detection head is used to determine a hand detection frame according to the hand features.

4. The training method of claim 1, wherein, The trained detection model is used to detect hand information corresponding to a target hand in a to-be-detected image, the hand information further comprising a hand attribute used to represent whether the target hand in the to-be-detected image is a left hand or a right hand; the training sample further comprises a hand attribute corresponding to each of the training images; the detection model further comprises a hand attribute detection network connected to the backbone network; and the hand attribute detection network is used to determine a hand attribute according to hand features output by the backbone network.

5. The training method of claim 1, wherein The training of the detection model according to the training sample and the third hand detection frame corresponding to each of the training images to obtain a trained detection model comprises: inputting a plurality of training images in the training samples into the detection model to obtain a hand key point prediction result corresponding to each training image and a hand detection frame prediction result corresponding to each training image; determining a first loss according to a difference between the hand key point prediction result corresponding to each training image and a hand key point; determining a second loss according to a difference between the hand detection frame prediction result corresponding to each training image and a third hand detection frame; determining an overall loss of the detection model according to the first loss, a first weight corresponding to the first loss, the second loss and a second weight corresponding to the second loss; adjusting a model parameter of the detection model according to the overall loss of the detection model.

6. The training method of claim 5, wherein, The first weight is equal to the second weight.

7. A method of detecting a gesture, characterized by The detection method comprises: obtaining a to-be-detected image and inputting the to-be-detected image into the trained detection model to obtain hand information corresponding to a target hand in the to-be-detected image; wherein the hand information comprises a hand detection frame and a plurality of hand key points; wherein the plurality of information in the hand information is obtained through the same detection; the trained detection model is trained by using the training method in any one of claims 1-6; determining a region image corresponding to the hand detection frame in the to-be-detected image according to the hand detection frame; determining position information corresponding to each hand key point in the region image in the plurality of hand key points; determining a gesture detection result corresponding to the target hand in the to-be-detected image according to a preset position relationship between a plurality of preset hand key points in a preset gesture and a plurality of position information corresponding to the plurality of preset hand key points in the plurality of hand key points.

8. The detection method of claim 7, wherein, The determination of the gesture detection result corresponding to the target hand in the to-be-detected image according to the preset position relationship between the plurality of preset hand key points in the preset gesture and the plurality of position information corresponding to the plurality of preset hand key points in the plurality of hand key points comprises: adjusting the position information corresponding to each hand key point according to a preset hand orientation; determining the gesture detection result corresponding to the target hand in the to-be-detected image according to the preset position relationship between the plurality of preset hand key points in the preset gesture and the plurality of position information corresponding to the plurality of preset hand key points in the plurality of hand key points after adjustment.

9. The detection method of claim 8, wherein, The adjustment of the position information corresponding to each hand key point according to the preset hand orientation comprises: determining an angle difference between a line connecting two first key points in the plurality of hand key points and a line connecting the two first key points in the preset hand orientation as a rotation angle; determining an external rectangle corresponding to the plurality of hand key points and taking a center of the external rectangle as a rotation center; determining a mapping matrix according to the rotation center and the rotation angle; mapping and adjusting the position information of each hand key point according to the mapping matrix.

10. The method of claim 9, wherein the detecting is performed by a method selected from the group consisting of mass spectrometry, nuclear magnetic resonance, and chromatography. The step of adjusting the position information corresponding to each hand key point according to the preset hand orientation further includes: determining the circumscribed square corresponding to the multiple hand key points after mapping adjustment, and normalizing the position information of the multiple hand key points after mapping adjustment according to the circumscribed square.

11. The method of claim 7, wherein the detecting is performed by a method selected from the group consisting of mass spectrometry, nuclear magnetic resonance, and chromatography. The step of inputting the image to be detected into the trained detection model to obtain the hand information corresponding to the target hand in the image to be detected includes: The image to be detected is input into the trained detection model to obtain the hand information corresponding to each hand in the image to be detected and the confidence level of the corresponding hand information. The hand with the highest confidence among the plurality of hands, and the hand information corresponding to that hand, are taken as the target hand and the hand information corresponding to the target hand.

12. The method of claim 7, wherein the detecting is performed by a method selected from the group consisting of mass spectrometry, nuclear magnetic resonance, and chromatography. The images to be detected are multiple images. The step of inputting the images to be detected into the trained detection model to obtain hand information corresponding to the target hand in the images to be detected includes: The multiple images to be detected are sequentially input into the trained detection model to obtain hand information corresponding to each hand in the images to be detected; Determine the target hand in each of the plurality of images to be detected; Determine the hand information corresponding to the target hand in each of the plurality of images to be detected.

13. The detection method of claim 12, wherein, The plurality of images to be detected are arranged sequentially in time. Determining the target hand in each of the plurality of images to be detected includes: Identify multiple hands in the first image among the plurality of images to be detected; Generate hand information and confidence level for each of the plurality of hands; The hand with the highest confidence among the plurality of hands is selected as the target hand, and the target position information corresponding to the target hand is determined. Based on the temporal order of the multiple images to be detected, the following steps are performed sequentially, taking the images following the first image as the current image, until each image corresponds to a target hand: determining multiple hands in the current image and the position information corresponding to each hand; taking the hand with the smallest difference between its position information and the target position information as the target hand in the current image, and updating the target position information according to the position information of that hand.

14. The detection method as described in claim 7, characterized in that, The image to be detected is a plurality of images to be detected, and the preset gesture is a plurality of preset gestures. The step of determining the gesture detection result corresponding to the target hand in the image to be detected based on the preset positional relationship between the plurality of preset hand key points in the preset gesture and the multiple positional information corresponding to the plurality of preset hand key points includes: For each of the plurality of to-be-detected images, according to a preset position relationship between a plurality of preset hand key points in each of a plurality of preset gestures, and a plurality of position information corresponding to the plurality of preset hand key points in the plurality of hand key points in the to-be-detected image, a probability that the target hand in the to-be-detected image is the each preset gesture is determined; in a case where the probability that the target hand in the to-be-detected image is a first preset gesture is greater than a preset probability, a first preset gesture corresponding to the maximum probability is taken as a gesture detection sub-result of the to-be-detected image; The plurality of to-be-detected images corresponding to the plurality of gesture detection sub-results are obtained, and the gesture detection sub-result with the largest quantity is taken as a gesture detection result corresponding to the target hand in the plurality of to-be-detected images.

15. A training device for a detection model, characterized in that, The training device comprises: A training sample acquisition module is configured to acquire a training sample, wherein the training sample comprises a plurality of training images and hand key points corresponding to each of the plurality of training images; A first hand detection box generation module is configured to input the training sample into a preset model to obtain a first hand detection box corresponding to each of the training images; A second hand detection box generation module is configured to determine a second hand detection box corresponding to each of the training images according to the hand key points corresponding to each of the training images; A third hand detection box generation module is configured to determine a third hand detection box corresponding to each of the training images according to an intersection-over-union ratio between the first hand detection box corresponding to each of the training images and the second hand detection box corresponding to each of the training images; A model training module is configured to train the detection model according to the training sample and the third hand detection box corresponding to each of the training images to obtain a trained detection model; The third hand detection box corresponding to each of the training images is determined according to the intersection-over-union ratio between the first hand detection box corresponding to each of the training images and the second hand detection box corresponding to each of the training images, and the method comprises at least one of the following: In a case where the intersection-over-union ratio between the first hand detection box corresponding to each of the training images and the second hand detection box corresponding to each of the training images is less than a preset threshold, the second hand detection box corresponding to each of the training images is taken as the third hand detection box corresponding to each of the training images; In a case where the intersection-over-union ratio between the first hand detection box corresponding to each of the training images and the second hand detection box corresponding to each of the training images is greater than or equal to the preset threshold, a union set between the first hand detection box corresponding to each of the training images and the second hand detection box corresponding to each of the training images is taken as the third hand detection box corresponding to each of the training images.

16. A gesture detection apparatus, characterized by The detection device comprises: A hand information determination module is configured to acquire a to-be-detected image and input the to-be-detected image into a trained detection model to obtain hand information corresponding to a target hand in the to-be-detected image; wherein the hand information comprises a hand detection box and a plurality of hand key points; wherein the plurality of information in the hand information is obtained through the same detection; and the trained detection model is trained by using the training method in any one of claims 1-6. A region image determination module is configured to determine a region image corresponding to the hand detection frame in the to-be-detected image according to the hand detection frame. A position information determination module is configured to determine position information corresponding to each hand key point in the region image in the plurality of hand key points. A detection result determination module is configured to determine a hand gesture detection result corresponding to the target hand in the to-be-detected image according to a preset position relationship between a plurality of preset hand key points in a preset hand gesture and a plurality of position information corresponding to the plurality of preset hand key points in the plurality of hand key points.

17. An electronic device, comprising: The method comprises the following steps: A processor; A memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the gesture detection method in any one of claims 7 to 14 or the training method of the detection model in any one of claims 1 to 6.

18. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions are executed by the processor to implement the gesture detection method in any one of claims 7 to 14 or the training method of the detection model in any one of claims 1 to 6.

Images