A method, device and medium for generating a gesture data set

By obtaining video materials for hand and pedestrian detection, labeling and converting, and randomly pasting pedestrian foreground to background pictures, the problem of high generation cost of multiple gesture data sets in the existing technology under complex backgrounds is solved, and the rapid and automated Dataset extension.

CN115346274BActive Publication Date: 2024-11-05GUANGZHOU ZIWEIYUN TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202211011730.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-08-23
Publication Date
2024-11-05
Estimated Expiration
2042-08-23

AI Technical Summary

Technical Problem

The prior art cannot effectively generate data sets with multiple people making different gestures in complex contexts, and the cost and time-consuming production of such data sets is high and time-consuming, and there is a lack of automated, batch and rapid generation methods.

Method used

By obtaining video materials containing gestures, using hand detection models for hand detection and gesture annotation, pedestrian detection and marking conversion, randomly paste pedestrian foreground into background pictures, forming gesture data pictures, and using portrait segmentation network to identify the foreground area, realizing Extension of the dataset.

Benefits of technology

A method of quickly, automatically, and batch generation of different gesture data sets for multiple people in complex contexts is realized, reducing production costs and time, and expanding the scale of existing gesture data sets.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115346274B_ABST
    Figure CN115346274B_ABST
Patent Text Reader

Abstract

The present invention provides a method for generating a gesture data set, S1, obtaining a video material containing gestures; S2, using a hand detection model to perform hand detection on the video material, and obtaining a gesture label corresponding to the hand; obtaining a first image set corresponding to the gesture; S3, performing pedestrian detection on each image in the first image set to obtain the number of pedestrians Mn detected, and obtaining a pedestrian image set; S4, traversing each image in the first image set B, and converting the gesture annotation corresponding to the hand into the pedestrian annotation of the pedestrian image in the pedestrian image set; S5, foregrounding the pedestrians of each single image in the pedestrian image set, and randomly pasting the pedestrian foreground to a background image to form a gesture data image, and converting the annotation of the pedestrian image into the annotation of the gesture data image. The present invention pastes the foreground area into the background image, so that the existing gesture data set can be quickly and conveniently expanded, and a large number of gesture data sets can be obtained.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device and medium for generating a gesture data set. Background Art

[0002] Gesture detection has always been a hot research direction in the field of human-computer interaction. The gesture detection task includes hand regression and gesture classification. Regression is to locate the position of the hand in the image, and classification is to determine which type of gesture this hand belongs to.

[0003] Currently, deep learning is commonly used to recognize gestures, but deep learning-based methods require a large number of gestures to train the model. Currently, gesture detection datasets are roughly divided into three categories, namely first-person perspective datasets, second-person perspective datasets, and datasets with only one hand in the image. The first-person perspective dataset uses a camera worn on the head to shoot gestures, such as Egogesture. The second-person perspective dataset only has one person in the middle of the image making different gestures, such as Multi-modal gesture, OUHANDS, Kinect Leap Dataset, Senz3D, ChaLearn LAP IsoGD, and ConGD. The datasets with only one hand in the image include American Sign Language, Jochen TrieschStatic Hand Posture, NUS, NUS2, etc. With the current datasets, it is not possible to perform gesture detection and recognition tasks with only one dataset in a complex multi-person background.

[0004] In addition, the cost of producing a dataset of multiple people doing different gestures in a complex background is huge, requiring a lot of manpower, financial resources, material resources and time. Currently, there is still a lack of a method to automatically, batch and quickly generate a dataset of multiple people doing different gestures in a complex background at a very low cost.

[0005] The background description provided herein is for the purpose of generally presenting the context of the present disclosure. Unless otherwise indicated herein, the materials described in this section are not prior art to the claims of this application and are not admitted to be prior art by inclusion in this section. Summary of the invention

[0006] In view of the above technical problems in the related art, the present invention proposes a method for generating a gesture data set, which comprises the following steps:

[0007] S1, obtaining video material containing gestures;

[0008] S2, performing hand detection on the video material using a hand detection model, and performing gesture annotation on the detected hand to obtain a gesture label corresponding to the hand; and obtaining a first image set B corresponding to the gesture;

[0009] S3, performing pedestrian detection on each image in the first image set B to obtain the number of detected pedestrians Mn, and intercepting the pedestrian image from each image in the first image set B according to the pedestrian detection frame to form a pedestrian image set P;

[0010] S4, traversing each image in the first image set B, and when the hand is within a pedestrian detection frame corresponding to a pedestrian image in the pedestrian image set P, converting the gesture annotation corresponding to the hand into a pedestrian annotation of the pedestrian image in the pedestrian image set P;

[0011] S5, for each pedestrian foreground of a single person image in the pedestrian image set, randomly paste the pedestrian foreground onto a background image to form a gesture data image, and convert the annotation of the pedestrian image into the annotation of the gesture data image.

[0012] Specifically, step S3 specifically includes: performing pedestrian detection on each image in the first image set B to obtain a pedestrian detection frame, and expanding the length and width of the pedestrian detection frame.

[0013] Specifically, the step S5 further includes: using a portrait segmentation network to identify each single person image in the pedestrian image set to obtain a pedestrian foreground area.

[0014] Specifically, the step S5 further includes randomly scaling each single person image in the pedestrian image set to obtain a scaled image, and using a portrait segmentation network to identify the scaled image to obtain a pedestrian foreground area.

[0015] Specifically, step S5 also includes: randomly pasting the predetermined number of pedestrian foregrounds onto a background image to form a gesture data image, wherein the predetermined number does not exceed 6 persons.

[0016] In a second aspect, another embodiment of the present invention discloses a gesture data set generating device, which includes the following units:

[0017] A gesture video material acquisition unit, used to acquire video material containing gestures;

[0018] A gesture labeling unit, configured to perform hand detection on the video material using a hand detection model, and perform gesture labeling on the detected hand to obtain a gesture label corresponding to the hand; and obtain a first image set B corresponding to the gesture;

[0019] A single pedestrian image acquisition unit is used to perform pedestrian detection on each image in the first image set B to obtain the number of pedestrians M detected n , and intercepting the pedestrian image from each image in the first image set B according to the pedestrian detection frame to form a pedestrian image set P;

[0020] A single pedestrian image gesture annotation unit is used to traverse each image in the first image set B, and when the hand is within the pedestrian detection frame corresponding to the pedestrian image in the pedestrian image set P, convert the gesture annotation corresponding to the hand into the pedestrian annotation of the pedestrian image in the pedestrian image set P;

[0021] The gesture data set acquisition unit is used to randomly paste the pedestrian foreground of each single person image in the pedestrian image set onto a background image to form a gesture data image, and convert the annotations of the pedestrian image into annotations of the gesture data image.

[0022] Specifically, the single pedestrian image acquisition unit specifically includes: performing pedestrian detection on each image in the first image set B to obtain a pedestrian detection frame, and expanding the length and width of the pedestrian detection frame.

[0023] Specifically, the gesture data set acquisition unit further includes: using a portrait segmentation network to identify each single person image in the pedestrian image set to acquire a pedestrian foreground area.

[0024] Specifically, the gesture data set acquisition unit further includes randomly scaling each single person image in the pedestrian image set to obtain a scaled image, and using a portrait segmentation network to identify the scaled image to obtain a pedestrian foreground area.

[0025] Specifically, the gesture data set acquisition unit further includes: randomly pasting the predetermined number of pedestrian foregrounds onto a background image to form a gesture data image, wherein the predetermined number does not exceed 6 persons.

[0026] In a third aspect, another embodiment of the present invention further discloses a non-volatile storage medium, on which instructions are stored, and when the instructions are executed by a processor, they are used to implement the above-mentioned method for generating a gesture data set.

[0027] The present invention first obtains the annotations of the captured gestures based on the existing gesture dataset and the self-shot gesture dataset, and then crops each image in the gesture dataset into a gesture image containing only one person, and then identifies the foreground area of ​​the single-person gesture image, and pastes the foreground area into the background image, so that the existing gesture dataset can be quickly and conveniently expanded, and a large number of gesture datasets can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 is a flow chart of a method for generating a gesture data set provided by an embodiment of the present invention;

[0030] Figure 2 is a schematic diagram of a gesture data set generating device provided by an embodiment of the present invention;

[0031] Figure 3 It is a schematic diagram of a gesture data set generating device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0033] Embodiment 1

[0034] refer to Figure 1 , this embodiment provides a method for generating a gesture data set, which includes the following steps:

[0035] S1, obtaining video material containing gestures;

[0036] Specifically, the video containing gestures in this embodiment can use a gesture recognition data set from a second-person perspective or a gesture video shot as material, and the gesture video shot is ensured to have only one gesture in each video.

[0037] S2, performing hand detection on the video material using a hand detection model, and performing gesture annotation on the detected hand to obtain a gesture label corresponding to the hand; wherein a first image set B corresponding to the gesture is obtained;

[0038] The gesture detection is pre-labeled using a large model for hand detection, and then manually labeled using labelimg. The hand detection model of this embodiment uses an existing hand detection model, and the hand detection model of this embodiment only needs to be able to detect the hand.

[0039] After detecting the hand, the corresponding gesture annotation can be performed in labelimg.

[0040] In another embodiment, a gesture detection model may be used to detect and output the detected gesture. After the gesture is detected, a manual check may be performed to correct the gesture that is detected incorrectly.

[0041] All images obtained in step S2 are denoted by B, b represents a single image in B, l b Indicates the Yolo format annotation file corresponding to b. Assume that there are N images in B. Represents the nth image b in B n The number corresponding to the i-th row and j-th column of the annotation file.

[0042] S3, performing pedestrian detection on each image in the first image set B to obtain the number of pedestrians detected M n , and intercepting the pedestrian image from each image in the first image set B according to the pedestrian detection frame to form a pedestrian image set P;

[0043] For image b n Perform pedestrian detection and assume that M n Individual, mth n The coordinates of the upper left and lower right corners of the individual pedestrian are And according to the mth n The coordinates of the upper left and lower right corners of the individual pedestrian are From the image b n The images are cut out to form an image p containing only a single person, and all p are grouped into a pedestrian image set P.

[0044] In order to prevent the problem that the pedestrian detection frame does not surround all the hands of the person due to the error of pedestrian detection, the pedestrian frame is enlarged by one tenth of the length and width to obtain the new pedestrian frame coordinates. All pedestrian images obtained by performing pedestrian detection on all images in B are denoted by P, where p represents a single image in P, and l p Indicates the YOLO format annotation file corresponding to p. Assume Indicates the mth image in the nth image in B n The number of the individual's annotation file in row i and column j.

[0045] S4, traversing each image in the first image set B, and when the hand is within a pedestrian detection frame corresponding to a pedestrian image in the pedestrian image set P, converting the gesture annotation corresponding to the hand into a pedestrian annotation of the pedestrian image in the pedestrian image set P;

[0046] In this embodiment, it is assumed that there are N images b1, ..., b2 in the first image set B. N , traverse M in each image n Personal, traversal Each row in the figure is a hand. If the hand is in the pedestrian frame If the hand is inside, place it against b n The annotation is converted into the annotation relative to the person currently traversing. The conversion formula is:

[0047]

[0048]

[0049]

[0050]

[0051]

[0052] The above formula converts the label of the i-th hand in the n-th image in B into the label of the m-th hand in the n-th image in B. n Take the label of the first hand of an individual as an example, where W n , H n is the width and length of the nth image in B.

[0053] S5, for each pedestrian foreground of a single person image in the pedestrian image set, randomly paste the pedestrian foreground onto a background image to form a gesture data image, and convert the annotation of the pedestrian image into the annotation of the gesture data image.

[0054] This embodiment obtains all single person images P and the logically generated annotation information l corresponding to each single person image p in P. p Finally, the size of p is randomly scaled to p_scaled while keeping the aspect ratio unchanged, and then p_scaled is sent to the portrait segmentation network deeppV3+ to obtain the confidence level of each pixel in p_scaled as a pedestrian foreground or background (a decimal between 0 and 1, the larger the value, the greater the probability of being the foreground, and the smaller the value, the greater the probability of being the background).

[0055] Collect and take a number of background images from the Internet, take out a random image with the same size as p_scaled, and put it back to the original position after weighted fusion with p_scaled according to the foreground background confidence. That is, the foreground image in p can be randomly pasted to the background image, and 1-6 people are randomly pasted in each background image. When p_scaled is pasted to the background according to the random position, if there is an area in the position to be pasted that has been pasted, the pasting fails, and then paste it at another random position. If it is not successfully pasted after 100 random attempts, change the next background and continue.

[0056] Assuming that pedestrian detection generates a total of K people, then The number in the i-th row and j-th column of the annotation file representing the k-th person in P. Assume that all images generated after pasting the foreground pedestrian are represented by F, and a single image in F is represented by f. A total of T data are generated, then Indicates the number corresponding to the i-th row and j-th column in the annotation file corresponding to the t-th image in F. The process is as follows: loop through all background images, loop through K p, randomly scale and replace the background for each p, loop through For each row (i.e. each gesture annotation) in , the annotation information corresponding to p is converted into the corresponding annotation information on the background image f. The conversion process formula is:

[0057]

[0058]

[0059]

[0060]

[0061]

[0062] In the above formula, x_random and y_random are random coordinate values ​​in f, and W p , H p , W f , H f are the width and height of the pedestrian image p to be pasted and the width and height of the background image. The above formula takes the conversion of the annotation of the i-th hand of the k-th person in P into the annotation of the first hand in the f-th background as an example.

[0063] This embodiment first obtains the annotations of the captured gestures based on the existing gesture dataset and the self-shot gesture dataset, and then crops each image in the gesture dataset into a gesture image containing only one person, and then identifies the foreground area of ​​the single-person gesture image, and pastes the foreground area into the background image, so that the existing gesture dataset can be quickly and conveniently expanded, and a large number of gesture datasets can be obtained.

[0064] Embodiment 2

[0065] refer to Figure 2 , this embodiment discloses a gesture data set generation device, which includes the following units:

[0066] A gesture video material acquisition unit, used to acquire video material containing gestures;

[0067] Specifically, the video containing gestures in this embodiment can use a gesture recognition data set from a second-person perspective or a gesture video shot as material, and the gesture video shot is ensured to have only one gesture in each video.

[0068] A gesture labeling unit, used to perform hand detection on the video material using a hand detection model, and perform gesture labeling on the detected hand to obtain a gesture label corresponding to the hand; wherein a first image set B corresponding to the gesture is obtained;

[0069] The gesture detection is pre-labeled using a large model for hand detection, and then manually labeled using labelimg. The hand detection model of this embodiment uses an existing hand detection model, and the hand detection model of this embodiment only needs to be able to detect the hand.

[0070] After detecting the hand, the corresponding gesture annotation can be performed in labelimg.

[0071] In another embodiment, a gesture detection model may be used to detect and output the detected gesture. After the gesture is detected, a manual check may be performed to correct the gesture that is detected incorrectly.

[0072] All images obtained by the gesture annotation unit are denoted by B, b represents a single image in B, and l b Indicates the Yolo format annotation file corresponding to b. Assume that there are N images in B. Represents the nth image b in B n The number corresponding to the i-th row and j-th column of the annotation file.

[0073] A single pedestrian image acquisition unit is used to perform pedestrian detection on each image in the first image set B to obtain the number of pedestrians M detected n , and intercepting the pedestrian image from each image in the first image set B according to the pedestrian detection frame to form a pedestrian image set P;

[0074] For image b n Perform pedestrian detection and assume that M n Individual, mth n The coordinates of the upper left and lower right corners of the individual pedestrian are And according to the mth n The coordinates of the upper left and lower right corners of the individual pedestrian are From the image b n The images are cut out to form an image p containing only a single person, and all p are grouped into a pedestrian image set P.

[0075] In order to prevent the problem that the pedestrian detection frame does not surround all the hands of the person due to the error of pedestrian detection, the pedestrian frame is enlarged by one tenth of the length and width to obtain the new pedestrian frame coordinates. All pedestrian images obtained by performing pedestrian detection on all images in B are denoted by P, where p represents a single image in P, and l p Indicates the YOLO format annotation file corresponding to p. Assume Indicates the mth image in the nth image in B n The number of the individual's annotation file in row i and column j.

[0076] A single pedestrian image gesture annotation unit is used to traverse each image in the first image set B, and when the hand is within the pedestrian detection frame corresponding to the pedestrian image in the pedestrian image set P, convert the gesture annotation corresponding to the hand into the pedestrian annotation of the pedestrian image in the pedestrian image set P;

[0077] In this embodiment, it is assumed that there are N images b1, ..., b2 in the first image set B. N , traverse M in each image n Personal, traversal Each row in the figure is a hand. If the hand is in the pedestrian frame If the hand is inside, place it against b n The annotation is converted into the annotation relative to the person currently traversing. The conversion formula is:

[0078]

[0079]

[0080]

[0081]

[0082]

[0083] The above formula converts the label of the i-th hand in the n-th image in B into the label of the m-th hand in the n-th image in B. n Take the label of the first hand of an individual as an example, where W n , H n is the width and length of the nth image in B.

[0084] The gesture data set acquisition unit is used to randomly paste the pedestrian foreground of each single person image in the pedestrian image set onto a background image to form a gesture data image, and convert the annotations of the pedestrian image into annotations of the gesture data image.

[0085] This embodiment obtains all single person images P and the logically generated annotation information l corresponding to each single person image p in P. p Finally, the size of p is randomly scaled to p_scaled while keeping the aspect ratio unchanged, and then p_scaled is sent to the portrait segmentation network deeppV3+ to obtain the confidence level of each pixel in p_scaled as a pedestrian foreground or background (a decimal between 0 and 1, the larger the value, the greater the probability of being the foreground, and the smaller the value, the greater the probability of being the background).

[0086] Collect and take a number of background images from the Internet, take out a random image with the same size as p_scaled, and put it back to the original position after weighted fusion with p_scaled according to the foreground background confidence. That is, the foreground image in p can be randomly pasted to the background image, and 1-6 people are randomly pasted in each background image. When p_scaled is pasted to the background according to the random position, if there is an area in the position to be pasted that has been pasted, the pasting fails, and then paste it at another random position. If it is not successfully pasted after 100 random attempts, change the next background and continue.

[0087] Assuming that pedestrian detection generates a total of K people, then The number in the i-th row and j-th column of the annotation file representing the k-th person in P. Assume that all images generated after pasting the foreground pedestrian are represented by F, and a single image in F is represented by f. A total of T data are generated, then Indicates the number corresponding to the i-th row and j-th column in the annotation file corresponding to the t-th image in F. The process is as follows: loop through all background images, loop through K p, randomly scale and replace the background for each p, loop through For each row (i.e. each gesture annotation) in , the annotation information corresponding to p is converted into the corresponding annotation information on the background image f. The conversion process formula is:

[0088]

[0089]

[0090]

[0091]

[0092]

[0093] In the above formula, x_random and y_random are random coordinate values ​​in f, and W p , H p , W f , H fare the width and height of the pedestrian image p to be pasted and the width and height of the background image. The above formula takes the conversion of the annotation of the i-th hand of the k-th person in P into the annotation of the first hand in the f-th background as an example.

[0094] This embodiment first obtains the annotations of the captured gestures based on the existing gesture dataset and the self-shot gesture dataset, and then crops each image in the gesture dataset into a gesture image containing only one person, and then identifies the foreground area of ​​the single-person gesture image, and pastes the foreground area into the background image, so that the existing gesture dataset can be quickly and conveniently expanded, and a large number of gesture datasets can be obtained.

[0095] Embodiment 3

[0096] refer to Figure 3 , Figure 3 : is a schematic diagram of the structure of a gesture data set generation device of this embodiment. The gesture data set generation device 20 of this embodiment includes a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the processor 21 executes the computer program, the steps in the above method embodiment are implemented. Alternatively, when the processor 21 executes the computer program, the functions of each module / unit in the above device embodiments are implemented.

[0097] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 22 and executed by the processor 21 to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the gesture data set generating device 20. For example, the computer program may be divided into the modules in the second embodiment. For the specific functions of each module, please refer to the working process of the device described in the above embodiment, which will not be repeated here.

[0098] The gesture dataset generating device 20 may include, but is not limited to, a processor 21 and a memory 22. Those skilled in the art will appreciate that the schematic diagram is merely an example of the gesture dataset generating device 20 and does not constitute a limitation on the gesture dataset generating device 20. The gesture dataset generating device 20 may include more or fewer components than shown in the diagram, or may combine certain components, or different components. For example, the gesture dataset generating device 20 may also include input and output devices, network access devices, buses, and the like.

[0099] The processor 21 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor 21 is the control center of the gesture data set generation device 20, and uses various interfaces and lines to connect various parts of the entire gesture data set generation device 20.

[0100] The memory 22 can be used to store the computer program and / or module. The processor 21 implements various functions of the gesture data set generation device 20 by running or executing the computer program and / or module stored in the memory 22 and calling the data stored in the memory 22. The memory 22 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 22 can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0101] Wherein, if the module / unit integrated in the gesture data set generating device 20 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor 21, the steps of the above-mentioned method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media does not include electrical carrier signals and telecommunication signals.

[0102] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0103] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for generating a gesture data set, comprising the following steps: S1, obtaining video material containing gestures; S2, performing hand detection on the video material using a hand detection model, and performing gesture annotation on the detected hand to obtain a gesture label corresponding to the hand; Acquire a first image set B corresponding to the gesture; S3, performing pedestrian detection on each image in the first image set B to obtain the number of detected pedestrians Mn, and intercepting the pedestrian image from each image in the first image set B according to the pedestrian detection frame to form a pedestrian image set P; S4, traversing each image in the first image set B, and when the hand is within a pedestrian detection frame corresponding to a pedestrian image in the pedestrian image set P, converting the gesture annotation corresponding to the hand into a pedestrian annotation of the pedestrian image in the pedestrian image set P; S5, for each pedestrian foreground of a single person image in the pedestrian image set, randomly paste the pedestrian foreground onto a background image to form a gesture data image, and convert the annotation of the pedestrian image into the annotation of the gesture data image.

2. According to the method of claim 1, step S3 specifically comprises: Pedestrian detection is performed on each image in the first image set B to obtain a pedestrian detection frame, and the length and width of the pedestrian detection frame are enlarged.

3. The method according to claim 1, wherein step S5 further comprises: For each single person image in the pedestrian image set, a portrait segmentation network is used to identify and obtain the pedestrian foreground area.

4. According to the method of claim 3, step S5 further comprises randomly scaling each single person image in the pedestrian image set to obtain a scaled image, and using a portrait segmentation network to identify the scaled image to obtain a pedestrian foreground area.

5. According to the method of claim 4, step S5 further comprises: A predetermined number of pedestrian foregrounds are randomly pasted onto a background image to form a gesture data image, wherein the predetermined number does not exceed 6 persons.

6. A gesture data set generation device, comprising the following units: A gesture video material acquisition unit, used to acquire video material containing gestures; A gesture labeling unit, used to perform hand detection on the video material using a hand detection model, and perform gesture labeling on the detected hand to obtain a gesture label corresponding to the hand; Acquire a first image set B corresponding to the gesture; A single pedestrian image acquisition unit is used to perform pedestrian detection on each image in the first image set B to obtain the number of pedestrians M detected n , and intercepting the pedestrian image from each image in the first image set B according to the pedestrian detection frame to form a pedestrian image set P; A single pedestrian image gesture annotation unit is used to traverse each image in the first image set B, and when the hand is within the pedestrian detection frame corresponding to the pedestrian image in the pedestrian image set P, convert the gesture annotation corresponding to the hand into the pedestrian annotation of the pedestrian image in the pedestrian image set P; The gesture data set acquisition unit is used to randomly paste the pedestrian foreground of each single person image in the pedestrian image set onto a background image to form a gesture data image, and convert the annotation of the pedestrian image into the annotation of the gesture data image.

7. The device according to claim 6, wherein the single pedestrian image acquisition unit specifically comprises: Pedestrian detection is performed on each image in the first image set B to obtain a pedestrian detection frame, and the length and width of the pedestrian detection frame are enlarged.

8. The device according to claim 6, wherein the gesture data set acquisition unit further comprises: For each single person image in the pedestrian image set, a portrait segmentation network is used to identify and obtain the pedestrian foreground area.

9. According to the device of claim 8, the gesture data set acquisition unit further comprises randomly scaling each single person image in the pedestrian image set to obtain a scaled image, and using a portrait segmentation network to identify the scaled image to obtain a pedestrian foreground area.

10. The device according to claim 9, wherein the gesture data set acquisition unit further comprises: A predetermined number of pedestrian foregrounds are randomly pasted onto a background image to form a gesture data image, wherein the predetermined number does not exceed 6 persons.

11. A non-volatile storage medium, wherein instructions are stored on the non-volatile storage medium, and when the instructions are executed by a processor, they are used to implement a gesture data set generation method according to any one of claims 1 to 5.