Method and device for generating training data, computer device and storage medium

Abstract

The present disclosure provides a training data generation method and device, computer equipment and a storage medium, belonging to the technical field of data processing, wherein the training data generation method comprises: obtaining a target image; the target image has a preset background and a hand feature; generating a labeled mask for the hand feature according to the pixel value of each pixel point in the target image and the target pixel value of the preset background stored in advance; determining the channel value of the transparent channel in the target image according to the pixel value of each pixel point in the labeled mask, and taking the target image with the transparent channel as a target foreground image; determining training data according to the background image set stored in advance, the target foreground image and the labeled mask, so as to train a gesture segmentation network model.

Description

Technical Field

[0001] This disclosure belongs to the field of data processing technology, and specifically relates to a method, apparatus, computer device, and storage medium for generating training data. Background Technology

[0002] For any neural network model applied in any field, a large amount of training data is required during the training process to obtain a trained neural network model with the target function.

[0003] In recent years, with the continuous improvement of education quality, intelligent learning products have been increasingly used to assist education, among which gesture detection has received increasing attention in human-computer interaction applications. For intelligent learning products, gesture segmentation network models differ from object detection models. For example, the training data required for a face detection model only needs to be annotated with a bounding box on a face image. However, the training data required for a gesture segmentation network model includes sample images and a labeled mask formed by multi-point annotation of the hand contour in the sample image. Compared to object detection models that only annotate a single bounding box for the target object, the gesture segmentation network model uses multi-point annotation on the sample images, resulting in a huge annotation cost. For example, for the hand contour of each sample image, approximately the number of pixels corresponding to the hand contour need to be annotated, thus the annotation cost is huge and the efficiency of generating training data is low. Summary of the Invention

[0004] This disclosure aims to at least solve one of the technical problems existing in the prior art, and to provide a method, apparatus, computer device, and storage medium for generating training data.

[0005] Firstly, the technical solution adopted to solve the technical problem disclosed herein is a method for generating training data, including:

[0006] Acquire a target image; the target image has a preset background and hand features;

[0007] Based on the pixel values ​​of each pixel in the target image and the target pixel values ​​of the preset background stored in advance, a labeled mask is generated for the hand features;

[0008] Based on the pixel values ​​of each pixel in the labeled mask, the channel value of the transparency channel in the target image is determined, and the target image with the transparency channel is used as the target foreground image;

[0009] Training data is determined based on a pre-stored set of background images, the target foreground image, and the labeled mask, for use in training a gesture segmentation network model.

[0010] In some embodiments, the preset background is a solid color background;

[0011] The steps for determining the target image include:

[0012] Hand images are captured in an environment with the aforementioned solid color background to obtain the original image;

[0013] The original image is converted to the HSV color model to obtain an HSV image;

[0014] The target image is obtained by applying Gaussian blur to the HSV image.

[0015] In some embodiments, generating a labeled mask corresponding to the hand features based on the pixel values ​​of each pixel in the target image and the target pixel values ​​of the preset background stored in advance includes:

[0016] Based on the size information of the target image, a first solid color image with the same size as the target image is generated; the pixel value of each pixel in the first solid color image is a first preset value.

[0017] Based on the pixel values ​​of each pixel in the target image and the target pixel values ​​of the preset background stored in advance, a first pixel in the target image that is different from the target pixel value is determined;

[0018] The pixel value of the second pixel point corresponding to the first pixel point in the first solid color image is adjusted to a second preset value to obtain the first mask;

[0019] The first mask is processed using a preset algorithm to obtain the labeled mask.

[0020] In some embodiments, processing the first mask using a preset algorithm to obtain the labeled mask includes:

[0021] The first mask is expanded to obtain a first intermediate mask;

[0022] The first intermediate mask is etched to obtain the second intermediate mask;

[0023] Using a contour detection algorithm, contour recognition is performed on the second intermediate mask to determine the largest region where the contours are connected;

[0024] The pixel values ​​of other regions outside the maximum region corresponding to the second intermediate mask are adjusted to the first preset value to obtain the annotation mask.

[0025] In some embodiments, processing the first mask using a preset algorithm to obtain the labeled mask includes:

[0026] If it is determined that the hand features in the target image include arm features, the first mask is processed using a preset algorithm to obtain the second mask;

[0027] The key points of the palm in the target image are extracted using a pre-trained neural network model, and a first detection box is determined based on the key points of the palm.

[0028] The first detection frame is expanded using a preset expansion factor to obtain a second detection frame;

[0029] The annotation mask is obtained based on the second detection box and the second mask.

[0030] In some embodiments, obtaining the annotation mask based on the second detection box and the second mask includes:

[0031] Based on the center position of the second detection frame, a circular area is determined with the center position as the center and the distance from the center to a preset key point in the palm as the radius;

[0032] The pixel values ​​of the pixels outside the circular area in the second mask are adjusted to the first preset value to obtain the annotation mask.

[0033] In some embodiments, determining the training data based on a pre-stored set of background images, the target foreground image, and the labeled mask includes:

[0034] Using the same preset parameters, perform graphic transformation on the target foreground image and the corresponding annotation mask to obtain an updated target foreground image and an updated annotation mask;

[0035] Based on the pixel values ​​of each pixel in the updated annotation mask, determine the target region corresponding to the hand feature in the updated annotation mask;

[0036] Extract a portion of the foreground image corresponding to the target region from the updated target foreground image, and extract a portion of the annotation mask corresponding to the target region from the updated annotation mask;

[0037] For a background image in the set of background images, the foreground image is pasted onto the area to be pasted in the background image to generate a composite image.

[0038] Training data is generated based on the synthesized image and the partially labeled mask.

[0039] In some embodiments, the method for generating training data further includes:

[0040] For different target foreground images and the annotation mask corresponding to the target foreground images, multiple partial foreground images and partial annotation masks corresponding to each of the partial foreground images are determined respectively;

[0041] The step of pasting the foreground image onto a predetermined location area in the background image to generate the composite image includes:

[0042] Multiple partial annotation masks are applied to the areas to be applied in the background image to generate the composite image; the overlap of each partial foreground image in the composite image is less than any value between 0.3 and 0.5.

[0043] In some embodiments, generating training data based on the synthesized image and the partially labeled mask includes:

[0044] Based on the synthesized image and the partial labeled mask, the contour of the synthesized image is processed to obtain a first optimized image;

[0045] Based on the first optimized image and the partial labeled mask, the contour of the first optimized image is processed to obtain the second optimized image;

[0046] The second optimized image and the partially labeled mask are used as the training data.

[0047] In some embodiments, processing the contour of the synthesized image based on the synthesized image and the partial labeled mask to obtain a first optimized image includes:

[0048] The partial label mask is expanded to obtain a third intermediate mask;

[0049] The first contour mark map is determined based on the pixel values ​​of each pixel in the third intermediate mask and the pixel values ​​of each pixel in the partial annotation mask;

[0050] A third optimized image is obtained based on the synthesized image, the first contour mark image, a second solid color image of the same size as the synthesized image, and a first preset synthesis weight;

[0051] The third intermediate mask is expanded to obtain the fourth intermediate mask;

[0052] The second contour mark map is determined based on the pixel values ​​of each pixel in the fourth intermediate mask and the pixel values ​​of each pixel in the third intermediate mask;

[0053] The first optimized image is obtained based on the third optimized image, the second contour marker image, the second solid color image, and the second preset synthesis weight.

[0054] In some embodiments, the step of contour processing of the first optimized image based on the first optimized image and the partial annotation mask to obtain a second optimized image includes:

[0055] Using a contour detection algorithm, contour extraction is performed on the first optimized image to determine a third contour marker image;

[0056] The third contour mark image is dilated to obtain the fourth contour mark image;

[0057] Gaussian blur is applied to the first optimized image to obtain the fourth optimized image;

[0058] A second optimized image is obtained based on the first optimized image, the fourth contour marker image, and the fourth optimized image.

[0059] Secondly, this disclosure also provides a training data generation apparatus, including an image acquisition module, a mask generation module, a foreground image generation module, and a training data determination module;

[0060] The image acquisition module is configured to acquire a target image; the target image has a preset background and hand features;

[0061] The mask generation module is configured to generate a labeled mask for the hand features based on the pixel values ​​of each pixel in the target image and the target pixel values ​​of the preset background stored in advance.

[0062] The foreground image generation module is configured to determine the channel value of the transparency channel in the target image based on the pixel value of each pixel in the labeled mask, and to use the target image with the transparency channel as the target foreground image;

[0063] The training data determination module is configured to determine training data based on a pre-stored set of background images, the target foreground image, and the labeled mask, for use in training a gesture segmentation network model.

[0064] Thirdly, embodiments of this disclosure also provide a computer device, comprising: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the steps of the training data generation method as described in any one of the first aspects are performed.

[0065] Fourthly, embodiments of this disclosure also provide a computer non-transient readable storage medium, wherein a computer program is stored on the computer non-transient readable storage medium, and the computer program is executed by a processor to perform the steps of the training data generation method as described in any one of the first aspects. Attached Figure Description

[0066] Figure 1 A flowchart illustrating a method for generating training data provided in this embodiment of the disclosure;

[0067] Figure 2 A schematic diagram of an exemplary original image provided for an embodiment of this disclosure;

[0068] Figure 3 A schematic diagram of an exemplary annotation mask provided for an embodiment of this disclosure;

[0069] Figure 4 A schematic diagram of key hand point detection and inner and outer detection frames provided for embodiments of this disclosure;

[0070] Figure 5 This is a schematic diagram of a circular region in a target image provided in an embodiment of this disclosure;

[0071] Figure 6 A schematic diagram of another exemplary annotation mask provided for an embodiment of this disclosure;

[0072] Figure 7 A schematic diagram of a composite image formed by pasting a portion of a foreground image onto a background image, as provided in an embodiment of this disclosure;

[0073] Figure 8 A schematic diagram of a first optimized image formed by shading an exemplary synthetic image, provided in an embodiment of this disclosure;

[0074] Figure 9 A schematic diagram of a training data generation apparatus provided in an embodiment of this disclosure;

[0075] Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0077] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an,” “a,” or “the,” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “comprising,” “including,” or “including,” and similar terms mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects.

[0078] In this disclosure, "multiple or several" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0079] Firstly, embodiments of this disclosure provide a method for generating training data. Figure 1 A flowchart of a method for generating training data provided in this disclosure embodiment is shown below. Figure 1 As shown, steps S11 to S14 are included:

[0080] S11. Obtain the target image.

[0081] The target image has a preset background and hand features.

[0082] For example, the target image can be the original image obtained by capturing a hand image in a preset background environment. Alternatively, the target image can also be an image obtained by processing the original image. Here, processing the original image includes, for example, color space conversion and / or Gaussian blurring. The captured original image is an RGB image, and the color space conversion can be, for example, converting from RGB color mode to HSV color model.

[0083] The RGB color model is an industry-standard color model that uses variations in the red (R), green (G), and blue (B) color channels and their combinations to create a wide variety of colors. RGB represents the colors of the red, green, and blue channels, and this standard encompasses almost all colors perceptible to human vision. The HSV color model (Hue, Saturation, Value) is a color space created based on the intuitive characteristics of color, also known as the Hexcone Model. The HSV color model refers to a subset of visible light in the H, S, V three-dimensional color space, containing all colors within a specific color gamut. Here, H represents hue, S represents saturation, and V represents lightness.

[0084] For example, the preset background can be a solid color background, such as a single color background like green or red; or, the preset background can also be a background composed of multiple solid colors, such as a background composed of red and green stripes. This disclosure uses a green curtain background as an example for illustration, but this does not constitute a limitation on the scope of protection for other solid color backgrounds.

[0085] S12. Generate a labeled mask for hand features based on the pixel values ​​of each pixel in the target image and the target pixel values ​​of the preset background stored in advance.

[0086] The target pixel value of the preset background does not refer to a single pixel value. Instead, it refers to the pixel values ​​(R, G, B) or (H, S, V) covered by each color within the preset background. Taking a green screen as the preset background and an HSV format image as an example, the pixel value of any pixel includes sub-pixel values ​​from the H channel, S channel, and V channel. The target pixel value (H, S, V) for the pixels corresponding to the green screen portion can have sub-pixel values ​​ranging from [50, 150], [120, 255], and [120, 255] respectively. Since the range of each channel for the target pixel value of the pixels corresponding to the green screen portion is known in advance, it can be stored beforehand for later retrieval.

[0087] For the annotation masking of hand features, the pixel values ​​of the pixels corresponding to the hand features are distinguished from the pixel values ​​of the pixels corresponding to the background. For example, the pixel values ​​of the pixels corresponding to the hand features are all (1,1,1), while the pixel values ​​of the pixels corresponding to non-hand features are all (0,0,0).

[0088] S13. Based on the pixel values ​​of each pixel in the labeled mask, determine the channel value of the transparency channel in the target image, and use the target image with the transparency channel as the target foreground image.

[0089] The transparency channel can be an alpha channel. An alpha channel value of 0 indicates transparency, and an alpha channel value of 1 indicates opacity. The target image is a three-channel image, such as an RGB image or an HSV format image.

[0090] The pixel values ​​of each point in the annotation mask are (1,1,1) or (0,0,0). The size of the annotation mask is the same as the size of the target image, meaning the number of pixels is the same, for example, M×N pixels. Pixel A in the annotation mask corresponds one-to-one with pixel B in the target image. The alpha channel value of pixel B can be determined to be 1 based on the pixel value of pixel A (1,1,1); conversely, the alpha channel value of pixel B can be determined to be 0 based on the pixel value of pixel A (0,0,0).

[0091] When the target image is in HSV format, color space conversion is performed to determine an RGB image. A target image with an alpha channel is a four-channel image, i.e., an RGBA image, where A represents the alpha channel. By determining the alpha channel value of each pixel in the target image, an RGBA image, i.e., the target foreground image, is obtained. For example, the pixel value of the hand feature in the target foreground image is (R, G, B, 1), meaning the hand feature in the target foreground image is a non-transparent area; the pixel value of the background portion of the target foreground image excluding the hand feature is (R, G, B, 0), meaning the portion of the target foreground image excluding the hand feature is a transparent area.

[0092] S14. Determine the training data based on the pre-stored background image set, target foreground image, and labeled mask.

[0093] The background image set includes images collected from real-world application scenarios of the gesture segmentation network model to be trained. For example, the gesture segmentation network model is used in intelligent learning products, primarily generating training data for the "finger-tip reading" application scenario. Therefore, the background images are mainly picture book images, such as children's cartoon drawings.

[0094] For example, for each background image, at least a portion of the target foreground image can be pasted onto the background image to form a synthetic image, and the synthetic image and the labeled mask can be used as training data to train a gesture segmentation network model.

[0095] For example, for each image, the processed target foreground image can be pasted onto the background image to form a synthetic image, and the synthetic image and the labeled mask can be used as training data to train the gesture segmentation network model.

[0096] It should be noted that the training data generated in this embodiment is mainly used for the gesture segmentation network model. Unlike the image recognition network model, the label of the training data for the gesture segmentation network model is a labeled mask, not a detection box marked on the training data.

[0097] In steps S11 to S14 above, a background for gesture acquisition is preset (e.g., a green screen). This allows the target pixel values ​​of the preset background to be compared with the pixel values ​​of each pixel in the target image, automatically generating a labeled mask for hand features. Compared to the manual labeling method in the prior art, the embodiments of this disclosure greatly improve the efficiency of generating labeled masks and reduce labeling costs. Meanwhile, the labeled mask generated in step S12 can not only serve as a necessary condition for generating training data in step S14 (or as part of the training data), but also, in step S13, the generated labeled mask can be used to directly determine the alpha channel data of the target image. This saves the process of repeatedly detecting and generating transparent layers in the target image, shortens the overall processing cycle for generating training data, and thus improves the efficiency of generating training data.

[0098] In some embodiments, taking a preset background as a solid color background as an example, the step of determining the target image in S11 includes S111 to S113, wherein:

[0099] S111. Acquire hand images in an environment with a solid color background to obtain the original image.

[0100] A solid color background is, for example, a background of a single color such as green or red. This embodiment of the disclosure uses a green curtain background as an example. Figure 2 A schematic diagram of an exemplary original image provided for an embodiment of this disclosure, such as... Figure 2 As shown, the original image has a green screen background and hand features. These hand features include those of a complete hand and those of a portion of the arm.

[0101] Of course, in actual image acquisition, one can also acquire raw images containing only the palm as a hand feature, so that the hand features in subsequent processes will only include the palm feature. However, in actual image acquisition, it is quite difficult to acquire raw images containing only the palm as a hand feature. For example, during image acquisition, it is necessary to strictly ensure that the palm feature is not missing and that the arm feature is not included. The palm would have to be close to the edge of the image, which is not conducive to subsequent image optimization of the palm feature. Therefore, this disclosure takes the acquisition of hand feature images with complete palm features and some arm features as an example for processing.

[0102] S112. Perform HSV color model conversion on the original image to obtain an HSV image.

[0103] The original image is an RGB image.

[0104] Under normal, uniform lighting conditions, the sub-pixel values ​​of the H channel in the green screen background can be in the range of [50, 150], the sub-pixel values ​​of the S channel can be in the range of [120, 255], and the sub-pixel values ​​of the V channel can be in the range of [120, 255].

[0105] S113. Apply Gaussian blur to the HSV image to obtain the target image.

[0106] For example, a Gaussian blur algorithm is used to smooth the HSV image with a Gaussian kernel of 3×3 to remove small black noise in the green screen background and obtain the target image.

[0107] The above steps S111 to S113 optimize the acquired original image and remove the background interference to the foreground. This preliminary preparation improves the accuracy of the subsequent generation of annotation masks.

[0108] In some embodiments, the specific steps for generating the annotation mask in step S12 include S121 to S124, wherein:

[0109] S121. Generate a first solid color image with the same size as the target image based on the size information of the target image.

[0110] In the first solid color image, the pixel value of each pixel is a first preset value.

[0111] If the first preset value is 0, then the pixel value of each pixel in the first solid color image is (0,0,0), which means that a pure black image with the same size as the target image is generated. Here, "same size" means that the number of pixels is the same.

[0112] S122. Based on the pixel values ​​of each pixel in the target image and the target pixel values ​​of the preset background stored in advance, determine the first pixel in the target image that is different from the target pixel value.

[0113] For each pixel in the target image, if the sub-pixel value of any of the H, S, and V channels corresponding to its pixel value is different from the channel value of any of the H, S, and V channels in the target pixel value, or in other words, the sub-pixel value of the H channel corresponding to the pixel value is not within [50, 150], the sub-pixel value of the S channel is not within [120, 255], or the sub-pixel value of the V channel is not within [120, 255], then it indicates that the pixel in the target image is the first pixel. This can also be understood as... Figure 2 The pixels corresponding to the features of the middle hand.

[0114] S123. Adjust the pixel value of the second pixel corresponding to the first pixel in the first solid color image to the second preset value to obtain the first mask.

[0115] The second preset value is different from the first preset value. When the first preset value is 0, the second preset value can be set to 1.

[0116] The size of the first solid color image is the same as the size of the target image, so the pixels in the first solid color image correspond one-to-one with the pixels in the target image. Given the first pixel in the target image, the second pixel in the first solid color image that corresponds one-to-one with the first pixel is determined, and the pixel value (0,0,0) of the second pixel is adjusted to (1,1,1), resulting in the first mask. At this point, the first mask appears as a black and white image.

[0117] S124. Process the first mask using a preset algorithm to obtain the labeled mask.

[0118] As shown in S123, the process of generating the first mask involves adjusting the second pixel in the pure black image to a white pixel. The determination of the second pixel depends entirely on the target pixel value and is limited by the shooting environment (lighting, green screen flatness, etc.) and the detection error of the first pixel. Therefore, step S124 optimizes the first mask. For example, the preset algorithm may include, but is not limited to, image erosion algorithms, image dilation algorithms, contour detection algorithms, keypoint extraction algorithms, detection box scaling algorithms, etc.

[0119] In some embodiments, in addition to generating the annotation mask using S121-S124 described above, the method described in this embodiment can also be used. For example, based on the pixel values ​​of each pixel in the target image and the target pixel values ​​of a pre-stored preset background, a third pixel in the target image whose value differs from the target pixel value is determined. The pixel value of the third pixel is adjusted to a second preset value, and the pixel value of a fourth pixel in the target image (excluding the third pixel) is adjusted to a first preset value, thus obtaining a first mask. The first mask is then processed using a preset algorithm to obtain the annotation mask. Here, the first preset value is 0, and the second preset value is 1.

[0120] In some embodiments, the specific steps for optimizing the first mask to obtain the annotation mask in step S124 include S124-1-1 to S124-1-4, wherein:

[0121] S124-1-1. Expand the first mask to obtain the first intermediate mask.

[0122] For example, using an image dilation algorithm, the first mask is processed with a dilation kernel of 4×4 to obtain a first intermediate mask. The dilation kernel of 4×4 is only an example, and other sizes of dilation kernels can also be selected, which is not limited in this embodiment.

[0123] S124-1-2. The first intermediate mask is etched to obtain the second intermediate mask.

[0124] For example, using an image erosion algorithm, the first intermediate mask is processed with an erosion kernel of 6×6 to obtain the second intermediate mask. The 6×6 erosion kernel is only an example; of course, other sizes of erosion kernels can be selected, and this embodiment of the disclosure does not limit this.

[0125] The above steps S124-1-1 and S124-1-2 can remove contour noise in the first mask and improve the accuracy of subsequent annotation mask generation.

[0126] S124-1-3. Using a contour detection algorithm, perform contour recognition on the second intermediate mask to determine the largest region where the contours are connected.

[0127] Contour detection algorithms can include, for example, the contour search function provided by the cross-platform computer vision and machine learning software library OpenCV. Using OpenCV, all connected regions in the second intermediate mask are found, and the areas of these connected regions are calculated to determine the largest connected region, which is the region containing the hand features.

[0128] S124-1-4. Adjust the pixel values ​​of other areas outside the maximum area in the second intermediate mask to the first preset value to obtain the annotation mask.

[0129] The largest area is where the hand feature is located. Therefore, the areas outside the largest area in the second intermediate mask are the non-hand feature areas. The pixel values ​​of the non-hand feature areas are adjusted to the first preset value of 0, which means adjusting white pixels in the non-hand feature areas to black.

[0130] Due to limitations imposed by the shooting environment (lighting, green screen flatness, etc.), interference ripples may appear in the background area of ​​the original image. Step S124-1-4 adjusts white pixels in non-hand feature areas to black, optimizing the effect of interference ripples and improving the accuracy of the annotation mask.

[0131] Figure 3 A schematic diagram of an exemplary annotation mask provided for an embodiment of this disclosure, such as... Figure 3 As shown, the annotation mask is in Figure 2 The image shown is generated based on the original image, and the visual effect of the masked annotation is a black and white image. Figure 3 The annotation mask shown is the annotation mask after optimization in steps S124-1-1 to S124-1-4.

[0132] In some embodiments, when the hand features in the target image include arm features, in order to eliminate the influence of arm features, which are interfering features relative to the palm, after optimizing the first mask according to steps S124-1-4 to S124-1-4, the arm features in the second mask are removed, and the second mask with the arm features removed is used as the annotation mask.

[0133] Regarding step S124 above, the steps for optimizing and obtaining the annotation mask include S124-2-1 to S124-2-4, wherein:

[0134] S124-2-1. If it is determined that the hand features in the target image include arm features, the first mask is processed using a preset algorithm to obtain the second mask.

[0135] For example, an image detection algorithm is used to detect gestures in the target image. If it is determined that the hand features include arm features, a preset algorithm is used to process the first mask to obtain the second mask.

[0136] For example, if it is known that arm features will definitely exist during the process of acquiring the original image, then there is no need to perform gesture detection. The process of "processing the first mask using a preset algorithm to obtain the second mask" can be performed directly.

[0137] This step S124-2-1 processes the first mask using a preset algorithm, which can be referred to in the processing steps S124-1-1 to S124-1-4 above. Specifically, it includes: dilating the first mask to obtain a first intermediate mask; eroding the first intermediate mask to obtain a second intermediate mask; using a contour detection algorithm to perform contour recognition on the second intermediate mask to determine the maximum connected region; adjusting the pixel values ​​of other regions outside the maximum region in the second intermediate mask to a first preset value to obtain the second mask. Figure 3 As shown. For detailed processing procedures, please refer to the specific implementation process of steps S124-1-1 to S124-1-4 above. Repeated parts will not be repeated.

[0138] S124-2-2. Use a pre-trained neural network model to extract key points of the palm in the target image, and determine the first detection box based on the key points of the palm.

[0139] Pre-trained neural network models can be, for example, pre-trained models for palm detection and keypoint extraction. Specifically, a YOLOv5 detector with a backbone network as the ReXNeT keypoint extractor can be chosen; alternatively, the open-source model MediaPipe can be selected. This embodiment uses the MediaPipe model to obtain palm keypoints as an example for illustration. Using MediaPipe for auxiliary annotation can reduce annotation costs.

[0140] Figure 4 A schematic diagram of the palm key point detection and the inner and outer detection frames provided in the embodiments of this disclosure is shown below. Figure 4 As shown, 41 represents the first detection box; 42 represents the second detection box; and 43 represents the key point at the base of the palm. The hand key points are extracted from the target image using MediaPipe, specifically 21 hand key points. Based on the coordinate values ​​of these key points, for example, the minimum x-coordinate is used as the x-coordinate of the top-left corner of the first detection box 41, and the minimum y-coordinate is used as the y-coordinate of the top-left corner of the first detection box 41; the maximum x-coordinate is used as the x-coordinate of the bottom-right corner of the first detection box 41; and the maximum y-coordinate is used as the y-coordinate of the bottom-right corner of the first detection box 41. Thus, the first detection box 41 is determined. Here, the generated first detection box is a rectangle.

[0141] S124-2-3. Expand the first detection frame using a preset expansion factor to obtain the second detection frame.

[0142] To ensure that complete full-hand features can be obtained, the first detection box 41 is expanded outward. The preset expansion factor can be determined based on experience, for example, the value of the preset expansion factor is between 1.01 and 1.5.

[0143] For example, such asFigure 4 As shown, with the geometric center of the first detection frame 41 as the reference, the width and height are increased by a factor of 1.05 respectively to obtain the second detection frame 42. The geometric centers of the first detection frame 41 and the second detection frame 42 are at the same position.

[0144] S124-2-4. Based on the second detection frame and the second mask, the annotation mask is obtained.

[0145] In one possible implementation, to improve efficiency, the pixel values ​​of pixels outside the second detection box in the second mask can be directly adjusted to the first preset value to obtain the annotation mask.

[0146] In another possible implementation, if the arm features are cut according to the second rectangular detection box, the final connection between the palm and the arm will be a straight line, resulting in unnatural palm features. Training data generated based on such hand features is not conducive to subsequent model training and affects training accuracy. Therefore, based on the second detection box, this implementation further adjusts the corresponding region of the palm features.

[0147] Specifically, Figure 5 This is a schematic diagram of a circular region in a target image provided in an embodiment of this disclosure. Figure 6 A schematic diagram of another exemplary annotation mask provided for embodiments of this disclosure, such as... Figure 5 and Figure 6 As shown, based on the center position of the second detection box, a circular region is determined with the center position as the center and the distance from the center to a preset key point located in the palm as the radius; the pixel values ​​of pixels outside the circular region in the second mask are adjusted to the first preset value to obtain the annotation mask. Here, the first preset value is 0. The preset key point can be the palm root key point 43 among the 21 palm key points.

[0148] In some embodiments, the specific process of generating training data for step S14 includes steps S141 to S145, wherein:

[0149] S141. Using the same preset parameters, perform graphic transformation on the target foreground image and its corresponding annotation mask to obtain the updated target foreground image and the updated annotation mask.

[0150] It should be noted that this disclosure uses the generation of training data from a single original image as an example, in which the generated target foreground image and the labeled mask have a one-to-one correspondence. That is, the target foreground image has a corresponding labeled mask. The process of generating training data from other original images is the same as the method for generating training data provided in this disclosure, and will not be listed one by one.

[0151] Graphical transformations refer to performing basic geometric transformations on graphics, such as rotation, scaling, cropping, etc.

[0152] For rotation operations, preset parameters include parameters such as rotation reference point and rotation angle.

[0153] For scaling operations, preset parameters include, for example, the scaling reference point and the scaling factor.

[0154] The above rotation and scaling operations can be implemented using OpenCV's commonly used warpAffine and resize operations.

[0155] For the cropping operation, preset parameters include, for example, the cropping size and the proportion of hand features to be retained. Here, the target foreground image and the labeled mask can be randomly cropped according to the cropping size to obtain cropped images corresponding to the target foreground image and the labeled mask, respectively. Then, hand feature detection can be performed on the cropped image of the target foreground image to determine whether the ratio of the number (or area) of pixels corresponding to the hand features to the number (or area) of pixels in the cropped image is greater than a preset threshold (e.g., 30%). If so, the cropped image is retained; otherwise, the cropping result is discarded.

[0156] S142. Based on the pixel values ​​of each pixel in the updated annotation mask, determine the target region corresponding to the hand features in the updated annotation mask.

[0157] For example, based on the pixel values ​​of each pixel in the updated annotation mask, a contour detection algorithm can be used to find the connected regions in the updated annotation mask, i.e., the area where the palm is located, and the connected region can be directly used as the target region.

[0158] For example, based on the pixel values ​​of each pixel in the updated annotation mask, a contour detection algorithm can be used to find the connected regions in the updated annotation mask, i.e., the region where the palm is located. Then, to ensure that complete full-hand features are obtained, this connected region is expanded outwards to determine the target region. The preset expansion factor can be determined empirically, for example, the preset expansion factor can be in the range of 1.01 to 1.5. For example, using the geometric center of the connected region as a reference, the width and height are expanded by a factor of 1.35 respectively to obtain the target region.

[0159] S143. Extract a portion of the foreground image corresponding to the target region from the updated target foreground image, and extract a portion of the annotation mask corresponding to the target region from the updated annotation mask.

[0160] S144. For a background image in the background image set, a portion of the foreground image is pasted onto a predetermined position area in the background image to generate a composite image.

[0161] As can be seen from the implementation process of step S13 above, the target foreground image is an RGBA image. Therefore, the partial foreground image obtained based on the target foreground image is also an RGBA image, including four channels: R, G, B, and A.

[0162] Given a background image (RGB), a fourth solid color image with the same dimensions as the background image can be generated based on its size. In this fourth solid color image, all pixels have a value of (1,1,1), resulting in a pure white image. A portion of the foreground image is determined as the area to be pasted, and the positions of this area in the fourth solid color image correspond to those in the background image. The dimensions of the area to be pasted, the foreground image, and the partial annotation mask are the same. The fifth pixel in the fourth solid color image located within the area to be pasted corresponds one-to-one with the sixth pixel in the partial annotation mask. The pixel values ​​of each pixel in the partial annotation mask are (1,1,1) or (0,0,0). Based on the pixel values ​​of each pixel in the partial annotation mask, the sixth pixel with a value of (1,1,1) is determined. The pixel value of the fifth pixel in the fourth solid color image located within the area to be pasted, corresponding to this sixth pixel, is adjusted to (0,0,0), resulting in a transparent mask. In the transparent mask, each pixel has a value of (1,1,1) or (0,0,0). It should be noted that the sixth pixel corresponds to the hand feature, so the fifth pixel in the generated transparent mask corresponding to the sixth pixel is a transparent position. Pixel E in the transparent mask corresponds one-to-one with pixel F in the area to be fitted in the background image. The alpha channel value of pixel F can be determined to be 1 based on the pixel value of E (1,1,1); the alpha channel value of pixel F can be determined to be 0 based on the pixel value of E (0,0,0). The final result is a background image (R,G,B,A) with an alpha channel, where pixels with a value of (R,G,B,1) are non-transparent background positions, and pixels with a value of (R,G,B,0) are transparent foreground positions (i.e., the location of the hand feature).

[0163] Figure 7 This is a schematic diagram of a composite image formed by pasting a portion of the foreground image onto a background image, as provided in the embodiments of this disclosure. Figure 7 As shown, 71 represents a portion of the foreground image (a human hand), and 72 represents a background image with a transparency channel (a picture book image).

[0164] The four-channel foreground image is pasted onto the background image with an alpha channel. Specifically, the RGB pixel value of each pixel in the foreground image is multiplied by the A channel value of the corresponding pixel in the foreground image, and the RGB pixel value of each pixel in the background image is multiplied by the A channel value of the corresponding pixel in the background image to obtain the composite image.

[0165] In some embodiments, the above-described synthesized image is an example of pasting a partial foreground image onto a background image. Of course, for the same background image, multiple partial foreground images can also be pasted to enrich the training dataset.

[0166] Specifically, for different target foreground images and corresponding annotation masks, multiple partial foreground images and corresponding partial annotation masks are determined. These partial annotation masks are then applied to the areas to be applied in the background image to generate a composite image. The principle of the application process for each of the multiple partial foreground images and each of the corresponding partial annotation masks is the same as that for applying a single partial foreground image to the background image, as described above. Please refer to the specific application process described above; repeated details will not be repeated.

[0167] In this composite image, the overlap between the foreground images of each part is less than any value between 0.3 and 0.5. Here, the overlap can be data of the Intersection over Union (IOU).

[0168] Step S144 uses a single background image as an example to generate a composite image. The principle for generating a composite image is the same for any background image in the background image set; repeated parts will not be elaborated upon. By obtaining the composite images corresponding to each background image in the background image set, a training dataset can be obtained. The training dataset includes training data corresponding to multiple composite images.

[0169] S145. Generate training data based on the synthesized image and partial labeled masking.

[0170] In one possible implementation, the synthesized image and partially labeled mask can be directly used as training data.

[0171] In another possible implementation, to make the generated training samples more realistic, the synthetic images need to be further optimized. Figure 8 A schematic diagram of an exemplary synthetic image shading formed according to an embodiment of this disclosure, such as... Figure 8 As shown, 81 represents the outline shadow. The specific process of shadowing includes steps S145-1 to S145-3, where:

[0172] S145-1. Based on the synthesized image and partial labeled mask, process the contour of the synthesized image to obtain the first optimized image.

[0173] The specific steps in step S145-1 to obtain the first optimized image include S145-1-1 to S145-1-6, where:

[0174] S145-1-1. Expand the partial annotation mask to obtain the third intermediate mask.

[0175] For example, an image dilation algorithm can be used to dilate a portion of the labeled mask with a 3×3 kernel to obtain a third intermediate mask mask_dilate1. A 3×3 kernel is merely an example; other kernel sizes can also be chosen, and this disclosure does not limit the scope of the embodiments.

[0176] S145-1-2. Determine the first contour mark map based on the pixel values ​​of each pixel in the third intermediate mask and the pixel values ​​of each pixel in the partial annotation mask.

[0177] Given that the pixel values ​​mask_dilate1 of each pixel in the third intermediate mask and mask of each pixel in the partial annotation mask are known, the pixel values ​​of the corresponding pixels are subtracted to obtain the first contour marker image edge1, for example, edge1 = mask_dilate1 - mask. The size of the first contour marker image is the same as the size of the partial annotation mask and the size of the third intermediate mask. The visual effect of the first contour marker image is that there is only a white outline, and the rest of the position is black.

[0178] S145-1-3. Based on the synthesized image, the first contour mark image, the second solid color image with the same size as the synthesized image, and the first preset synthesis weight, a third optimized image is obtained.

[0179] The pixel value new_img_1 of the pixel in the third optimized image is calculated using the following formula:

[0180] new_img_1=image×(1-ratio1)×edge1+zero_image×(ratio1×edge1)

[0181] Wherein, image represents the pixel value of a pixel in the composite image; zero_image represents a second solid color image with the same size as the composite image, such as a pure black image; edge1 represents the first contour marker image; and ratio1 represents the first preset composite weight, such as 0.45.

[0182] The smaller the pixel value, the darker the display effect. After the optimization process of steps S145-1-1 to S145-1-3, the original pixel value of the non-outline part of the gesture remains unchanged. The original pixel value of the outline part of the gesture is weighted and averaged with the black pixels (0 value) to simulate the visual effect of "shadow" and improve the realism of the synthesized image.

[0183] S145-1-4. Expand the third intermediate mask to obtain the fourth intermediate mask.

[0184] For example, an image dilation algorithm can be used to dilate the third intermediate mask mask_dilate1 with a 3×3 kernel to obtain the fourth intermediate mask mask_dilate2. The 3×3 kernel is just an example; other kernel sizes can also be chosen, and this disclosure does not limit the scope of the embodiments.

[0185] S145-1-5. Determine the second contour mark map based on the pixel values ​​of each pixel in the fourth intermediate mask and the pixel values ​​of each pixel in the third intermediate mask.

[0186] Given the pixel values ​​mask_dilate2 and mask_dilate1 of each pixel in the fourth intermediate mask, the corresponding pixel values ​​are subtracted to obtain the second contour marker image edge2. For example, edge2 = mask_dilate2 - mask_dilate1. The size of the second contour marker image is the same as the size of the fourth and third intermediate masks. The visual effect of the second contour marker image is that it has only a white outline, and the rest of the area is black.

[0187] S145-1-6. Based on the third optimized image, the second contour marker image, the second solid color image, and the second preset synthesis weight, the first optimized image is obtained.

[0188] The pixel value of new_img_2 in the first optimized image is calculated using the following formula:

[0189] new_img_2=new_img_1×(1-ratio2)×edge2+zero_image×(ratio2×edge2)

[0190] Wherein, new_img_1 represents the pixel value of the pixel in the third optimized image; zero_image represents the second solid color image, such as a pure black image; edge2 represents the second contour marker image; and ratio2 represents the second preset synthesis weight, such as 0.3.

[0191] After optimization processes in steps S145-1-4 to S145-1-6, hand contour shadows are further added to the third optimized image to further improve the realism of the synthesized image.

[0192] Of course, other algorithms can also be selected to achieve contour shading, and this disclosure does not impose specific limitations on this embodiment.

[0193] S145-2. Based on the first optimized image and the partial annotation mask, the contour of the first optimized image is processed to obtain the second optimized image.

[0194] In one possible implementation, to make the generated training samples more realistic, the first optimized image can be further optimized. The specific process of naturalization includes steps S145-2-1 to S145-2-4, wherein:

[0195] S145-2-1. Using a contour detection algorithm, extract the contours from the first optimized image to determine the third contour marker image.

[0196] For example, the Canny operator can be used to extract hand contour information from the first optimized image (the Canny operator is a commonly used image processing operator in OpenCV). The size of the third contour marker map is the same as the size of the partial annotation mask. The visual effect of this third contour marker map is that there is only a white outline, and the rest of the area is black.

[0197] S145-2-2. Dilatate the third contour mark image to obtain the fourth contour mark image.

[0198] For example, an image dilation algorithm can be used to process the third contour marker image with a dilation kernel of 3×3 to obtain the fourth contour marker image edge3. The dilation kernel of 3×3 is only an example, and other sizes of dilation kernels can also be selected, which is not limited in this embodiment.

[0199] S145-2-3. Apply Gaussian blur to the first optimized image to obtain the fourth optimized image.

[0200] For example, a Gaussian blur algorithm is used to smooth the first optimized image image with a Gaussian kernel of 7×7 and a variance of 0.9 to obtain the fourth optimized image image_blur.

[0201] S145-2-4. Based on the first optimized image, the fourth contour marker image, the third solid color image with the same size as the first optimized image, and the third preset synthesis weight, the second optimized image is obtained.

[0202] The pixel value of new_img_3 in the second optimized image is calculated using the following formula:

[0203] new_img_3=new_img_2×(1-edge3)+image_blur×edge3

[0204] Where new_img_2 represents the pixel value of a pixel in the first optimized image; edge3 represents the fourth contour marker image; and image_blur represents the fourth optimized image.

[0205] After optimization processes in steps S145-2-1 to S145-2-4, the hand contour is further naturalized on the first optimized image to further improve the realism of the synthesized image.

[0206] Of course, other algorithms can also be selected to achieve contour naturalization, and this disclosure does not impose specific limitations on this embodiment.

[0207] S145-3, Use the second optimized image and partially labeled mask as training data.

[0208] In another possible implementation, the first optimized image and the partially labeled mask can be directly used as training data.

[0209] This disclosure provides an embodiment for automatically generating training data for a gesture segmentation network. The method involves capturing images of a human hand against a green screen background, automatically generating labels for the hand region. The hand region is then extracted from the captured image as foreground data and naturally integrated into various backgrounds, serving as training data for the gesture segmentation network. This embodiment can rapidly generate a large amount of labeled segmentation data, significantly reducing the cost of manual annotation. Furthermore, continuous image optimization throughout the processing ensures that the generated training data is realistic, reliable, and natural.

[0210] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0211] This disclosure also provides a training data generation device corresponding to the training data generation method. Since the principle of the device in this disclosure for solving the problem is similar to the training data generation method described above, the implementation of the device can refer to the implementation of the method, and repeated details will not be repeated.

[0212] Secondly, embodiments of this disclosure provide an apparatus for generating training data. Figure 9 This is a schematic diagram of a training data generation apparatus provided in an embodiment of the present disclosure, as shown below. Figure 9As shown, it includes an image acquisition module 91, a mask generation module 92, a foreground image generation module 93, and a training data determination module 94, wherein:

[0213] The image acquisition module 91 is configured to acquire a target image; the target image has a preset background and hand features;

[0214] The mask generation module 92 is configured to generate a labeled mask for the hand features based on the pixel values ​​of each pixel in the target image and the target pixel values ​​of the preset background stored in advance.

[0215] The foreground image generation module 93 is configured to determine the channel value of the transparency channel in the target image based on the pixel value of each pixel in the annotation mask, and to use the target image with the transparency channel as the target foreground image;

[0216] The training data determination module 94 is configured to determine training data based on a pre-stored set of background images, the target foreground image, and the labeled mask, for use in training a gesture segmentation network model.

[0217] This embodiment of the invention presets a background (e.g., a green screen) for gesture acquisition. By using the pre-known target pixel values ​​of the preset background and comparing them with the pixel values ​​of each pixel in the target image, a labeled mask for hand features can be automatically generated. Compared to the manual labeling method in the prior art, this embodiment significantly improves the efficiency of generating labeled masks and reduces labeling costs. Simultaneously, the generated labeled mask can not only serve as a necessary condition for generating training data (or as part of the training data), but also directly determine the alpha channel data of the target image using the generated labeled mask. This saves the process of repeatedly detecting and generating transparent layers in the target image, reducing the overall processing cycle for generating training data and thus improving the efficiency of training data generation.

[0218] In some embodiments, the preset background is a solid color background; the image acquisition module 91 is specifically configured to acquire a hand image in an environment with the solid color background to obtain an original image; perform HSV color model conversion on the original image to obtain an HSV image; and perform Gaussian blur on the HSV image to obtain the target image.

[0219] In some embodiments, the mask generation module 92 is specifically configured to generate a first solid color image with the same size as the target image based on the size information of the target image; the pixel value of each pixel in the first solid color image is a first preset value; based on the pixel values ​​of each pixel in the target image and the target pixel values ​​of the preset background stored in advance, a first pixel in the target image that is different from the target pixel value is determined; the pixel value of the second pixel in the first solid color image corresponding to the first pixel is adjusted to a second preset value to obtain a first mask; and the first mask is processed using a preset algorithm to obtain the labeled mask.

[0220] In some embodiments, the mask generation module 92 processes the first mask using a preset algorithm to obtain the annotation mask, specifically including: performing dilation processing on the first mask to obtain a first intermediate mask; performing erosion processing on the first intermediate mask to obtain a second intermediate mask; using a contour detection algorithm to perform contour recognition on the second intermediate mask to determine the maximum region of contour connectivity; and adjusting the pixel values ​​of other regions outside the maximum region corresponding to the second intermediate mask to the first preset value to obtain the annotation mask.

[0221] In some embodiments, the mask generation module 92 processes the first mask using a preset algorithm to obtain the labeled mask, specifically including: when it is determined that the hand features in the target image include arm features, processing the first mask using a preset algorithm to obtain a second mask; extracting key points of the palm in the target image using a pre-trained neural network model, and determining a first detection box based on the key points of the palm; expanding the first detection box using a preset expansion factor to obtain a second detection box; and obtaining the labeled mask based on the second detection box and the second mask.

[0222] In some embodiments, the mask generation module 92 obtains the annotation mask based on the second detection box and the second mask, specifically including: determining a circular region with the center position as the center and the distance from the center to a preset key point located in the palm as the radius based on the center position of the second detection box; adjusting the pixel values ​​of the pixels outside the circular region in the second mask to the first preset value to obtain the annotation mask.

[0223] In some embodiments, the training data determination module 94 is specifically configured to perform graphic transformation on the target foreground image and the corresponding annotation mask using the same preset parameters to obtain an updated target foreground image and an updated annotation mask; determine the target region corresponding to the hand feature in the updated annotation mask based on the pixel values ​​of each pixel in the updated annotation mask; extract a portion of the foreground image corresponding to the target region from the updated target foreground image, and extract a portion of the annotation mask corresponding to the target region from the updated annotation mask; for a background image in the background image set, attach the portion of the foreground image to the area to be attached in the background image to generate a composite image; and generate training data based on the composite image and the portion of the annotation mask.

[0224] In some embodiments, the foreground image generation module 93 is further configured to determine a plurality of partial foreground images and partial annotation masks corresponding to each of the different target foreground images and the annotation masks corresponding to the target foreground images;

[0225] The training data determination module 94 describes attaching the partial foreground image to a predetermined position area in the background image to generate the composite image, specifically including: attaching multiple partial labeled masks to the area to be attached in the background image to generate the composite image; the overlap of each partial foreground image in the composite image is less than any value between 0.3 and 0.5.

[0226] In some embodiments, the training data determination module 94 generates training data based on the synthesized image and the partial annotation mask, specifically including: processing the contour of the synthesized image based on the synthesized image and the partial annotation mask to obtain a first optimized image; processing the contour of the first optimized image based on the first optimized image and the partial annotation mask to obtain a second optimized image; and using the second optimized image and the partial annotation mask as the training data.

[0227] In some embodiments, the training data determination module 94 processes the contour of the synthesized image based on the synthesized image and the partial annotation mask to obtain a first optimized image, specifically including: dilating the partial annotation mask to obtain a third intermediate mask; determining a first contour marker map based on the pixel values ​​of each pixel in the third intermediate mask and the pixel values ​​of each pixel in the partial annotation mask; obtaining a third optimized image based on the synthesized image, the first contour marker map, a second solid color image of the same size as the synthesized image, and a first preset synthesis weight; dilating the third intermediate mask to obtain a fourth intermediate mask; determining a second contour marker map based on the pixel values ​​of each pixel in the fourth intermediate mask and the pixel values ​​of each pixel in the third intermediate mask; and obtaining the first optimized image based on the third optimized image, the second contour marker map, the second solid color image, and the second preset synthesis weight.

[0228] In some embodiments, the training data determination module 94 performs contour processing on the first optimized image based on the first optimized image and the partial annotation mask to obtain a second optimized image, specifically including: extracting contours from the first optimized image using a contour detection algorithm to determine a third contour marker map; performing dilation processing on the third contour marker map to obtain a fourth contour marker map; performing Gaussian blur on the first optimized image to obtain a fourth optimized image; and obtaining the second optimized image based on the first optimized image, the fourth contour marker map, and the fourth optimized image.

[0229] Thirdly, embodiments of this disclosure provide a computer device, Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure, such as... Figure 10 As shown, the computer device includes: one or more processors 101, a memory 102, and one or more I / O interfaces 103. The memory 102 stores one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement a training data generation method as described in any of the above embodiments; the one or more I / O interfaces 103 are connected between the processor and the memory, configured to enable information interaction between the processor and the memory.

[0230] The processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read / write interface) 103 is connected between the processor 101 and the memory 102, enabling information exchange between the processor 101 and the memory 102, including but not limited to a data bus (Bus).

[0231] In some embodiments, the processor 101, memory 102, and I / O interface 103 are interconnected via bus 104, and thus connected to other components of the computing device.

[0232] Fourthly, embodiments of this disclosure also provide a computer non-transient readable storage medium. This computer non-transient readable storage medium stores a computer program, wherein when executed by a processor, the program implements the steps in any of the training data generation methods described in the above embodiments.

[0233] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a machine-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs the functions defined above in the system of this disclosure.

[0234] It should be noted that the computer-readable non-transient readable medium disclosed herein may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. Computer-readable storage media may be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any non-transient readable computer storage medium other than a computer-readable storage medium, which can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the non-transient readable computer storage medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0235] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two adjacent blocks may actually represent substantially parallel execution, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0236] It is understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of this disclosure, and this disclosure is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and substance of this disclosure, and these modifications and improvements are also considered to be within the scope of protection of this disclosure.

Claims

1. A method for generating training data, wherein, include: Acquire the target image; The target image has a preset background and hand features; Based on the size information of the target image, a first solid color image with the same size as the target image is generated; the pixel value of each pixel in the first solid color image is a first preset value. Based on the pixel values ​​of each pixel in the target image and the target pixel values ​​of the preset background stored in advance, a first pixel in the target image that is different from the target pixel value is determined; The pixel value of the second pixel point corresponding to the first pixel point in the first solid color image is adjusted to a second preset value to obtain the first mask; The first mask is processed using a preset algorithm to obtain the labeled mask; Based on the pixel values ​​of each pixel in the labeled mask, the channel value of the transparency channel in the target image is determined, and the target image with the transparency channel is used as the target foreground image; Training data is determined based on a pre-stored set of background images, the target foreground image, and the labeled mask, for use in training a gesture segmentation network model.

2. The method for generating training data according to claim 1, wherein, The preset background is a solid color background; The steps for determining the target image include: Hand images are captured in an environment with the aforementioned solid color background to obtain the original image; The original image is converted to the HSV color model to obtain an HSV image; The target image is obtained by applying Gaussian blur to the HSV image.

3. The method for generating training data according to claim 1, wherein processing the first mask using a preset algorithm to obtain the labeled mask includes: The first mask is expanded to obtain a first intermediate mask; The first intermediate mask is etched to obtain the second intermediate mask; Using a contour detection algorithm, contour recognition is performed on the second intermediate mask to determine the largest region where the contours are connected; The pixel values ​​of other regions outside the maximum region corresponding to the second intermediate mask are adjusted to the first preset value to obtain the annotation mask.

4. The method for generating training data according to claim 1, wherein processing the first mask using a preset algorithm to obtain the labeled mask includes: If it is determined that the hand features in the target image include arm features, the first mask is processed using a preset algorithm to obtain the second mask; The key points of the palm in the target image are extracted using a pre-trained neural network model, and a first detection box is determined based on the key points of the palm. The first detection frame is expanded using a preset expansion factor to obtain a second detection frame; The annotation mask is obtained based on the second detection box and the second mask.

5. The method for generating training data according to claim 4, wherein obtaining the annotation mask based on the second detection box and the second mask comprises: Based on the center position of the second detection frame, a circular area is determined with the center position as the center and the distance from the center to a preset key point in the palm as the radius; The pixel values ​​of the pixels outside the circular area in the second mask are adjusted to the first preset value to obtain the annotation mask.

6. The method for generating training data according to claim 1, wherein determining the training data based on a pre-stored set of background images, the target foreground image, and the labeled mask comprises: Using the same preset parameters, perform graphic transformation on the target foreground image and the corresponding annotation mask to obtain an updated target foreground image and an updated annotation mask; Based on the pixel values ​​of each pixel in the updated annotation mask, determine the target region corresponding to the hand feature in the updated annotation mask; Extract a portion of the foreground image corresponding to the target region from the updated target foreground image, and extract a portion of the annotation mask corresponding to the target region from the updated annotation mask; For a background image in the set of background images, the foreground image is pasted onto the area to be pasted in the background image to generate a composite image. Training data is generated based on the synthesized image and the partially labeled mask.

7. The method for generating training data according to claim 6, further comprising: For different target foreground images and the annotation mask corresponding to the target foreground images, multiple partial foreground images and partial annotation masks corresponding to each of the partial foreground images are determined respectively; The step of pasting the foreground image onto a predetermined location area in the background image to generate the composite image includes: The composite image is generated by attaching multiple partial labeled masks to the areas to be attached in the background image. The overlap of each of the foreground images in the synthesized image is less than any value between 0.3 and 0.

5.

8. The method for generating training data according to claim 6 or 7, wherein generating training data based on the synthesized image and the partially labeled mask comprises: Based on the synthesized image and the partial labeled mask, the contour of the synthesized image is processed to obtain a first optimized image; Based on the first optimized image and the partial labeled mask, the contour of the first optimized image is processed to obtain the second optimized image; The second optimized image and the partially labeled mask are used as the training data.

9. The method for generating training data according to claim 8, wherein processing the contour of the synthesized image based on the synthesized image and the partially labeled mask to obtain a first optimized image includes: The partial label mask is expanded to obtain a third intermediate mask; The first contour mark map is determined based on the pixel values ​​of each pixel in the third intermediate mask and the pixel values ​​of each pixel in the partial annotation mask; A third optimized image is obtained based on the synthesized image, the first contour mark image, a second solid color image of the same size as the synthesized image, and a first preset synthesis weight; The third intermediate mask is expanded to obtain the fourth intermediate mask; The second contour mark map is determined based on the pixel values ​​of each pixel in the fourth intermediate mask and the pixel values ​​of each pixel in the third intermediate mask; The first optimized image is obtained based on the third optimized image, the second contour marker image, the second solid color image, and the second preset synthesis weight.

10. The method for generating training data according to claim 8, wherein the step of contour processing of the first optimized image based on the first optimized image and the partial labeled mask to obtain a second optimized image comprises: Using a contour detection algorithm, contour extraction is performed on the first optimized image to determine a third contour marker image; The third contour mark image is dilated to obtain the fourth contour mark image; Gaussian blur is applied to the first optimized image to obtain the fourth optimized image; A second optimized image is obtained based on the first optimized image, the fourth contour marker image, and the fourth optimized image.

11. An apparatus for generating training data, wherein, It includes an image acquisition module, a mask generation module, a foreground image generation module, and a training data determination module; The image acquisition module is configured to acquire a target image; the target image has a preset background and hand features; The mask generation module is configured to generate a first solid color image with the same size as the target image based on the size information of the target image; the pixel value of each pixel in the first solid color image is a first preset value; based on the pixel values ​​of each pixel in the target image and the target pixel values ​​of the preset background stored in advance, a first pixel in the target image that is different from the target pixel value is determined; the pixel value of a second pixel in the first solid color image corresponding to the first pixel is adjusted to a second preset value to obtain a first mask; and the first mask is processed using a preset algorithm to obtain a labeled mask. The foreground image generation module is configured to determine the channel value of the transparency channel in the target image based on the pixel value of each pixel in the labeled mask, and to use the target image with the transparency channel as the target foreground image; The training data determination module is configured to determine training data based on a pre-stored set of background images, the target foreground image, and the labeled mask, for use in training a gesture segmentation network model.

12. A computer device, wherein, include: The computer device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the training data generation method as described in any one of claims 1 to 10.

13. A computer-defined non-transient readable storage medium, wherein, The computer non-transient readable storage medium stores a computer program that, when executed by a processor, performs the steps of the method for generating training data as described in any one of claims 1 to 10.

Images