Sample generation method of attitude image and electronic equipment

Abstract

The invention provides a sample generation method of a posture image and electronic equipment, relates to the technical field of computers, and is used for solving the problem of insufficient sample data. The method comprises the following steps: creating a posture action database; the posture action database comprises multiple pieces of posture action text data; a deep learning model is utilized to process the multiple pieces of posture action text data, multiple posture action images are generated, and one piece of posture action text data corresponds to one posture action image; performing feature extraction on the plurality of posture action images to obtain a plurality of posture action features corresponding to the plurality of posture action images; generating a plurality of reconstructed images by using the plurality of attitude motion features and the attitude motion sample data set; the posture action sample data set comprises a plurality of posture action sample images; and processing the plurality of reconstructed images to obtain a plurality of posture action sample images.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a sample generation method of a posture image and an electronic device. Background Art

[0002] With the rapid development of computer technology, artificial intelligence models have shown great potential in many fields. Generally speaking, the performance of artificial intelligence models is closely related to sample training data.

[0003] However, in certain scenarios (for example, scenarios related to human body posture and movement), the cost of data collection is high and the number of samples is relatively scarce; it is extremely difficult to obtain sufficient sample training data. The lack of sample data will directly affect the training effect and generalization ability of the artificial intelligence model. Therefore, in certain scenarios, how to obtain a large amount of sample data is an urgent problem to be solved. Summary of the invention

[0004] In order to solve the above technical problems, the present disclosure provides a sample generation method and electronic device for a posture image, which can solve the problem of insufficient sample data.

[0005] The technical solution of the present disclosure is as follows: In a first aspect, the present disclosure provides a sample generation method for a posture image, which is applied to an electronic device, and the method includes: creating a posture action database; the posture action database includes multiple posture action text data; using a deep learning model, processing the multiple posture action text data to generate multiple posture action images, wherein one posture action text data corresponds to one posture action image; extracting features from the multiple posture action images to obtain multiple posture action features corresponding to the multiple posture action images; using multiple posture action features and a posture action sample data set to generate multiple reconstructed images; the posture action sample data set includes multiple posture action sample images; processing the multiple reconstructed images to obtain multiple posture action sample images.

[0006] In combination with the first aspect, in another possible implementation method, creating a posture and action database includes: obtaining multiple text data; using the text data to generate human body posture and action; each text data in the multiple text data corresponds to a different human body posture and action; and obtaining a posture and action database based on the multiple text data.

[0007] In combination with the first aspect, in another possible implementation method, a deep learning model is used to process multiple posture action text data to generate multiple posture action images, including: based on multiple posture action text data, obtaining a text feature vector corresponding to each posture action text data in the multiple posture action text data; inputting the text feature vector corresponding to each posture action text data into the deep learning model to obtain multiple posture action images; the deep learning model has the ability to generate posture action images that match each text feature vector.

[0008] In combination with the first aspect, in another possible implementation method, multiple posture action features and a posture action sample data set are used to generate multiple reconstructed images, including: performing feature extraction on multiple posture action sample images in the posture action sample data set to obtain multiple posture action sample features; replacing multiple posture action sample features with multiple posture action features, and generating multiple reconstructed images.

[0009] In combination with the first aspect, in another possible implementation, multiple reconstructed images are processed to obtain multiple posture and action sample images, including: standardizing the multiple reconstructed images to obtain multiple first reconstructed images; performing quality improvement processing on the multiple first reconstructed images to obtain multiple posture and action sample images, and the quality improvement processing includes at least one of adjusting clarity, adding noise, and color space transformation.

[0010] In combination with the first aspect, in another possible implementation method, based on multiple posture action text data, a text feature vector corresponding to each posture action text data in the multiple posture action text data is obtained, including: extracting key information from each posture action text data to obtain each target posture action text data; encoding each target posture action text data to obtain a text feature vector corresponding to each posture action text data.

[0011] In combination with the first aspect, in another possible implementation manner, each target posture action text data includes at least one of an action type, an action subject, and an action amplitude.

[0012] In a second aspect, the present disclosure provides a sample generation device for a gesture image, which can be applied to an electronic device to implement the method in the first aspect. The functions of the sample generation device for a gesture image can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, such as a creation module, a generation module, and a processing module.

[0013] Among them, the creation module is configured to create a gesture action database; the gesture action database includes a plurality of gesture action text data.

[0014] The generation module is configured to use a deep learning model to process multiple posture action text data to generate multiple posture action images, wherein one posture action text data corresponds to one posture action image; and to use multiple posture action features and posture action sample data sets to generate multiple reconstructed images, wherein the posture action sample data sets include multiple posture action sample images.

[0015] The processing module is configured to extract features from a plurality of posture action images to obtain a plurality of posture action features corresponding to the plurality of posture action images; and to process a plurality of reconstructed images to obtain a plurality of posture action sample images.

[0016] In combination with the second aspect, in a possible implementation method, the creation module is also configured to obtain multiple text data; the text data is used to generate human body posture movements; each text data in the multiple text data corresponds to a different human body posture movement; and a posture movement database is obtained based on the multiple text data.

[0017] In combination with the second aspect, in a possible implementation method, the generation module is also configured to obtain a text feature vector corresponding to each posture action text data in the multiple posture action text data based on the multiple posture action text data; and input the text feature vector corresponding to each posture action text data into the deep learning model to obtain multiple posture action images; the deep learning model has the ability to generate posture action images that match each text feature vector.

[0018] In combination with the second aspect, in a possible implementation method, the generation module is also configured to perform feature extraction on multiple posture action sample images in the posture action sample data set to obtain multiple posture action sample features; and use multiple posture action features to replace multiple posture action sample features and generate multiple reconstructed images.

[0019] In combination with the second aspect, in a possible implementation, the processing module is also configured to perform standardization processing on the multiple reconstructed images to obtain multiple first reconstructed images; and perform quality improvement processing on the multiple first reconstructed images to obtain multiple posture action sample images, and the quality improvement processing includes at least one of adjusting clarity, adding noise, and color space transformation.

[0020] In combination with the second aspect, in a possible implementation method, the generation module is also configured to extract key information from each posture action text data to obtain each target posture action text data; and encode each target posture action text data to obtain a text feature vector corresponding to each posture action text data.

[0021] In conjunction with the second aspect, in a possible implementation manner, each target posture action text data includes at least one of an action type, an action subject, and an action amplitude.

[0022] In a third aspect, the present disclosure provides an electronic device, comprising: a memory, a display screen, and one or more processors; the memory, the display screen, and the processor are coupled. The memory is used to store computer program codes, and the computer program codes include computer instructions; when the electronic device is running, the processor is used to execute one or more computer instructions stored in the memory, so that the electronic device executes the sample generation method of the gesture image as described in any one of the first aspects above.

[0023] In a fourth aspect, the present disclosure provides a computer storage medium, comprising computer instructions, which, when executed on an electronic device, enable the electronic device to execute the sample generation method of a gesture image as described in any one of the first aspects.

[0024] In a fifth aspect, the present disclosure provides a computer program product. When the computer program product is executed on an electronic device, the electronic device executes the sample generation method of the gesture image as described in any one of the first aspects.

[0025] In a sixth aspect, a device (for example, the device may be a chip system) is provided, the device including a processor, which is used to support a first device to implement the functions involved in the first aspect. In a possible design, the device also includes a memory, which is used to store program instructions and data necessary for the first device. When the device is a chip system, it may be composed of a chip, or may include a chip and other discrete devices.

[0026] It should be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0027] In the present disclosure, the above names do not limit the devices or functional modules themselves. In actual implementation, these devices or functional modules may appear with other names. As long as the functions of each device or functional module are similar to those of the present disclosure, they fall within the scope of the claims of the present disclosure and their equivalent technologies.

[0028] These and other aspects of the present disclosure will become more apparent from the following description.

[0029] The technical solution provided by the present disclosure has the following advantages over the prior art: by creating a gesture action database, a large amount of gesture action text data can be collected. Then, by processing a plurality of gesture action text data, a plurality of gesture action images can be obtained, and then a plurality of gesture action sample images can be obtained using the plurality of gesture action images. In other words, the present disclosure can obtain a large number of gesture action sample images through a large amount of gesture action text data. In this way, not only can the number of samples be significantly increased, but also the problems of insufficient sample data and poor sample quality can be effectively solved, providing strong data support for the application of artificial intelligence in specific fields. Afterwards, a plurality of gesture action sample images can also be used for model training to improve the training effect and generalization ability of the model. And compared with the related art, the present disclosure can significantly reduce the cost of manpower and material resources by professional photographers squatting on the spot to capture gesture action images. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0032] Figure 1 One of the flowcharts of a method for generating a sample of a posture image provided in an embodiment of the present disclosure.

[0033] Figure 2 The second flowchart of a method for generating a sample of a posture image provided in an embodiment of the present disclosure.

[0034] Figure 3 The third flowchart of a method for generating a sample of a posture image provided in an embodiment of the present disclosure.

[0035] Figure 4 The fourth flowchart of a method for generating a sample of a posture image provided in an embodiment of the present disclosure.

[0036] Figure 5 The fifth flowchart of a method for generating a sample of a posture image provided in an embodiment of the present disclosure.

[0037] Figure 6 The sixth flowchart of a method for generating a sample of a posture image provided in an embodiment of the present disclosure.

[0038] Figure 7A schematic diagram of the structure of a gesture image sample generation device provided in an embodiment of the present disclosure.

[0039] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0040] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0041] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0042] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0043] In today's era, computer technology is developing at an unprecedented rate, and artificial intelligence models have shown remarkable potential. Generally speaking, the performance of artificial intelligence models depends largely on the quality and quantity of sample training data.

[0044] However, in some specific scenarios, such as those closely related to human posture and movement, the collection of sample training data is extremely difficult, the data collection cost is high, and the sample training data that can be obtained is also small. This has a direct negative impact on the training effect of the artificial intelligence model, making it difficult to achieve the ideal training accuracy and depth, and also greatly weakens the generalization ability of the model, resulting in poor processing efficiency of the model when facing new data. Therefore, in these specific scenarios, how to efficiently obtain a large amount of sample data has become a core issue that needs to be solved urgently.

[0045] In order to solve the above-mentioned problem of difficulty in obtaining sample training data, the relevant technology adopts data enhancement method to expand sample training data. The specific data enhancement method is: performing operations such as scaling, rotating, and enlarging existing sample training data (for example, the sample training data is image data) to generate new sample training data, thereby enriching the data set. In this way, although the amount of sample training data can be increased, it is difficult to effectively increase the diversity of sample training data, and it still cannot meet the actual sample needs.

[0046] To this end, in view of the above problems, the embodiment of the present disclosure provides a sample generation method of a posture image, by creating a posture action database, a large amount of posture action text data can be collected. Then, by processing a plurality of posture action text data, a plurality of posture action images can be obtained, and then a plurality of posture action sample images can be obtained using the plurality of posture action images. In other words, the present disclosure can obtain a large number of posture action sample images through a large amount of posture action text data. In this way, not only can the number of samples be significantly increased, but also the problems of insufficient data and poor sample quality can be effectively solved, providing strong data support for the application of artificial intelligence in specific fields. Afterwards, a plurality of posture action sample images can also be used for model training to improve the training effect and generalization ability of the model. And compared with the related art, the present disclosure can significantly reduce the cost of manpower and material resources by professional photographers squatting on the spot to capture posture action images.

[0047] The following describes a sample generation method of a posture image provided by an embodiment of the present disclosure. A person skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solution provided by an embodiment of the present disclosure is also applicable to similar technical problems.

[0048] The sample generation method of the gesture image provided by the present disclosure can be applied to electronic devices.

[0049] In some embodiments, the electronic device may be a server, a terminal, or other electronic device for generating gesture image samples, which is not limited in the present disclosure.

[0050] The server may be a single server, or a server cluster consisting of multiple servers. In some implementations, the server cluster may also be a distributed cluster. The present disclosure does not limit the specific implementation of the server.

[0051] The terminal may be a mobile phone, tablet computer, desktop, laptop, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, cellular phone, personal digital assistant (PDA), augmented reality (AR)\virtual reality (VR) device, etc., which can install and use content community applications (such as Kuaishou), and the present disclosure does not impose any special restrictions on the specific form of the electronic device. It can interact with the user through one or more methods such as keyboard, touchpad, touch screen, remote control, voice interaction or handwriting device.

[0052] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0053] refer to Figure 1 , a flow chart of a method for generating a sample of a posture image provided by an embodiment of the present disclosure. Figure 1 As shown, the method may include steps 101 to 105.

[0054] Step 101: Create a gesture action database.

[0055] The gesture action database includes a plurality of gesture action text data.

[0056] Exemplarily, the gesture text data can be used to describe the gesture of a person. It can also be used to describe the gesture of other objects, such as auxiliary robots, animals, etc. The present disclosure does not limit this, and the specific scene of the application of the gesture shall prevail.

[0057] In some examples, the posture and action text data can come from literature and research reports related to posture and action. For example, the posture and action text data can include content about human body structure, skeletal structure, muscle distribution, and range of motion of joints. It can also include physical rules related to human body movements (such as the influence of gravity, inertia, and friction on changes in human body posture) and biomechanical principles (such as the mechanism by which muscle contraction generates force and drives skeletal joint movement). These posture and action text data can provide accurate and powerful theoretical guidance for the subsequent generation of real and logical human posture and action, ensuring that the constructed posture and action database is highly scientific and practical.

[0058] Optional, such as Figure 2 As shown, step 101 of creating a gesture action database includes: Step 201: Acquire multiple text data.

[0059] Among them, text data is used to generate human posture movements.

[0060] In some examples, each text data in the plurality of text data corresponds to a different human body posture action.

[0061] In combination with step 101, it can be known that the multiple text data can be derived from literature and research reports related to posture and movement. Exemplarily, the multiple text data may include first text data and second text data. For example, the human body posture and movement corresponding to the first text data is a standing movement. For example, the first text data is "When standing, both feet are firmly planted on the ground, the leg bones are supported in place, the knees are straight to ensure stability, the upper body is balanced and straight, and the arms are relaxed and hanging down." The human body posture and movement corresponding to the second text data is a running movement. For example, the second text data is "the footsteps are stepped on the ground alternately and quickly, the step frequency is about 160-180 times per minute, the stride is between 0.8 and 1.2 meters depending on the height, the body leans forward about 5-10 degrees when running, the sole of the foot touches the ground first, the outside of the heel transitions to the inside of the sole (mostly in middle and long distance running) or the forefoot touches the ground (common in short distance running), the knee joint is flexed about 130-150 degrees when the swinging leg swings forward, and the supporting leg is straightened to assist in propulsion when extending, the arms swing naturally, the elbows are bent about 90 degrees, swing forward to chest height and back to waist, and coordinate with the leg movements to maintain balance and rhythm". Of course, the text data can also correspond to other human postures and movements, which will not be repeated here.

[0062] In other examples, two or more text data may have the same human body gestures in multiple text data. The present disclosure does not limit whether two or more text data have the same human body gestures in multiple text data, as long as it is in accordance with actual use.

[0063] Step 202: Obtain a gesture action database based on multiple text data.

[0064] After the multiple text data are acquired, the multiple text data may be stored according to a data storage rule of a preset database, thereby obtaining a gesture action database.

[0065] In some combing, the gesture action database may be a relational database, such as MySQL, PostgreSQL, etc. For example, the gesture action database includes human body parts and actions corresponding to the human body parts. The human body parts may include: bones, joints, muscles, etc. The actions corresponding to the human body parts may include: walking, jumping, etc.

[0066] It can also be a non-relational database, for example, a document database such as MongoDB. Non-relational databases can be applied to scenarios where text data is irregular and text data is complexly nested. The present disclosure does not limit the gesture action database, which is subject to actual application.

[0067] Through multiple text data, the content that may have been scattered in different materials can be unified into a posture and motion database. In this way, data integration can be achieved, the inconvenience of obtaining data caused by data fragmentation can be avoided, and human posture and motion data can be saved in a more systematic and orderly manner.

[0068] Step 102: Use a deep learning model to process multiple posture action text data to generate multiple posture action images.

[0069] Among them, one gesture action text data corresponds to one gesture action image.

[0070] In some examples, multiple gesture action images may be the same or different.

[0071] When the deep learning model training is completed, multiple posture action text data can be input into the trained deep learning model in sequence to output the posture action image corresponding to each posture action text.

[0072] Optional, such as Figure 3 As shown, step 102 uses a deep learning model to process multiple posture action text data to generate multiple posture action images including: Step 301: Based on a plurality of gesture action text data, a text feature vector corresponding to each gesture action text data in the plurality of gesture action text data is obtained.

[0073] Optional, such as Figure 4 As shown, step 301 obtains a text feature vector corresponding to each gesture action text data in the plurality of gesture action text data based on the plurality of gesture action text data, including: Step 401: extract key information from each posture action text data to obtain each target posture action text data.

[0074] After obtaining a plurality of posture action text data, entity recognition, keyword extraction algorithm, semantic understanding, etc. in natural language processing technology can be used to extract key information from each posture action text data in the plurality of posture action text data.

[0075] In some examples, each target gesture action text data includes at least one of an action type, an action subject, and an action amplitude.

[0076] The action type is used to characterize the category to which the posture action belongs. For example, the posture action text data is "When doing physical exercise, first hold the horizontal bar with both hands, then use the strength of your arms and shoulders to pull your body up until your chin is above the horizontal bar", and the action type corresponding to this posture action text data is pull-ups. This helps to carry out targeted processing of different types of action areas in the future.

[0077] The action subject is used to characterize the body parts or body systems of the gesture action. For example, the text data of the gesture action is "When performing, the ballet dancer completes a series of elegant toe rotations on the ground through the flexible rotation of the ankle and the control of the leg muscles." The action subject corresponding to this gesture action text data is "ankle and leg". The key body parts that produce the action can be described in detail through the action subject, which is of great significance for analyzing the source of force, joint movement, and related biomechanical principles of the action, and it is also convenient for comparing the differences between actions led by different subjects.

[0078] The amplitude of movement is used to characterize the extent of the gesture movement, and generally involves specific quantitative descriptions such as angle, distance, and range. For example, the text data of the gesture movement is "When playing tennis, the player swings the racket, and the arm stretches from the starting position of the backswing to the forward swing to hit the ball, and the elbow joint angle stretches from about 160 degrees to nearly 180 degrees." The amplitude of movement information corresponding to the gesture movement text data is "the elbow joint angle stretches from about 160 degrees to 180 degrees." Understanding the amplitude of movement helps to accurately grasp the scale and intensity of the movement.

[0079] Of course, the target posture action text data may also include other action-related content, which is not limited in the present disclosure.

[0080] In some examples, the process of extracting key information from each posture action text data may include: the electronic device may pre-set some keyword libraries related to the action type, action subject, and action amplitude (such as "running", "jumping", "bending", etc. used to characterize the action type; the name of the body part used to characterize the action subject; quantifiers and descriptive words related to the angle and length used to characterize the action amplitude, etc.), and filter out the key information in each posture action text data (i.e., the target posture action text data) through text matching.

[0081] In other examples, the process of extracting key information from each gesture action text data may include: inputting each gesture action text data into a target classification model, and the target classification model outputs each target gesture action text data. The target classification model has the ability to accurately extract key contents such as action type, action subject, and action amplitude from the gesture action text data. The target classification model can be obtained by training sample data using a large amount of labeled key information.

[0082] Generally, gesture action text data often contains a large amount of detailed description content, which may be complex and contain a lot of non-critical information. By extracting key information, more concise and focused target gesture action text data can be obtained, which is convenient for subsequent further processing and utilization.

[0083] Step 402: Encode the text data of each target posture action to obtain a text feature vector corresponding to each posture action text data.

[0084] In some examples, encoding each target gesture text data to obtain a text feature vector corresponding to each gesture text data may be performed by converting words in each target gesture text data into corresponding word vectors, and then concatenating them to obtain a text feature vector corresponding to each target gesture text data. Converting each target gesture text data into a text feature vector by encoding can facilitate subsequent further processing.

[0085] Step 302: Input the text feature vector corresponding to each posture action text data into the deep learning model to obtain multiple posture action images.

[0086] Among them, the deep learning model has the ability to generate posture and action images that match the feature vectors of each text.

[0087] In some examples, the deep learning model may be trained by sample data, wherein the sample data includes gesture action text sample data and gesture action sample images matching the gesture action text sample data.

[0088] The process of inputting the text feature vector corresponding to each posture action text data into a deep learning model to obtain multiple posture action images can be: inputting the text feature vector corresponding to each posture action text data into a deep learning model, and the deep learning model outputs a posture action image that matches the text feature vector corresponding to the posture action text data, thereby obtaining multiple posture action images.

[0089] Step 103: extract features from the plurality of posture action images to obtain a plurality of posture action features corresponding to the plurality of posture action images.

[0090] After obtaining a plurality of posture action images, feature extraction may be performed on the plurality of posture action images to obtain a plurality of posture action features corresponding to the plurality of posture action images.

[0091] In some examples, the process of extracting features from multiple gesture action images to obtain multiple gesture action features corresponding to the multiple gesture action images may include: extracting features from the multiple gesture action images using a trained convolutional neural network to obtain multiple gesture action features corresponding to the multiple gesture action images. Exemplarily, the trained convolutional neural network may be a classic VGG, ResNet, Inception, or other trained CNN model.

[0092] In other examples, the process of extracting features from multiple posture action images to obtain multiple posture action features corresponding to the multiple posture action images may include: using a human posture estimation model to extract features from multiple posture action images to obtain multiple posture action features corresponding to the multiple posture action images.

[0093] Exemplarily, the human pose estimation model can be an OpenPose model. This model focuses specifically on human poses. It can not only locate the joints of the human body in the image (such as key joints such as shoulders, elbows, wrists, hips, knees, ankles, etc.), but also construct a skeleton diagram of the human pose through the lines and angles between the joints. In this way, it can provide more targeted feature data for subsequent processing.

[0094] After obtaining multiple gesture and action features, a gesture and action feature database can be generated using the multiple gesture and action features. In this way, when there is a need to generate gesture and action image samples in the future, the gesture and action feature database can be directly used for processing without having to perform the aforementioned process again, thereby improving data processing efficiency.

[0095] Step 104: Generate multiple reconstructed images using multiple posture and motion features and posture and motion sample data sets.

[0096] The posture action sample data set includes a plurality of posture action sample images.

[0097] After acquiring multiple posture and motion features, multiple posture and motion features and posture and motion sample data sets may be used to generate multiple reconstructed images.

[0098] In some examples, the gesture action sample dataset may include a plurality of gesture action sample images.

[0099] Optional, such as Figure 5 As shown, step 104 generates multiple reconstructed images using multiple posture action features and posture action sample data sets, including: Step 501: extract features from a plurality of gesture action sample images in a gesture action sample data set to obtain a plurality of gesture action sample features.

[0100] In some examples, the process of extracting features from multiple gesture action sample images in a gesture action sample data set to obtain gesture action sample features may be: using a trained convolutional neural network to extract features from multiple gesture action sample images in a gesture action sample data set to obtain gesture action sample features. It may also be: using a human posture estimation model to extract features from multiple gesture action sample images in a gesture action sample data set to obtain gesture action sample features. For example, the trained convolutional neural network may be a classic VGG, ResNet, Inception or other pre-trained CNN model. The human posture estimation model is an OpenPose model.

[0101] Step 502: Replace multiple gesture action sample features with multiple gesture action features, and generate multiple reconstructed images.

[0102] The number of the plurality of posture action features is consistent with the number of the plurality of posture action sample features.

[0103] Combining the contents of the aforementioned step 103 and step 501, it can be known that the method of obtaining multiple gesture action features is similar to the method of obtaining multiple gesture action sample features. Therefore, after obtaining multiple gesture action features and multiple gesture action sample features, multiple gesture action features can be used to replace multiple gesture action sample features to obtain multiple reconstructed images. That is, the number of gesture action samples (i.e., reconstructed images) is increased by replacing multiple gesture action sample features with multiple gesture action features.

[0104] In some examples, the process of replacing multiple gesture action sample features with multiple gesture action features may also be referred to as action transfer.

[0105] Step 105: Process the multiple reconstructed images to obtain multiple posture action sample images.

[0106] After obtaining multiple reconstructed images using multiple posture action features, the multiple reconstructed images can be further processed, so that the image effect of the processed images can be better. When these posture action sample images are then used for other operations (for example, model training), the accuracy of the output results can be further improved.

[0107] Optional, such as Figure 6 As shown, step 105 processes the multiple reconstructed images to obtain multiple gesture action sample images, including: Step 601: Perform standardization processing on a plurality of reconstructed images to obtain a plurality of first reconstructed images.

[0108] In some examples, after obtaining the multiple reconstructed images, the multiple reconstructed images may be standardized to obtain multiple first reconstructed images.

[0109] Exemplarily, the standardization process may include image format standardization, image size standardization, and image color standardization. Of course, the standardization process may also include other contents, which will not be described in detail here.

[0110] Through standardization, the image format, size, color, etc. of multiple first reconstructed images can be kept consistent. In this way, it will be more convenient to store and manage these first reconstructed images. When performing operations such as image quality assessment and content comparison, consistent standards can ensure fairness and accuracy. At the same time, consistent standards can also adapt to subsequent processing procedures. When performing batch processing, unified image specifications can make the processing process more efficient, without the need to make separate adjustments for the different attributes of each image, reducing the complexity and error probability of the processing process.

[0111] Step 602: Perform quality improvement processing on the multiple first reconstructed images to obtain multiple posture action sample images.

[0112] The quality improvement process includes at least one of adjusting definition, adding noise, and color space conversion.

[0113] After the multiple reconstructed images are standardized, the quality of the multiple first reconstructed images can be further improved. Specifically, by adjusting the clarity, the details in the gesture action sample images can be made more obvious, and the clear images are visually easier to observe and analyze. By adding noise, the subsequent models trained based on multiple gesture action sample images (such as gesture action classification models) can have better tolerance to noise, and the gesture action sample images can be made closer to the real sample images. Since different color spaces describe colors differently, some color spaces may be more conducive to highlighting certain features of gestures. By changing the color space, some image processing algorithms or models may also perform better.

[0114] A method for generating a sample of a gesture image provided by an embodiment of the present disclosure can collect a large amount of gesture text data by creating a gesture database. Then, by processing a plurality of gesture text data, a plurality of gesture images can be obtained, and then a plurality of gesture sample images can be obtained using the plurality of gesture images. In other words, the present disclosure can obtain a large number of gesture sample images through a large amount of gesture text data. In this way, not only can the number of samples be significantly increased, but also the problems of insufficient data and poor sample quality can be effectively solved, providing strong data support for the application of artificial intelligence in specific fields. Afterwards, a plurality of gesture sample images can also be used for model training to improve the training effect and generalization ability of the model. And compared with the related art, the present disclosure can significantly reduce the cost of manpower and material resources by professional photographers squatting on the spot to capture gesture images.

[0115] The above mainly introduces the solution provided by the embodiment of the present disclosure from the perspective of the method. In order to achieve the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present disclosure.

[0116] The disclosed embodiment can divide the electronic device into functional modules according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the disclosed embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0117] Corresponding to the method in the aforementioned embodiment, the embodiment of the present disclosure further provides a sample generation device for a posture image. The sample generation device for a posture image is used to implement the aforementioned sample generation method for a posture image. The function of the sample generation device for a posture image can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0118] For example, Figure 7 A schematic diagram of the structure of a decision-making device is shown. Figure 7As shown, the decision-making device may include: a creation module 701 , a generation module 702 and a processing module 703 .

[0119] The creation module 701 is configured to create a gesture action database; the gesture action database includes a plurality of gesture action text data. Figure 1 , creation module 701 can be used to execute step 101.

[0120] The generating module 702 is configured to process a plurality of gesture action text data using a deep learning model to generate a plurality of gesture action images, wherein one gesture action text data corresponds to one gesture action image; and generate a plurality of reconstructed images using a plurality of gesture action features and a gesture action sample data set; the gesture action sample data set includes a plurality of gesture action sample images; for example, in combination with Figure 1 , the generation module 702 can be used to execute step 102 and step 104.

[0121] The processing module 703 is configured to extract features from the plurality of gesture action images to obtain a plurality of gesture action features corresponding to the plurality of gesture action images; and process the plurality of reconstructed images to obtain a plurality of gesture action sample images. Figure 1 , the generation module 702 can be used to execute step 103 and step 105.

[0122] Optionally, the creation module 701 is further configured to obtain a plurality of text data; the text data is used to generate a human body posture action; each of the plurality of text data corresponds to a different human body posture action; and obtain a posture action database based on the plurality of text data. Figure 2 , the creation module 701 can also be used to execute step 201 and step 202.

[0123] Optionally, the generating module 702 is further configured to obtain a text feature vector corresponding to each of the plurality of gesture action text data based on the plurality of gesture action text data; and input the text feature vector corresponding to each of the gesture action text data into the deep learning model to obtain a plurality of gesture action images; the deep learning model has the ability to generate gesture action images matching each of the text feature vectors. Figure 3 , generating module 702, can also be used to execute step 301 and step 302.

[0124] Optionally, the generating module 702 is further configured to extract features from multiple gesture action sample images in the gesture action sample data set to obtain multiple gesture action sample features; and replace multiple gesture action sample features with multiple gesture action features to generate multiple reconstructed images. Figure 5, generating module 702, can also be used to execute step 501 and step 502.

[0125] Optionally, the processing module 703 is further configured to perform standardization processing on the multiple reconstructed images to obtain multiple first reconstructed images; and perform quality improvement processing on the multiple first reconstructed images to obtain multiple gesture action sample images, wherein the quality improvement processing includes at least one of adjusting clarity, adding noise, and color space transformation. Figure 6 , processing module 703 can also be used to execute step 601 and step 602.

[0126] Optionally, the generating module 702 is further configured to extract key information from each gesture action text data to obtain each target gesture action text data; and encode each target gesture action text data to obtain a text feature vector corresponding to each gesture action text data. Figure 4 , generating module 702, can also be used to execute step 401 and step 402.

[0127] Optionally, each target posture action text data includes at least one of an action type, an action subject, and an action range.

[0128] Of course, the path selection device provided by the embodiment of the present disclosure includes but is not limited to the above modules, for example, the path selection device may also include a storage module 704. The storage module 704 may be used to store the program code of the write path selection device, and may also be used to store data generated by the write path selection device during operation, such as data in a write request, etc.

[0129] Figure 8 A structural diagram of an electronic device provided in an embodiment of the present disclosure, such as Figure 8 As shown, the electronic device 800 includes one or more processors 801 and a memory 802 .

[0130] Understandably, Figure 8 The electronic device 800 shown can implement all functions of the above-mentioned gesture image sample generation method. The functions of each module in the above-mentioned gesture image sample generation device can be implemented in the processor 801 of the electronic device 800. The storage module of the gesture image sample generation device is equivalent to the memory 802 of the electronic device 800.

[0131] The processor 801 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 800 to perform desired functions.

[0132] The memory 802 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 801 may execute the program instructions to implement the sample generation method of the gesture image, the sample generation method of the gesture image, and / or other desired functions of the various embodiments of the present disclosure described above.

[0133] In one example, the electronic device 800 may further include: an input device 803 and an output device 804 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0134] Of course, to simplify, Figure 8 Only some of the components related to the present disclosure in the electronic device 800 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 800 may also include any other appropriate components.

[0135] In addition to the above methods and devices, the embodiments of the present disclosure may also be a computer program product, which includes computer program instructions. When the computer program instructions are executed by a processor, the processor executes the steps in the sample generation method of the posture image in the above various embodiments of the present specification.

[0136] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0137] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the sample generation method of the posture image according to various embodiments of the present disclosure described in the above "Exemplary Method" section of this specification.

[0138] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0139] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0140] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and do not mean or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The word "such as" used here refers to the phrase "such as but not limited to", and can be used interchangeably with it.

[0141] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0142] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not meant to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0143] The above description has been given for the purpose of illustration and description. In addition, this description is not meant to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for generating a sample of a posture image, applied to an electronic device, the method comprising: Creating a gesture action database; the gesture action database includes a plurality of gesture action text data; Using a deep learning model, the plurality of gesture action text data are processed to generate a plurality of gesture action images, wherein one gesture action text data corresponds to one gesture action image; Extracting features from the plurality of posture action images to obtain a plurality of posture action features corresponding to the plurality of posture action images; Generate multiple reconstructed images using the multiple posture action features and the posture action sample data set; the posture action sample data set includes multiple posture action sample images; The multiple reconstructed images are processed to obtain multiple posture action sample images.

2. The method according to claim 1, characterized in that The creation of a gesture action database comprises: Acquire multiple text data; the text data is used to generate human body posture and action; each text data in the multiple text data corresponds to a different human body posture and action; Based on the multiple text data, the gesture action database is obtained.

3. The method according to claim 1, characterized in that The method of processing the plurality of posture action text data by using a deep learning model to generate a plurality of posture action images includes: Based on the plurality of gesture action text data, obtaining a text feature vector corresponding to each gesture action text data in the plurality of gesture action text data; The text feature vectors corresponding to the posture action text data are input into the deep learning model to obtain multiple posture action images; the deep learning model has the ability to generate posture action images matching each text feature vector.

4. The method according to claim 1, characterized in that The step of generating a plurality of reconstructed images by using the plurality of posture action features and the posture action sample data set comprises: Performing feature extraction on the plurality of gesture action sample images in the gesture action sample data set to obtain a plurality of gesture action sample features; The plurality of posture and action features are used to replace the plurality of posture and action sample features, and the plurality of reconstructed images are generated.

5. The method according to claim 1, characterized in that The processing of the multiple reconstructed images to obtain multiple gesture action sample images includes: performing standardization processing on the multiple reconstructed images to obtain multiple first reconstructed images; The multiple first reconstructed images are subjected to quality improvement processing to obtain the multiple gesture action sample images, wherein the quality improvement processing includes at least one of adjusting clarity, adding noise, and color space transformation.

6. The method according to claim 3, characterized in that The obtaining, based on the plurality of gesture action text data, a text feature vector corresponding to each gesture action text data in the plurality of gesture action text data comprises: Extract key information from the text data of each gesture action to obtain text data of each target gesture action; The target posture action text data are encoded to obtain text feature vectors corresponding to the posture action text data.

7. The method according to claim 6, characterized in that The target posture action text data includes at least one of an action type, an action subject and an action range.

8. A sample generation device for a posture image, characterized in that: The device comprises: A creation module configured to create a gesture action database; the gesture action database includes a plurality of gesture action text data; A generating module is configured to process the plurality of gesture action text data using a deep learning model to generate a plurality of gesture action images, wherein one gesture action text data corresponds to one gesture action image and multiple reconstructed images are generated using the plurality of gesture action features and a gesture action sample data set; the gesture action sample data set includes a plurality of gesture action sample images; The processing module is configured to perform feature extraction on the plurality of posture action images to obtain a plurality of posture action features corresponding to the plurality of posture action images; and to process the plurality of reconstructed images to obtain a plurality of posture action sample images.

9. An electronic device, characterized in that: include: A touch screen, comprising a touch sensor and a display screen; one or more processors; Memory; Wherein, one or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the electronic device, the electronic device executes the sample generation method of the gesture image as described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon; characterized in that: When the computer program instructions are executed by an electronic device, the electronic device implements the method for generating a sample of a gesture image as claimed in any one of claims 1 to 7.

11. A computer program product, characterized in that include: A memory and a processor, wherein the memory is used to store a computer program; and the processor is used to enable the electronic device to implement the sample generation method of the gesture image as described in any one of claims 1 to 7 when executing the computer program.

Images