A training method and system for user motion capture model

The training method for user motion capture models ensures accuracy by parsing scenarios, selecting data sets, and performing robustness calculations, enhancing model precision and reducing the need for post-deployment adjustments.

CN116645570BActive Publication Date: 2024-12-06SHIYOU (BEIJING) TECH CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202310421766.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-04-19
Publication Date
2024-12-06
Estimated Expiration
2043-04-19

AI Technical Summary

Technical Problem

Existing user motion capture models lack accuracy and reliability, requiring adaptive adjustments and optimizations, leading to decreased efficiency in subsequent processing.

Method used

A training method for user motion capture models involves parsing business scenarios, selecting appropriate data sets, training an initial model, and performing robustness calculations using noise operations to ensure accuracy before deployment.

Benefits of technology

The method enhances model accuracy, eliminating the need for subsequent adjustments, thereby improving processing efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116645570B_ABST
    Figure CN116645570B_ABST
Patent Text Reader

Abstract

The present invention discloses a training method and system for a user motion capture model, comprising: first responding to a user motion capture instruction of a target platform, selecting a training motion data set and a test motion data set corresponding to a business scenario; then completing the training of an initial user motion capture model based on the training motion data set; then performing robustness calculation on the user motion capture model according to the test motion data set to obtain a robustness result of the user motion capture model; and finally sending the user motion capture model to the target platform when the robustness result indicates that the user motion capture model passes the test. With such a design, the user motion capture model obtained through training can have a higher accuracy, and the target platform does not need to adjust and optimize the user motion capture model, thereby improving the processing efficiency of subsequent user motion capture.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a training method and system for a user motion capture model. Background Art

[0002] At present, with the rapid development of virtual reality technology, users' demand for realistic experience in virtual environments has also increased. As a key technology of virtual reality, motion capture generally uses artificial intelligence models to solve related problems. However, after users get the corresponding model, there is no guarantee that the model is accurate and reliable, and often adaptive adjustments and optimizations are required, which leads to a decrease in the efficiency of subsequent motion capture processing. Summary of the invention

[0003] The purpose of the present invention is to provide a training method and system for a user motion capture model.

[0004] In a first aspect, an embodiment of the present invention provides a method for training a user motion capture model, comprising:

[0005] Responding to the user motion capture instruction of the target platform, parsing the business scenario corresponding to the user motion capture instruction;

[0006] According to the business scenario, a training action data set and a test action data set corresponding to the business scenario are selected from a preset training database;

[0007] Calling an initial user motion capture model, and completing training of the initial user motion capture model based on a training motion data set, to obtain a trained user motion capture model;

[0008] The robustness of the user motion capture model is calculated according to the test motion data set to obtain the robustness result of the user motion capture model;

[0009] When the robustness result indicates that the user motion capture model passes the detection, the user motion capture model is sent to the target platform.

[0010] In a possible implementation, performing robustness calculation on the user motion capture model according to the test motion data set includes:

[0011] Obtain the test action dataset and the trained user motion capture model;

[0012] Performing a noise operation on the test action data set according to at least one noise strategy to obtain a test number of action data sets to be tested, wherein each noise strategy corresponds to at least one action data set to be tested;

[0013] Based on the user motion capture model, motion capture processing is performed on the test motion data set and the test number of to-be-tested motion data sets respectively, so as to obtain a first representation motion corresponding to each to-be-tested motion data set and a second representation motion corresponding to the test motion data set;

[0014] Obtaining a comparison result that the first representation action is consistent with the second representation action, and obtaining a motion capture confidence of the user motion capture model according to the comparison result;

[0015] The robustness of the user motion capture model is obtained according to the motion capture confidence, wherein the motion capture confidence and the robustness are in a positive feedback relationship.

[0016] In a possible implementation, performing a noise operation on the test action data set according to at least one noise strategy to obtain a test number of action data sets to be tested includes:

[0017] For each noise strategy of at least one noise strategy, the following processing is performed: a noise adding operation is performed on the test action data set according to the noise strategy to obtain at least one action data set to be tested corresponding to the noise strategy;

[0018] Performing action data evaluation on at least one action data set to be tested corresponding to each noise strategy, and obtaining a test number of action data sets to be tested according to the obtained action data evaluation results;

[0019] The categories of noise strategies include: first noise strategy;

[0020] When the category of the noise strategy is the first noise strategy, a noise adding operation is performed on the test action data set according to the noise strategy to obtain at least one action data set to be tested corresponding to the noise strategy, including:

[0021] Perform the following operations on the test action dataset to obtain at least one action dataset to be tested:

[0022] The following processing is performed on each test action data in the test action data set: at least one target area data is selected from the test action data, and a noise adding operation is performed on each target area data to obtain noise action data;

[0023] Integrate each noise action data to obtain a to-be-tested action data set, wherein each noise action data in the to-be-tested action data set has a mapping relationship with each test action data in the test action data set;

[0024] The categories of noise strategies include: second noise strategy;

[0025] When the noise strategy is the second noise strategy, performing a noise adding operation on the test action data set according to the noise strategy to obtain at least one action data set to be tested corresponding to the noise strategy includes:

[0026] Perform the following operations on the test action dataset to obtain at least one action dataset to be tested:

[0027] The following processing is performed on each test action data in the test action data set: obtaining key area data and non-key area data in the test action data, selecting at least one target area data in the leading position from the arrangement of the preset key degree coefficients of each non-key area data from large to small to perform a noise addition operation, and

[0028] Performing a noise adding operation on each key area data according to the image data of each key area data to obtain noise action data;

[0029] Each noise action data is integrated to obtain a to-be-tested action data set, wherein a mapping relationship exists between each noise action data in the to-be-tested action data set and each test action data in the test action data set.

[0030] In a possible implementation manner, before selecting at least one target area data of a leading position from the preset criticality coefficients of each non-critical area data arranged from large to small to perform the noise adding operation, the method further includes:

[0031] Performing an action region division operation on the test action data to obtain an action region division result of the test action data;

[0032] The following processing is performed for each non-critical area data: the non-critical area data is removed from the action area division result of the test action data to obtain pending action data corresponding to the non-critical area data;

[0033] Performing user motion capture on each pending action data and the test action data to obtain a representative action of each pending action data and a representative action of the test action data;

[0034] Obtain the difference characterization value between the characterization action of each pending action data and the characterization action of the test action data, and obtain the preset criticality coefficient of the non-critical area data corresponding to the pending action data according to the difference characterization value, wherein the difference characterization value and the preset criticality coefficient of the non-critical area data in the test action data are in a positive feedback relationship.

[0035] In a possible implementation, performing a noise adding operation on each key area data according to the image data of each key area data includes:

[0036] Perform at least one of the following processing on each key area data:

[0037] Deleting data of the key area data according to the image data of the key area data;

[0038] An image similar to the key area data is obtained, and the key area data is replaced with the similar image.

[0039] In a possible implementation, the noise addition operation performed on each target area data includes at least one of the following processing:

[0040] Adding obstacle image data between the target area data and the area data adjacent to the target area data;

[0041] Performing an image supplementation operation on the subsequent area data of the target area data, and adding the supplemented image data to after the target area data;

[0042] replacing the target area data with a similar image of the target area data;

[0043] Performing a dislocation operation on the positions of the target area data and the area data adjacent to the target area data in the test action data;

[0044] Performing a deletion operation on the image data in the target area data;

[0045] The target area data is replaced with corresponding other image data.

[0046] In a possible implementation, performing action data evaluation on at least one action data set to be tested corresponding to each noise strategy, and obtaining a test number of action data sets to be tested according to the obtained action data evaluation results, includes:

[0047] When the category of the noise strategy is the second noise strategy, the noise state parameter of each noise action data in each action data set to be tested is obtained, wherein the noise state parameter includes at least one of the following: the noise ratio of key area data and the noise ratio of non-key area data, and the abnormality of the noisy image;

[0048] When the category of the noise strategy is the first noise strategy, the noise state parameter of each noise action data in each action data set to be tested is obtained, wherein the noise state parameter includes at least one of the following: noise addition ratio, noise addition processing times, and noise addition image abnormality;

[0049] The action data set to be tested corresponding to the noise action data whose noise state parameter is greater than the noise state parameter threshold is removed;

[0050] In at least one action data set to be tested corresponding to each noise strategy, the number of action data sets to be tested that have not been removed is counted, and each noise action data in the action data set to be tested that has not been removed is marked according to the noise state parameter corresponding to each noise action data, so as to obtain the action data sets to be tested of the test number.

[0051] In a possible implementation manner, before performing a noise adding operation on the test action data set according to at least one noise strategy to obtain a test number of action data sets to be tested, the method further includes:

[0052] Obtain multiple noise strategies to be selected;

[0053] Performing a noise adding operation on the test action data set according to a plurality of noise strategies to obtain a preprocessed number of noise action data sets, wherein each noise strategy corresponds to at least one noise action data set;

[0054] Perform action type prediction operations on each noise action data set and the test action data set to obtain the predicted action types corresponding to each noise action data set and the test action data set;

[0055] The following processing is performed for each noise strategy:

[0056] Obtaining a predicted difference characterization value corresponding to each noise action data set corresponding to the noise strategy, wherein the predicted difference characterization value is a difference characterization value between a predicted action type corresponding to each noise action data set and a predicted action type corresponding to a test action data set;

[0057] The sum of the predicted difference representation values ​​corresponding to each noise action data set is divided by the number of noise action data sets corresponding to the noise strategy as the corresponding noise impact score of each noise strategy, and at least one noise strategy in the leading position is selected from the arrangement of the noise impact scores of each noise strategy from large to small.

[0058] In a possible implementation, the first characterization action of each to-be-tested action data set includes: the characterization action corresponding to each noise action data in each to-be-tested action data set; the second characterization action includes: the characterization action corresponding to each test action data in the test action data set;

[0059] When the at least one noise strategy is a plurality of noise strategies, obtaining a comparison result that the first characterization action is consistent with the second characterization action, and obtaining a motion capture confidence of the user motion capture model according to the comparison result, including:

[0060] For each characterization action of the test action data, a traversal matching operation is performed with the characterization action of the noise action data corresponding to each test action data to obtain the number of consistent characterization actions;

[0061] The result of dividing the number of consistent representation actions by the total amount of action data corresponding to the test action data set of the number of tests is used as the comparison result of the consistency between the first representation action and the second representation action, and the comparison result is used as the motion capture confidence of the user motion capture model.

[0062] In a second aspect, an embodiment of the present invention provides a server system, including a server, where the server is used to execute a method in at least one possible implementation of the first aspect.

[0063] Compared with the prior art, the beneficial effects provided by the present invention include: adopting a user motion capture model training method and system provided by an embodiment of the present invention, by responding to the user motion capture instruction of the target platform, selecting a training motion data set and a test motion data set corresponding to the business scenario; then completing the training of the initial user motion capture model based on the training motion data set; then performing robustness calculation on the user motion capture model according to the test motion data set to obtain the robustness result of the user motion capture model; finally, when the robustness result indicates that the user motion capture model has passed the detection, the user motion capture model is sent to the target platform. Such a design can make the trained user motion capture model have higher accuracy, and the target platform does not need to adjust and optimize the user motion capture model, thereby improving the processing efficiency of subsequent user motion capture. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can also be obtained based on these drawings without creative work.

[0065] Figure 1 A schematic flow chart of the steps of a method for training a user motion capture model provided by an embodiment of the present invention;

[0066] Figure 2 A schematic diagram of the structure of a server provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0068] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0070] Furthermore, the terms “first”, “second”, etc. are merely used for distinguishing descriptions and should not be understood as indicating or implying relative importance.

[0071] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms such as "setting" and "connection" should be understood in a broad sense. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0072] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings.

[0073] In order to solve the technical problems in the aforementioned background technology, Figure 1 This is a schematic diagram of the step flow of a method for training a user motion capture model provided in an embodiment of the present disclosure. The following is a detailed introduction to the method for training a user motion capture model.

[0074] Step S201 , in response to a user motion capture instruction of a target platform, parsing a business scenario corresponding to the user motion capture instruction.

[0075] Step S202: selecting a training action data set and a test action data set corresponding to the business scenario from a preset training database according to the business scenario.

[0076] Step S203, calling the initial user motion capture model, and completing the training of the initial user motion capture model based on the training motion data set to obtain a trained user motion capture model.

[0077] Step S204, performing robustness calculation on the user motion capture model according to the test motion data set to obtain a robustness result of the user motion capture model.

[0078] Step S205 , when the robustness result indicates that the user motion capture model has passed the detection, the user motion capture model is sent to the target platform.

[0079] In an embodiment of the present invention, the target platform may refer to a terminal, device, server, etc. maintained by a user who does not have R&D capabilities, and no limitation is made here. In order to enable the user motion capture model required by the target platform to have high accuracy, the business scenario required by the target platform can be determined based on the user motion capture instructions of the target platform. The business scenario is then used to select a training action data set and a test action data set corresponding to the business scenario from a preset training database. In an embodiment of the present invention, the business scenario and the corresponding training action data set and test action data set can be associated through a tag or a preset mapping relationship, and no limitation is made here. Then, in order to avoid mixing with other platforms, the initial user motion capture model can be called, and the training of the initial user motion capture model can be completed based on the training action data set to obtain a trained user motion capture model. In order to further enable the target platform to obtain a user motion capture model with high precision and to accurately perform motion capture, the robustness calculation of the user motion capture model can be performed on the user motion capture model based on the test motion data set to obtain the robustness result of the user motion capture model. When the robustness result indicates that the user motion capture model has passed the test, the user motion capture model is sent to the target platform. With such a design, the training result of the user motion capture model can be well adapted, and the target platform does not need to make relevant adjustments and optimizations to the user motion capture model, thereby improving the processing efficiency of subsequent motion capture.

[0080] In order to more clearly describe the solution provided by the embodiment of the present invention, the aforementioned step S204 can be implemented through the following detailed steps.

[0081] In step S101, a test motion data set and a trained user motion capture model are obtained.

[0082] For example, the test action data set includes a large amount of test action data, each test action data is pre-labeled, and the labeling labels include: action category, area name of each key area data corresponding to the test action data, image data corresponding to the test action data, the start of the key area data corresponding to the test action data, the category of the key area data, etc.

[0083] In step S102, a noise operation is performed on the test action data set according to at least one noise strategy to obtain a test number of action data sets to be tested.

[0084] Here, each noise strategy corresponds to at least one action data set to be tested.

[0085] For example, the categories of noise strategies include the second noise strategy and the first noise strategy; the first noise strategy: randomly select image area data from the test action data for noise addition operation; the second noise strategy: use different noise addition operations for the key area data and non-key area data in the test action data according to the category (key area data and non-key area data). The overall noise addition operation process of different noise strategies is different. For example: Noise strategy 1, the test action data is processed in the following order: delete part of the area processing, similar image replacement processing and add image processing; Noise strategy 2, the test action data is processed in the following order: add interference image data to the test action data, and set the position of adjacent area data in the test action data to be staggered. Noise strategy 1 is different from noise strategy 2.

[0086] For example, there is a mapping relationship between each noise action data in the action data set to be tested and the test action data in the test action data set.

[0087] In an embodiment of the present invention, when at least one noise strategy is a plurality of different noise strategies, the number of tests can be set to a larger value, and a large number of action data sets to be tested can be used to obtain the robustness of the model by using the differences between the characterization actions in the action data set dimension; when at least one noise strategy is a second noise strategy, the number of tests can be a value greater than or equal to 1, and each noise action data in the action data set to be tested can be used to perform multiple confrontations on the user motion capture model (multiple motion capture processes are performed on the noise action data based on the user motion capture model), and the robustness of the model can be obtained by using the differences between the characterization actions in the test action data dimension.

[0088] In some embodiments, step S102 can be implemented through steps S1021 to S1022, which are described in detail below.

[0089] In step S1021, the following processing is performed for each noise strategy of the at least one noise strategy: a noise adding operation is performed on the test action data set according to the noise strategy to obtain at least one action data set to be tested corresponding to the noise strategy.

[0090] For example, at least one to-be-verified action data set may be obtained for each noise strategy, and the number of to-be-verified action data sets corresponding to each noise strategy may be different.

[0091] In some embodiments, when the category of the noise strategy is the first noise strategy, step S1021 can be implemented in the following manner: perform the following operations on the test action data set to obtain at least one action data set to be tested: perform the following processing on each test action data in the test action data set: select at least one target area data from the test action data, perform noise addition operation on each target area data to obtain noise action data; integrate each noise action data to obtain the action data set to be tested.

[0092] Here, there is a mapping relationship between each noise action data in the action data set to be tested and each test action data in the test action data set.

[0093] For example, at least one target area data can be randomly selected from the test action data, and the noise addition operation of each target area data can be performed concurrently without interfering with the noise addition operation of other target area data. For example: the test action data is "whole arm image data", and the target area data "palm area image data" and "forearm area image data" are selected. Under the condition that the noise addition operations do not interfere with each other, the noise addition operation can be performed on the above target area data at the same time, similar image replacement can be performed on the "palm area image data", and obstacle image data can be added to the "forearm area image data".

[0094] For example, performing a noise adding operation on each target area data may be implemented in at least one of the following ways.

[0095] (1) Adding obstacle image data between the target area data and the adjacent area data of the target area data; for example, after processing, "whole arm image data" becomes "whole arm image data" with multiple mosaic image data interspersed in the middle.

[0096] (2) Performing image supplementation operations on the subsequent area data of the target area data, and adding the supplemented image data to the end of the target area data; for example, "palm area image data" is processed to become "the entire arm area image data including the palm".

[0097] (3) The target area data is replaced with a similar image of the target area data; for example, "the image data of the user's first palm area" is processed into "the image data of the user's second palm area".

[0098] (4) Performing a dislocation operation on the target area data and the adjacent area data of the target area data in the test action data; for example, the complete "palm area image data" and "forearm area image data" are processed into dislocated "palm area image data" and "forearm area image data".

[0099] (5) Perform a deletion operation on the image data in the target area data; for example, "whole arm image data" is processed into "only palm image data".

[0100] (6) Replace the target area data with corresponding other image data. For example, "left arm image data" is replaced with "right arm image data".

[0101] In some embodiments, when the noise strategy is the second noise strategy, step S1021 can be implemented in the following manner: perform the following operations on the test action data set to obtain at least one action data set to be tested: perform the following processing on each test action data in the test action data set: obtain the key area data and non-key area data in the test action data, select at least one target area data in the leading position from the large to small arrangement of the preset criticality coefficient of each non-key area data for noise addition operation, and perform noise addition operation on each key area data according to the image data of each key area data to obtain noise action data; integrate each noise action data to obtain the action data set to be tested, wherein there is a mapping relationship between each noise action data in the action data set to be tested and each test action data in the test action data set.

[0102] For example, the preset criticality coefficient of the non-critical area data is used to reflect the importance of the non-critical area data in the test action data. By removing the non-critical area data from the test action data, the pending action data corresponding to the non-critical area data can be obtained. If the meanings expressed by the pending action data and the test action data are quite different, the preset criticality coefficient of the non-critical area data in the test action data. On the contrary, the preset criticality coefficient is low. The non-critical area data is used as the target area data. The noise addition operation for the target area data can refer to the above description and will not be repeated here.

[0103] For example, the number of target area data selected from the front position arranged from large to small according to the preset criticality coefficient of each non-critical area data can be obtained according to the number of non-critical area data in the test action data, and the number of non-critical area data and the number of selected target area data are in a positive feedback relationship.

[0104] For example, the noise addition operation for each key area data may be implemented by at least one of the following processes.

[0105] (1) Without affecting the meaning of the key area data, the key area data is deleted according to the image data of the key area data.

[0106] (2) Obtaining an image similar to the key area data and replacing the key area data with the similar image. In some embodiments, before step S1021, a preset criticality coefficient of the non-key area data may be obtained through steps S301 to S303.

[0107] In step S301, a motion region division operation is performed on the test motion data to obtain a motion region division result of the test motion data.

[0108] For the sake of explanation, the following is an example of the test action data "whole arm image data". The test action data is divided into "palm image data, forearm image data, shoulder image data". Among them, "forearm image data and shoulder image data" are non-critical area data.

[0109] In step S302, the following processing is performed on each non-critical area data: the non-critical area data is removed from the action area division result of the test action data to obtain the pending action data corresponding to the non-critical area data; user motion capture is performed on each pending action data and the test action data to obtain the characterization action of each pending action data and the characterization action of the test action data.

[0110] As an example, the above test action data is further described, and the non-critical area data are removed from the action area division result of the test action data in sequence, and the following pending action data can be obtained.

[0111] Undetermined action data 1, "palm image data, forearm image data";

[0112] Pending action data 2, "palm image data, shoulder image data";

[0113] Based on the user motion capture model to be detected, the pending motion data and the test motion data are captured and predicted respectively to obtain the corresponding representation motions of each pending motion data and the test motion data. Alternatively, the decision tree of the model's internal parameter structure can be used to capture and predict the pending motion data and the test motion data to obtain the corresponding representation motions.

[0114] In step S303, a difference characterization value between the characterization action of each pending action data and the characterization action of the test action data is obtained, and a preset criticality coefficient of the non-critical area data corresponding to the pending action data is obtained according to the difference characterization value.

[0115] Here, the difference characterization value and the preset criticality coefficient of the non-critical area data in the test action data are in a positive feedback relationship.

[0116] For example, the characterization action includes the number of action indication contents. The difference characterization value between the characterization actions is the difference characterization value between the number of action indication contents corresponding to the pending action data and the test action data, and the difference characterization value is used to characterize the preset criticality coefficient. Correspondingly, the higher the difference characterization value, the higher the importance of the pending action data compared to the non-critical area data missing from the test action data, and the greater the impact of the missing key area data on the meaning of the test action data. Assume that the characterization action obtained by the original test action data is 5 action indication contents, the characterization action of the pending action data N (corresponding to the non-critical area data n) is 4 action indication contents, and the difference characterization value is 1; the characterization action of the pending action data M (corresponding to the non-critical area data m) is 3 key area data, and the difference characterization value is 2; the importance of the non-critical area data m is higher than that of the non-critical area data n. The difference characterization values ​​corresponding to each non-critical area data are sorted in descending order, and at least one non-critical area data is selected as the target area data according to the arrangement from large to small, and the target area data is subjected to noise addition operation.

[0117] In step S1022, action data evaluation is performed on at least one action data set to be tested corresponding to each noise strategy, and a test number of action data sets to be tested are obtained according to the obtained action data evaluation results.

[0118] For example, the action data evaluation result includes the noise state parameters corresponding to the action data set to be tested. When the noise state parameters do not meet the corresponding threshold, the action data set to be tested whose noise state parameters do not meet the corresponding threshold is removed, and the action data set to be tested that meets the corresponding threshold is retained to obtain the test number of action data sets to be tested.

[0119] In some embodiments, the test number may be a preset number, and when the test number of action data sets to be tested is obtained, step S103 is executed. If the number of action data sets to be tested is less than the test number, the test action data sets are further denoised to obtain the action data sets to be tested.

[0120] In some embodiments, step S1022 may be implemented through steps S401 to S404, which are described in detail below.

[0121] In step S401, when the category of the noise strategy is the second noise strategy, the noise state parameters of each noise action data in each action data set to be tested are obtained, and the noise state parameters include at least one of the following: the noise ratio of key area data and the noise ratio of non-key area data, and the abnormality of the noisy image.

[0122] For example, the noise ratio is the ratio of the number of image data subjected to noise operation in the action data to the number of original image data in the action data. The key area data noise ratio is the noise ratio for the key area data in the action data, and the key area data noise ratio is the ratio of the number of image data subjected to noise operation in the action data to the number of original image data in the action data; the non-key area data noise ratio is the noise ratio for the non-key area data in the action data, and the non-key area data noise ratio is the ratio of the number of image data subjected to noise operation in the action data to the number of original image data in the action data.

[0123] Assume that the number of image data contained in the original test action data is 100 image data, of which the number of image data in the key area data is 10 and the number of image data in the non-key area data is 20; after the noise addition operation, 15 image data are noised, and the noise addition rate of the noise action data is 15%.

[0124] In step S402, when the category of the noise strategy is the first noise strategy, the noise state parameters of each noise action data in each action data set to be tested are obtained, and the noise state parameters include at least one of the following: noise ratio, noise processing times, and noisy image abnormality.

[0125] For example, the noise ratio and the abnormality of the noise image are explained in the above description, which will not be repeated here. The number of noise processing times specifically refers to the number of noise operations between the original test action data and the noise action data, and the number of noise operations can be characterized as the number of executions of the editing operation (in the embodiment of the present invention, the noise operation).

[0126] In step S403, the to-be-checked action data set corresponding to the noise action data whose noise state parameter is greater than the noise state parameter threshold is removed.

[0127] For example, if the noise state parameter is greater than the noise state parameter threshold, it means that there is a lot of noise in the noise action data and it is not suitable as a test sample.

[0128] For example, different noise state parameter thresholds can be set for different noise state parameters. For example: for the noise ratio, the corresponding noise ratio threshold can be 15% to avoid excessive noise in the noise action data. For the noise ratio of non-critical area data and the noise ratio of critical area data, the sum of the two can be less than or equal to 15%; or, thresholds are set for the two separately. For the noise operation number threshold, it can be set according to the length of the action data. The action data length and the noise operation number threshold are in a positive feedback relationship. The higher the action data length, the higher the corresponding noise operation number threshold. For the abnormality of the noisy image, it can be set uniformly. When the abnormality of the noisy image is higher than the abnormality threshold of the noisy image, it means that the noise action data does not meet the reasonable display state of the image and the noise is too large.

[0129] In some embodiments, when at least one noise action data whose noise state parameter is greater than a noise state parameter threshold value is detected in the action data set to be tested, the noise state parameters of other noise action data that have not been detected in the action data set to be tested may also be too large, and the action data set to be tested is removed. By removing the action data set to be tested that does not meet the test requirements, the accuracy of the samples used for the test is improved, and the computing resources required for action data evaluation are saved.

[0130] In some embodiments, the noise action data whose noise state parameter is greater than the noise state parameter threshold can be removed from the action data set to be tested, the noise action data whose noise state parameter is less than the noise state parameter threshold is retained, the noise action data whose corresponding test action data is different is selected, and the action data set to be tested for testing is combined, and the combined action data set to be tested is retained. By combining the action data sets to be tested, computing resources are saved and the generated action data sets to be selected are avoided.

[0131] In step S404, in at least one action data set to be tested corresponding to each noise strategy, the number of action data sets to be tested that have not been removed is counted, and each noise action data in the action data set to be tested that has not been removed is marked according to the noise state parameter corresponding to each noise action data, so as to obtain the test number of action data sets to be tested.

[0132] For example, the number of tests is obtained by counting the number of unremoved action data sets to be tested. The number of tests can be a large value. At the same time, the action data sets to be tested of the test number can be annotated according to the results of the quality inspection, which can save the time and manpower consumed by the manual annotation process. The annotation of each noise action data in the action data set to be tested can include: the test action data corresponding to the noise action data, the number of noise addition operations of the noise action data, the noise addition rate, the noise addition ratio of the key area data, the noise addition ratio of the non-key area data, the abnormality of the noise image, the key area data in the noise action data, the non-key area data, etc. By annotating each noise action data, the test number of action data sets to be tested is obtained.

[0133] In the embodiment of the present invention, the quality of the action data set to be tested is controlled by performing action data evaluation, thereby improving the accuracy of the test of the user motion capture model. By first obtaining the action data set to be tested and then screening the evaluation set, the amount of calculation required in the process of adding noise is reduced, the efficiency of generating the action data set to be tested is improved, and the quality of the action data set to be tested used in the test process is ensured by screening the set, thereby improving the accuracy of the test.

[0134] In some embodiments, before step S102, at least one noise strategy for testing the user motion capture model may be acquired through the following steps S501A to S504A.

[0135] In step S501A, multiple noise strategies to be selected are obtained.

[0136] For example, the noise strategy can be obtained by obtaining multiple noise addition operations and randomly combining the multiple noise addition operations. For example, multiple noise addition operations are obtained, at least one noise addition operation is randomly selected from the noise addition operations, and the processing order of the noise addition operations is randomly sorted (when there is only one noise addition operation, no sorting is required) to obtain multiple noise strategies to be selected. The number of the multiple noise strategies to be selected can be 30.

[0137] In step S502A, a noise operation is performed on the test action data set according to a plurality of noise strategies to obtain a preprocessed number of noise action data sets.

[0138] Here, each noise strategy corresponds to at least one noise action dataset.

[0139] For example, the number of preprocessing can be much smaller than the number of tests in the above description, for example, the number of preprocessing can be 300. Assume that the test action data set is denoised according to the above 30 noise strategies to obtain 300 noise action data sets, each noise strategy can correspond to at least one noise action data set, and the number of noise action data sets corresponding to each noise strategy can be the same or different, or the number of noise action data sets corresponding to all noise strategies is the same.

[0140] In step S503A, an action type prediction operation is performed on each noise action data set and each test action data set to obtain a predicted action type corresponding to each noise action data set and each test action data set.

[0141] For example, the action prediction model to be detected can be used, or other trained action data capture prediction models can be obtained to perform action type prediction operations on the noise action data and the test action data set, respectively, to obtain the predicted action types corresponding to each noise action data set and the test action data set. In the embodiment of the present invention, the predicted action type is the model prediction score as an example for explanation.

[0142] In step S504A, the noise impact score of each noise strategy is obtained according to the predicted action types corresponding to each noise action data set and the test action data set, and at least one noise strategy in the leading position is selected from the noise impact scores of each noise strategy arranged from large to small.

[0143] For example, the higher the noise impact score, the higher the noise addition effect corresponding to the noise strategy. That is, when the noise addition operation is performed according to the noise strategy, more noise can be added to the test action data. This noise strategy has a higher interference with the capture prediction process of the action prediction model.

[0144] Continuing with the above example, the noise impact scores of the 30 noise strategies are sorted in descending order to obtain an arrangement from large to small, and 5 noise strategies can be selected from the leading positions in the descending order.

[0145] For example, obtaining the noise impact score of each noise strategy in step S504A can be achieved in the following manner: performing the following processing for each noise strategy: obtaining the predicted difference characterization value corresponding to each noise action data set corresponding to the noise strategy, and dividing the sum of the predicted difference characterization values ​​corresponding to each noise action data set by the number of noise action data sets corresponding to the noise strategy as the corresponding noise impact score of each noise strategy.

[0146] Here, the predicted difference representation value is a difference representation value between the predicted action type corresponding to each noise action data set and the predicted action type corresponding to the test action data set.

[0147] For example, the noise impact score is defined as K, then K = Sum (AB) / C, where C is the number of noise action data sets corresponding to the noise strategy, A is the model prediction score corresponding to one of the noise action data sets, B is the model prediction score corresponding to the test action data set, AB is the prediction difference representation value, and Sum (AB) is the sum of the prediction difference representation values.

[0148] In step S103, motion capture processing is performed on the test motion data set and the test number of motion data sets to be tested based on the user motion capture model to obtain a first representation motion corresponding to each motion data set to be tested and a second representation motion corresponding to the test motion data set.

[0149] For example, different capture prediction processes can be used for different dimensional test methods and different noise strategies.

[0150] In some embodiments, step S103 can be implemented in the following ways: based on the user motion capture model, motion capture processing is performed on each noise motion data in each action data set to be tested to obtain the characterization action corresponding to each noise motion data, and the characterization action of each noise motion data is combined to obtain the first characterization action of each action data set to be tested; based on the user motion capture model, motion capture processing is performed on each test action data in the test action data set to obtain the characterization action corresponding to each test action data, and the characterization action of each test action data is combined to obtain the second characterization action of the test action data set.

[0151] For example, when the number of tests is large, a large number of verification conclusions can be obtained by performing a capture prediction on each action data set to be tested, thereby obtaining the robustness of the action prediction model based on a large number of verification conclusions and obtaining a better detection effect.

[0152] In some embodiments, when the category of the noise strategy is the second noise strategy, step S103 can be implemented in the following manner: perform the following processing on each noise action data in each action data set to be tested: perform multiple motion capture processes on the noise action data based on the user motion capture model; obtain multiple characterization actions corresponding to the noise action data, and combine the multiple characterization actions of the noise action data to obtain the first characterization action of the action data set to be tested; perform motion capture processing on each test action data in the test action data set based on the user motion capture model to obtain the characterization action corresponding to each test action data, and combine the characterization actions of each test action data to obtain the second characterization action of the test action data set.

[0153] For example, when the number of tests is small, the noisy motion data can be captured and predicted multiple times to obtain a large number of verification conclusions, so as to obtain the robustness of the motion prediction model based on a large number of verification conclusions and obtain better detection effects. The user motion capture model performs motion capture processing on the noisy motion data multiple times, which can also be regarded as the noise motion data confronting or interfering with the user motion capture model multiple times.

[0154] In step S104, a comparison result is obtained that shows that the first representation action is consistent with the second representation action, and a motion capture confidence of the user motion capture model is obtained according to the comparison result.

[0155] For example, if the capture prediction is successful, that is, the first representation action is consistent with the second representation action, when there are many representation actions, the motion capture confidence can be obtained in different dimensions.

[0156] In some embodiments, the confidence of motion capture can be obtained in the set dimension. Correspondingly, the first characterization action of each to-be-tested action data set includes: the characterization action corresponding to each noise action data in each to-be-tested action data set; the second characterization action includes: the characterization action corresponding to each test action data in the test action data set.

[0157] When the at least one noise strategy is multiple noise strategies, step S104 may be implemented through steps S501B to S502B, which are described in detail below.

[0158] In step S501B, a traversal matching operation is performed on the characterization action of each test action data and the characterization action of the noise action data corresponding to each test action data to obtain the number of consistent characterization actions.

[0159] For example, the characterization action is consistent, that is, the action indication content obtained by the action prediction model for capturing and predicting the noise action data is consistent with the action indication content obtained by capturing and predicting the test action data. If the computing resources of the device corresponding to the action prediction model are sufficient, the capture prediction of each test action data and each noise action data can be performed in parallel to improve the test efficiency.

[0160] In step S502B, the result of dividing the number of consistent representation actions by the total amount of action data corresponding to the test number of action data sets to be tested is used as the comparison result of the consistency between the first representation action and the second representation action, and the comparison result is used as the motion capture confidence of the user motion capture model.

[0161] For example, there is a mapping relationship between the action data of each action data set to be tested and the action data of the test action data set. The number of tests can be multiplied by the number of action data in the test action data set to obtain the total amount of action data corresponding to the action data set to be tested with the number of tests, and the ratio of the number of consistent representation actions to the total amount of action data is used as the confidence level of motion capture.

[0162] In some embodiments, the confidence of motion capture can be obtained in the sample dimension. Correspondingly, the first characterization action of each to-be-tested action data set includes: multiple characterization actions corresponding to each noise action data in each to-be-tested action data set; the second characterization action includes: the characterization action corresponding to each test action data in the test action data set;

[0163] When the category of the noise strategy is the second noise strategy, step S104 can be implemented through steps S501C to S502C, which are described in detail below.

[0164] In step S501C, the following processing is performed for each noise action data: multiple characterization actions of the noise action data are compared with the characterization actions of the test action data corresponding to the noise action data. When at least one of the multiple characterization actions of the noise action data is consistent with the characterization action of the test action data corresponding to the noise action data, a capture prediction success is obtained.

[0165] For example, the method of step S501C to step S502C can be applied when there are fewer action data sets to be tested. Assuming that there is one action data set to be tested, a maximum of Q capture predictions (for example, Q=10) are performed for each noise action data in the action data set to be tested. When there is a situation in the capture prediction process where the noise action data is consistent with the characterization action corresponding to the test action data, the capture prediction of the noise action data is stopped, and a successful capture prediction is counted. Through this capture prediction method and statistical method, the time required for the capture prediction process and the number of capture predictions can be saved, thereby improving the overall efficiency of the test.

[0166] In step S502C, the result of dividing the number of capture prediction successes corresponding to each action data set to be tested by the number of test action data in the test action data set is used as the comparison result of the consistency between the first characterization action and the second characterization action, and the comparison result is used as the motion capture confidence of the user motion capture model.

[0167] For example, the number of test action data in the test action data set is known, and the ratio of the number of successful capture predictions to the number of test action data is used as the confidence of motion capture. Assuming that the capture prediction of the action prediction model for noisy action data is used as the adversary of noisy action data against the action prediction model, the motion capture confidence can also be used as the defense success rate of the action prediction model. Assuming that the motion capture confidence is T, the adversarial success rate corresponding to the action data set to be tested is (1-T).

[0168] In step S105, the robustness of the user motion capture model is obtained according to the motion capture confidence.

[0169] Here, there is a positive feedback relationship between motion capture confidence and robustness.

[0170] For example, the same motion prediction model is trained to obtain different versions of optimized motion prediction models. The training method of the user motion capture model provided by the embodiment of the present invention can be repeatedly operated to obtain different results of robustness, thereby comparing different versions of the model to obtain the motion prediction model with the best robustness. The training method of the user motion capture model provided by the embodiment of the present invention can effectively improve the accuracy of the model in capturing user motions.

[0171] The embodiment of the present invention adds noise to a test action data set according to at least one noise strategy to obtain at least one action data set to be tested that includes noise action data, and captures and predicts the test action data set and at least one action data set to be tested based on a user motion capture model, and obtains a comparison result in which the output results of the user motion capture model for the test action data set and the action data set to be tested are consistent, and the robustness of the user motion capture model is obtained based on the comparison result.

[0172] The embodiment of the present invention provides a server 100, which includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the server 100 executes the training method of the user motion capture model. Figure 2 As shown, Figure 2 The server 100 is a block diagram of a structure of a server 100 provided in an embodiment of the present invention. The server 100 includes a memory 111, a processor 112 and a communication unit 113. To achieve data transmission or interaction, the memory 111, the processor 112 and the communication unit 113 are electrically connected to each other directly or indirectly.

[0173] For the purpose of illustration, the foregoing description is made with reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the present disclosure to the precise form disclosed. In accordance with the above teachings, numerous modifications and variations are possible. These embodiments are selected and described in order to best illustrate the principles of the present disclosure and its practical application, so that those skilled in the art can best utilize the present disclosure and utilize various embodiments with different modifications to suit the intended specific application. For the purpose of illustration, the foregoing description is made with reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the present disclosure to the precise form disclosed. In accordance with the above teachings, numerous modifications and variations are possible. These embodiments are selected and described in order to best illustrate the principles of the present disclosure and its practical application, so that those skilled in the art can best utilize the present disclosure and utilize various embodiments with different modifications to suit the intended specific application.

Claims

1. A training method for a user motion capture model, It is characterized in that include: In response to a user motion capture instruction of a target platform, parsing a business scenario corresponding to the user motion capture instruction; According to the business scenario, a training action data set and a test action data set corresponding to the business scenario are selected from a preset training database; Calling an initial user motion capture model, and completing training of the initial user motion capture model based on the training motion data set to obtain a trained user motion capture model; Performing robustness calculation on the user motion capture model according to the test motion data set to obtain a robustness result of the user motion capture model; When the robustness result indicates that the user motion capture model passes the detection, sending the user motion capture model to the target platform; The performing robustness calculation on the user motion capture model according to the test motion data set comprises: Obtain the test action dataset and the trained user motion capture model; Performing a noise adding operation on the test action data set according to at least one noise strategy to obtain a test number of action data sets to be tested, wherein each of the noise strategies corresponds to at least one action data set to be tested; Based on the user motion capture model, motion capture processing is performed on the test motion data set and the test number of the motion data sets to be tested, respectively, to obtain a first representation motion corresponding to each of the motion data sets to be tested and a second representation motion corresponding to the test motion data set; Obtaining a comparison result that the first characterization action is consistent with the second characterization action, and obtaining a motion capture confidence of the user motion capture model according to the comparison result; Obtaining the robustness of the user motion capture model according to the motion capture confidence, wherein the motion capture confidence and the robustness are in a positive feedback relationship; The step of performing a noise adding operation on the test action data set according to at least one noise strategy to obtain a test number of action data sets to be inspected includes: For each of the at least one noise strategy, the following processing is performed: performing a noise adding operation on the test action data set according to the noise strategy to obtain at least one action data set to be tested corresponding to the noise strategy; Performing action data evaluation on the at least one action data set to be tested corresponding to each noise strategy, and obtaining a test number of action data sets to be tested according to the obtained action data evaluation results; The categories of the noise strategies include: first noise strategy; When the category of the noise strategy is the first noise strategy, performing a noise adding operation on the test action data set according to the noise strategy to obtain at least one action data set to be tested corresponding to the noise strategy includes: Perform the following operations on the test action data set to obtain at least one action data set to be tested: The following processing is performed on each of the test action data in the test action data set: at least one target area data is selected from the test action data, and a noise adding operation is performed on each of the target area data to obtain noise action data; Integrate each of the noise action data to obtain a to-be-tested action data set, wherein each of the noise action data in the to-be-tested action data set has a mapping relationship with each of the test action data in the test action data set; The categories of the noise strategies include: a second noise strategy; When the noise strategy is the second noise strategy, performing a noise adding operation on the test action data set according to the noise strategy to obtain at least one action data set to be tested corresponding to the noise strategy includes: Perform the following operations on the test action data set to obtain at least one action data set to be tested: The following processing is performed on each of the test action data in the test action data set: obtaining the key area data and the non-key area data in the test action data, selecting at least one target area data in the leading position from the arrangement of the preset key degree coefficients of each of the non-key area data from large to small to perform a noise addition operation, and Performing a noise adding operation on each of the key area data according to the image data of each of the key area data to obtain noise action data; Integrate each of the noise action data to obtain a to-be-tested action data set, wherein each of the noise action data in the to-be-tested action data set has a mapping relationship with each of the test action data in the test action data set; The first characterization action of each of the to-be-tested action data sets includes: the characterization action corresponding to each of the noise action data in each of the to-be-tested action data sets; the second characterization action includes: the characterization action corresponding to each of the test action data in the test action data set; When the at least one noise strategy is a plurality of noise strategies, obtaining a comparison result that the first characterization action is consistent with the second characterization action, and obtaining the motion capture confidence of the user motion capture model according to the comparison result, includes: For each characterization action of the test action data, a traversal matching operation is performed with the characterization action of the noise action data corresponding to each test action data to obtain the number of consistent characterization actions; The result of dividing the number of consistent characterizing actions by the total amount of action data corresponding to the test number of action data sets to be tested is used as the comparison result of the consistency between the first characterizing action and the second characterizing action, and the comparison result is used as the action capture confidence of the user action capture model.

2. The method according to claim 1, It is characterized in that Before performing the noise adding operation on at least one target area data in the leading position selected from the preset criticality coefficients of each non-critical area data arranged from large to small, the method further includes: Performing an action area division operation on the test action data to obtain an action area division result of the test action data; The following processing is performed for each non-critical area data: the non-critical area data is removed from the action area division result of the test action data to obtain pending action data corresponding to the non-critical area data; Performing user motion capture on each of the pending action data and the test action data to obtain a representative action of each of the pending action data and a representative action of the test action data; Obtain a difference characterization value between the characterization action of each of the pending action data and the characterization action of the test action data, and obtain a preset criticality coefficient of the non-critical area data corresponding to the pending action data based on the difference characterization value, wherein the difference characterization value and the preset criticality coefficient of the non-critical area data in the test action data are in a positive feedback relationship.

3. The method according to claim 1, It is characterized in that The step of performing a noise adding operation on each of the key area data according to the image data of each of the key area data comprises: Perform at least one of the following processing on each of the key area data: Deleting data of the key area data according to the image data of the key area data; An image similar to the key area data is acquired, and the key area data is replaced with the similar image.

4. The method according to claim 1, It is characterized in that The noise adding operation performed on each target area data includes at least one of the following processing: Adding obstacle image data between the target area data and the adjacent area data of the target area data; Performing an image supplement operation on the subsequent area data of the target area data, and adding the supplemented image data to after the target area data; replacing the target area data with a similar image of the target area data; Performing a dislocation operation on the positions of the target area data and the area data adjacent to the target area data in the test action data; Performing a deletion operation on the image data in the target area data; The target area data is replaced with corresponding other image data.

5. The method according to claim 1, It is characterized in that The step of performing action data evaluation on the at least one action data set to be tested corresponding to each noise strategy, and obtaining a test number of action data sets to be tested according to the obtained action data evaluation results, comprises: When the category of the noise strategy is the second noise strategy, the noise state parameter of each noise action data in each of the action data sets to be tested is obtained, wherein the noise state parameter includes at least one of the following: the noise ratio of key area data and the noise ratio of non-key area data, and the abnormality of the noise image; When the category of the noise strategy is the first noise strategy, obtaining the noise state parameter of each noise action data in each of the action data sets to be inspected, wherein the noise state parameter includes at least one of the following: noise addition ratio, noise addition processing times, and noise addition image abnormality; Removing the to-be-tested action data set corresponding to the noise action data whose noise state parameter is greater than the noise state parameter threshold; In the at least one action data set to be tested corresponding to each noise strategy, the number of action data sets to be tested that have not been removed is counted, and each noise action data in the action data sets to be tested that have not been removed is marked according to the noise state parameter corresponding to each noise action data, so as to obtain a test number of action data sets to be tested.

6. The method according to claim 1, It is characterized in that Before performing a noise adding operation on the test action data set according to at least one noise strategy to obtain a test number of action data sets to be tested, the method further includes: Obtain multiple noise strategies to be selected; Performing a noise adding operation on the test action data set according to the multiple noise strategies to obtain a preprocessed number of noise action data sets, wherein each noise strategy corresponds to at least one of the noise action data sets; Performing an action type prediction operation on each of the noise action data set and the test action data set to obtain a predicted action type corresponding to each of the noise action data set and the test action data set; The following processing is performed for each of the noise strategies: Obtaining a predicted difference characterization value corresponding to each of the noise action data sets corresponding to the noise strategy, wherein the predicted difference characterization value is a difference characterization value between a predicted action type corresponding to each of the noise action data sets and a predicted action type corresponding to the test action data set; The sum of the predicted difference characterization values ​​corresponding to each of the noise action data sets is divided by the number of noise action data sets corresponding to the noise strategy as the corresponding noise impact score of each noise strategy, and at least one noise strategy in the leading position is selected from the arrangement of the noise impact scores of each of the noise strategies from large to small.

7. A server system, It is characterized in that The method comprises a server, wherein the server is used to execute the method according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • Model training method and device and electronic equipment

    CN112818898A

  • AI model generation method, electronic equipment and storage medium

    CN112884092A

  • Training data processing method and device, electronic equipment and storage medium

    CN114997396A

  • Image processing model training method and device

    CN115424088A

  • Decentralized artificial intelligence (AI) / machine learning training system

    US20220344049A1