Single-point motion learning method and apparatus

Abstract

The application discloses a single-point motion learning method, comprising the following steps: acquiring a position set of a target single point in a video, wherein the position set is used to constitute a motion trajectory sample of the target single point; acquiring a plurality of groups of trajectory attribute feature samples corresponding to a plurality of trajectory segments in the motion trajectory sample according to the position set; determining that a target trajectory attribute feature sample can be used for single-point motion learning, wherein the target trajectory attribute feature sample is any one of the plurality of groups of trajectory attribute feature samples; and in response to the target trajectory attribute feature sample being available for single-point motion learning, performing single-point motion learning according to the target trajectory attribute feature sample to obtain an updated latest motion trajectory of the target single point; and the trajectory attribute feature set of the latest motion trajectory is used as a sample for next single-point motion learning. The technical scheme provided by the application has small calculation and requires less learning data.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to a single-point motion learning method and device, computer equipment and a computer readable storage medium. BACKGROUND

[0002] Skill imitation and migration of robots can be achieved through machine learning, fixed trajectory, etc. Machine learning can generally have three types: motion model, strategy learning model and semantic reasoning model, and motion planning is achieved based on the above models.

[0003] However, current skill imitation and migration through machine learning requires a large amount of learning data and model solving process, and the calculation amount is large. SUMMARY

[0004] An embodiment of the present application provides a single-point motion learning method, device, computer equipment and computer readable storage medium, which can be used to solve the problems described above.

[0005] One aspect of an embodiment of the present application provides a single-point motion learning method, comprising:

[0006] Obtaining a position set of a target single point in a video, the position set being used to constitute a motion trajectory sample of the target single point;

[0007] According to the position set, a plurality of groups of trajectory attribute feature samples corresponding to a plurality of trajectory segments in the motion trajectory sample are obtained;

[0008] Determining that the target trajectory attribute feature sample can be used for single-point motion learning, the target trajectory attribute feature sample being any one of the plurality of groups of trajectory attribute feature samples; and

[0009] In response to the target trajectory attribute feature sample being available for single-point motion learning, performing single-point motion learning according to the target trajectory attribute feature sample to obtain an updated motion trajectory of the target single point; wherein a trajectory attribute feature set of the updated motion trajectory is used as a sample for next single-point motion learning.

[0010] Optionally, the obtaining, according to the position set, a plurality of groups of trajectory attribute feature samples corresponding to a plurality of trajectory segments in the motion trajectory sample comprises:

[0011] According to the position set, a trajectory attribute feature sample set of the motion trajectory sample is obtained;

[0012] The trajectory attribute feature sample set is divided into the plurality of groups of trajectory attribute feature samples in time sequence.

[0013] Optionally, the trajectory attribute feature comprises a slope of adjacent positions.

[0014] The arranging in time sequence comprises:

[0015] The trajectory attribute feature sample set is sliced to obtain M groups of trajectory attribute feature samples, M being a natural number greater than 1.

[0016] Each group of trajectory attribute feature samples comprises one or more slopes.

[0017] When the value of the slope is zero, the slope with the value of zero is divided into a group of separate trajectory attribute feature samples.

[0018] Optionally, the target trajectory attribute feature sample comprises one or more trajectory attribute features; and the determining that the target trajectory attribute feature sample can be used for single-point motion learning comprises:

[0019] Obtaining a Gaussian distribution of the target single point in a historical learning library;

[0020] In response to each trajectory attribute feature in the target trajectory attribute feature sample falling within a preset range of the Gaussian distribution, determining that the target trajectory attribute feature sample can be used for the single-point motion learning;

[0021] In response to at least one trajectory attribute feature in the target trajectory attribute feature sample not falling within the preset range of the Gaussian distribution, determining that the target trajectory attribute feature sample cannot be used for the single-point motion learning.

[0022] Optionally, the single-point motion learning according to the target trajectory attribute feature sample comprises:

[0023] Determining a historical trajectory attribute feature of the target single point; and

[0024] Updating a latest motion trajectory of the target single point according to the historical trajectory attribute feature and the trajectory attribute features in the target trajectory attribute feature sample; a trajectory attribute feature set of the latest motion trajectory comprises the trajectory attribute features learned this time and is used as a sample for next single-point motion learning.

[0025] Optionally, the method further comprises:

[0026] Using the trajectory attribute feature set of the latest motion trajectory obtained each time for single-point motion learning until the trajectory attribute feature set of the latest motion trajectory obtained after the last update cannot be used for single-point motion learning.

[0027] Optionally, further comprising:

[0028] The trajectory attribute feature set of the latest motion trajectory obtained each time of updating is used for single-point motion learning until the number of updates exceeds a preset number.

[0029] Optionally, further comprising:

[0030] The trajectory attribute feature set of the latest motion trajectory obtained each time of updating is used for single-point motion learning until the learning effect is lower than a preset threshold.

[0031] An aspect of an embodiment of the present application further provides a single-point motion learning device, comprising:

[0032] A first obtaining module is configured to obtain a position set of a target single point in a video, the position set being used to constitute a motion trajectory sample of the target single point;

[0033] A second obtaining module is configured to obtain, according to the position set, a plurality of groups of trajectory attribute feature samples corresponding to a plurality of trajectory segments in the motion trajectory sample;

[0034] A determining module is configured to determine that a target trajectory attribute feature sample can be used for single-point motion learning, the target trajectory attribute feature sample being any one of the plurality of groups of trajectory attribute feature samples; and

[0035] A learning module is configured to, in response to the target trajectory attribute feature sample being able to be used for single-point motion learning, perform single-point motion learning according to the target trajectory attribute feature sample to obtain a latest motion trajectory of the target single point; wherein a trajectory attribute feature set of the latest motion trajectory is used as a sample for next single-point motion learning.

[0036] An aspect of an embodiment of the present application further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to implement the steps of the single-point motion learning method.

[0037] An aspect of an embodiment of the present application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executable by at least one processor to cause the at least one processor to execute the steps of the single-point motion learning method.

[0038] The single-point motion learning method, device, computer device, and computer readable storage medium provided by the embodiments of the present application have the following advantages:

[0039] (1) In the present application, the motion trajectory of a single point is constantly modified based on a set of positions of the single point to achieve single-point motion learning, effectively reducing the learning data and avoiding the model solving process to save the amount of calculation, relative to the large amount of learning data and the model solving process required for motion trajectory learning imitation by machine learning.

[0040] (2) The motion trajectory of a video is divided into multiple trajectory segments, each trajectory segment corresponds to a set of trajectory attribute feature samples, so single-point motion learning is performed according to the trajectory segments. It is necessary to determine whether the trajectory attribute feature samples of each trajectory segment can be used for single-point motion learning. Unlike determining whether the trajectory description set of the entire motion trajectory can be used for single-point motion learning at one time, judging by trajectory segment by segment can avoid the problem that "once there is an abnormal trajectory attribute feature, the entire video trajectory is discarded".

[0041] The present application adopts a trajectory segment splitting strategy, the trajectory attribute feature samples of useful trajectory segments can be used for single-point motion learning, and the trajectory segments containing abnormal trajectory attribute features are discarded, fully utilizing the useful data of the video as samples.

[0042] (3) The learned and updated trajectory attribute features can be used as samples for single-point motion learning again, and the single-point motion learning by the cyclic learning strategy obtains more accurate results, indirectly reducing the dependence of learning on the amount of data.

[0043] In summary, the present application effectively reduces the learning data and avoids the model solving process to save the amount of calculation, adopts the trajectory segment splitting strategy to fully utilize the data, and the cyclic learning strategy reduces the dependence of learning on the amount of data. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 The application environment diagram of the single-point motion learning method according to the embodiment of the present application is schematically shown;

[0045] Figure 2 The flowchart of the single-point motion learning method according to the first embodiment of the present application is schematically shown;

[0046] Figure 3 The sub-flowchart of step S202 is schematically shown;

[0047] Figure 4 The sub-flowchart of step S204 is schematically shown;

[0048] Figure 5 The flowchart of an exemplary application is schematically shown;

[0049] Figure 6 The block diagram of the single-point motion learning device according to the second embodiment of the present application is schematically shown;

[0050] Figure 7 Fig. 1 shows a schematic diagram of a hardware architecture of a computer device suitable for implementing the single-point motion learning method according to Embodiment Three of the present application. DETAILED DESCRIPTION

[0051] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0052] It should be noted that the description of "first", "second" and the like in the embodiments of the present application is only for the purpose of description and should not be understood as indicating or implying the relative importance of the technical features indicated or implying the number of technical features indicated. Therefore, the features limited by "first", "second" can explicitly or implicitly include at least one of the features. In addition, the technical solutions of each embodiment can be combined with each other, but it must be based on the fact that a person skilled in the art can realize it. When the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist and is not within the scope of protection claimed by the present application.

[0053] In the description of the present application, it should be understood that the numerical reference before the step does not indicate the order of execution of the steps before and after the step, but is only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as limiting the present application.

[0054] The following is the explanation of the terms of the present application:

[0055] Single-point motion: the motion body can be divided into N parts according to the motion trajectory S, and all components in each part move according to the trajectory S, so this part can be regarded as a point, and the motion of this point is called single-point motion.

[0056] Machine learning: is a multi-disciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It is specially designed to study how computers simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structure to continuously improve their performance. Compared with data mining, which finds mutual characteristics between big data, machine learning pays more attention to the design of algorithms, so that computers can automatically "learn" the rules from data and use the rules to predict unknown data.

[0057] In order to facilitate those skilled in the art to understand the technical solutions provided by the embodiments of the present application, the related art will be described as follows:

[0058] The skill imitation or skill migration of the object can be implemented in the following ways:

[0059] (1) Machine learning:

[0060] a. Strategy model: represented by apprentice learning, maximum marginal planning, inverse reinforcement learning and generative adversarial imitation learning.

[0061] b. Motion model: represented by traditional teaching learning methods such as DMP (Dynamic Movement Primitives), GMM (Gaussian Mixture Model) and GMR (Gaussian Mixture Regression).

[0062] c. Inference model: segment the task, determine the key nodes, and extract the logical semantics of each segment node.

[0063] (2) Fixed trajectory: a predetermined skill motion trajectory is given, and the object runs according to the fixed motion trajectory.

[0064] However, the above implementation methods have various defects, such as:

[0065] (1) The machine skill imitation model based on machine learning requires a large amount of data to enable the machine to learn a skill, and a small amount of data cannot solve the model, and the calculation amount is large.

[0066] (2) Fixed trajectory: single input motion trajectory may have problems, which cannot be corrected by learning after the problem occurs, and must be manually edited and replaced with the wrong trajectory to correct it.

[0067] (3) The input motion sample is input as a whole, and once there is an exception at a certain point, the entire sample is discarded. However, actual motion samples may have correct and incorrect parts, and discarding them uniformly will also lose the correct part.

[0068] In view of the above problems, the present application aims to provide a single-point motion learning scheme to solve the above problems. Specifically:

[0069] (1) Remove the model solving process, and the required training (learning) data is relatively less than normal machine learning. And the specific model also has learning ability.

[0070] (2) The input sample will be split into sub-samples for learning, which can make full use of the useful data of the sample.

[0071] (3) According to the algorithm specificity, the sub-sample can be reused for training to obtain more accurate results, which indirectly reduces the dependence of training on the data volume.

[0072] (4) Based on the time slice, the trajectory feature attribute is extracted, so that the trajectory has the constraint of change rate in addition to the shape constraint (that is, discard the time scale variable trajectory, such as dance needs fast and slow rhythm).

[0073] An exemplary application environment of the present application is provided below. As shown in Figure 1 The environment schematic diagram can include a data source 100, a network 200, and a server 300. The data source 100 can be located in a data center such as a single site, or a distributed database distributed in different geographical locations (for example, in multiple sites), or a terminal device. The network 200 includes various network devices, such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, proxy devices, and / or the like. The network 9 can include physical links, such as coaxial cable links, twisted pair cable links, fiber optic links, combinations thereof, and the like. The network 200 can also include wireless links, such as cellular links, satellite links, Wi-Fi links, and the like.

[0074] The data source 100 is used to provide various learning data, such as video samples, etc.

[0075] The server 300 can provide various services, for example, the server 300 can obtain learning data from the data source 100, and perform motion trajectory learning (such as unit motion learning) according to the learning data, to realize trajectory imitation and migration.

[0076] It should be noted that Figure 1 The number of data sources 100, networks 200, and servers 300 shown in the above-mentioned is only schematic. According to the needs, there can be any number of data sources, networks and servers. It should be noted that in the case of storing learning data in the server 300, the data source 300 can also not be set.

[0077] Below, several embodiments will be provided under the above-mentioned exemplary application environment to illustrate the single-point motion learning scheme.

[0078] Embodiment One

[0079] Figure 2 The flowchart of the single-point motion learning method according to the embodiment one of the present application is schematically shown.

[0080] As shown in Figure 2 The single-point motion learning method can include steps S200-S206, wherein:

[0081] Step S200, the position set of the target single point in the video is obtained, and the position set is used to constitute a motion trajectory sample of the target single point.

[0082] The video can be used as a sample video for single-point motion learning.

[0083] The video can be various videos, such as dance or action videos of a person, or action videos of animals and the like.

[0084] For example, the video can be a recording video of a live room, and the recording video records dance content of an anchor.

[0085] If an AI robot of an anchor in a live room is to be made, the dance of the anchor can be learned through single-point motion learning. Specifically, each bone node of the AI robot can be regarded as a single point, and the overall motion is a series of single-point actions in linkage. The AI robot composed of the single points can learn the dance or action of the anchor by using the dance video.

[0086] Taking the above video as an example, the identification process is as follows:

[0087] (1) The target object (dancing anchor) in each frame of the video can be identified through a picture recognition algorithm. The image recognition algorithm is used to process, analyze and understand images to identify various different patterns of targets and objects.

[0088] (2) If the dancing anchor is identified in a frame, the position of the target single point (such as the top of the head of the dancing anchor) of the dancing anchor is obtained.

[0089] (3) According to the order of each frame in the video and the position of the target single point in each frame, a position set is generated.

[0090] For example, the position of the target single point in each frame is determined as P i (X i ,Y i ), the positions of the single point i in each frame are arranged in time sequence to generate a position set P = {P0, P1, P2, …, Pn}. An interval frame number D can also be set, and the average value of each component in the continuous D frames is calculated as a single point P', and a position set P' = {P0', P1', P2', …, Pn'} is generated. Wherein, i represents the sorting number in time sequence, X represents the horizontal axis, and Y represents the vertical axis.

[0091] For example, the human body bone nodes can include: right shoulder, right elbow, right wrist, left shoulder, left elbow, left wrist, right hip, right knee, right ankle, left hip, left knee, left ankle, top of the head, neck. Of course, other bone nodes can also be included.

[0092] Taking one of the skeleton nodes (e.g. the head top) as an example, the head top can be taken as a single point, and a position set corresponding to the head top (arranging the positions of the head top in each frame in the order of the frames in the video, from front to back) can be obtained. The position set corresponding to the head top can form / construct a motion trajectory of the head top in the video.

[0093] Step S202 According to the position set, a plurality of groups of trajectory attribute feature samples corresponding to a plurality of trajectory segments in the motion trajectory sample can be obtained.

[0094] The position set constitutes the overall motion trajectory of the target single point (e.g. the head top).

[0095] Based on the relationship (e.g. the slope) between each position or adjacent positions in the position set, a trajectory attribute feature can be obtained.

[0096] That is, according to the position set, a trajectory attribute feature for describing the trajectory trend can be obtained.

[0097] In this embodiment, a trajectory segment splitting strategy is adopted to split the overall motion trajectory in the video into a plurality of trajectory segments. Thus, according to the position set, a plurality of groups of trajectory attribute feature samples corresponding to a plurality of trajectory segments can be obtained. Based on the trajectory segment splitting strategy, a single video can be split as a plurality of samples, and each trajectory segment corresponds to a sample for single point motion learning, so that the useful data of the video can be fully mined and utilized.

[0098] In an optional embodiment, as shown in Figure 3 Step S202 "according to the position set, obtaining a plurality of groups of trajectory attribute feature samples corresponding to a plurality of trajectory segments in the motion trajectory sample" can be obtained by the following steps: step S300, obtaining a trajectory attribute feature sample set of the motion trajectory sample according to the position set; and step S302, arranging in time sequence, and cutting the trajectory attribute feature sample set into the plurality of groups of trajectory attribute feature samples. For example: according to the position set P={P0, P1, P2, …, Pn}, a corresponding trajectory attribute feature sample set S={S0, S1, S2, …, Sn} can be obtained. Then, a threshold X is set, and S is cut into n / X segments (a sample is generated for each trajectory segment). By being cut into the plurality of groups of trajectory attribute feature samples, the data can be fully utilized for single point motion learning.

[0099] In another optional embodiment, the trajectory attribute feature (sample) includes the slope of adjacent positions.

[0100] With a position set P = {P0, P1, P2, …, Pn}, the slopes between adjacent positions (e.g., P1, P2) are calculated to obtain S = {S0, S1, S2, …, Sn}. S0 corresponds to the slope between P0 and P1, S1 corresponds to the slope between P1 and P2, and so on.

[0101] Step S302 "arranging the trajectory attribute feature sample set in time sequence, and dividing the trajectory attribute feature sample set into the multiple groups of trajectory attribute feature samples" includes: slicing the trajectory attribute feature sample set to obtain M groups of trajectory attribute feature samples, M being a natural number greater than 1; each group of trajectory attribute feature samples includes one or more slopes; when the value of a slope is zero, the slope with the value of zero is divided into a separate group of trajectory attribute feature samples. The number of trajectory attribute features in each trajectory attribute feature sample is ≥1, and each trajectory attribute feature sample does not include a trajectory attribute feature with a slope of 0. When Si is 0, the trajectory attribute feature forms a trajectory attribute feature sample by itself. The trajectory attribute feature with a slope of 0 can be used as an extreme value for preset verification.

[0102] Step S204 determining that the target trajectory attribute feature sample can be used for single-point motion learning, the target trajectory attribute feature sample being any one of the multiple groups of trajectory attribute feature samples.

[0103] The target trajectory attribute feature sample includes one or more trajectory attribute features (e.g., slopes).

[0104] Since different trajectory attribute feature samples correspond to different trajectory segments, single-point motion learning is performed according to trajectory segments. In implementation, it is necessary to determine whether the trajectory attribute feature sample of each trajectory segment can be used for single-point motion learning. Unlike determining whether the trajectory description set of the entire motion trajectory can be used for single-point motion learning at one time, determining by trajectory segment can avoid the problem that "once there is an abnormal point (trajectory attribute feature) in the overall determination of the entire motion trajectory, the trajectory of the entire video is discarded".

[0105] In implementation, it can be determined according to a Gaussian distribution function or other abnormal monitoring function whether the trajectory attribute feature is an expected normal value. If yes, it means that the trajectory attribute feature can be used for single-point motion learning, otherwise, it means that the trajectory attribute feature cannot be used for single-point motion learning.

[0106] In an optional embodiment, in order to quickly and effectively detect abnormalities, the following steps can be performed. Figure 4As shown, the step 204 "determining that the target trajectory attribute feature sample can be used for single-point motion learning" can be implemented by the following steps: step S400, obtaining a Gaussian distribution of the target single point in the historical learning library; step S402, in response to each trajectory attribute feature in the target trajectory attribute feature sample falling within a preset range of the Gaussian distribution, determining that the target trajectory attribute feature sample can be used for the single-point motion learning; step S404, in response to at least one trajectory attribute feature in the target trajectory attribute feature sample not falling within the preset range of the Gaussian distribution, determining that the target trajectory attribute feature sample cannot be used for the single-point motion learning.

[0107] For example, the trajectory attribute features in the target trajectory attribute feature sample are S={S0, S1, S2, …, Sn}, each Si is sequentially traversed. First, the Gaussian distribution function f(x)=(x, μ, δ) of the target single point in the historical learning library is obtained, and it is detected whether Si is within the f(x)≤M region. If not, the target trajectory attribute feature sample is discarded. If all Si in the trajectory segment pass the detection, it is indicated that the target trajectory attribute feature sample corresponding to the trajectory segment can be used for single-point motion learning. μ represents the mean value, δ represents the variance, and m is an empirical value. Through the historical learning library and the Gaussian distribution, the trajectory attribute feature sample that can be used for single-point motion learning can be effectively determined.

[0108] Step S206 In response to the target trajectory attribute feature sample being used for single-point motion learning, single-point motion learning is performed according to the target trajectory attribute feature sample, to obtain an updated latest motion trajectory of the target single point; wherein a trajectory attribute feature set of the latest motion trajectory is used as a sample for next single-point motion learning.

[0109] In the case where the trajectory attribute features in the target trajectory attribute feature sample can be used for single-point motion learning, new trajectory attribute features can be generated according to the trajectory attribute features in the target trajectory attribute feature sample.

[0110] In an optional embodiment, in order to effectively learn and correct the motion trajectory, as shown in FIG. 6, the method comprises the following steps: Figure 4As shown, the step S206 "in response to the target trajectory attribute feature sample being available for single-point motion learning, performing single-point motion learning according to the target trajectory attribute feature sample to obtain the latest motion trajectory of the target single point" can be implemented by the following steps: step S400, determining the historical trajectory attribute feature of the target single point; and step S402, updating the latest motion trajectory of the target single point according to the historical trajectory attribute feature and the trajectory attribute feature in the target trajectory attribute feature sample; the trajectory attribute feature set of the latest motion trajectory includes the trajectory attribute feature learned this time and is used as a sample for next single-point motion learning.

[0111] Using the trajectory attribute feature set of the latest motion trajectory as a sample for next single-point motion learning, repeatedly using the sample for single-point motion learning to obtain more accurate results, indirectly reducing the dependence of learning on data volume.

[0112] For example, the trajectory attribute feature in the target trajectory attribute feature sample is S={S0, S1, S2, …, Sn}.

[0113] The trajectory attribute feature set of the latest motion trajectory is T={T0, T1, T2, …, Tn}.

[0114] Each trajectory attribute feature of the trajectory attribute feature set of the latest motion trajectory is Ti=(θ0 * Ti+θ1 * Si) / 2.

[0115] That is, the updated trajectory attribute feature Ti is calculated by weighted average of the last learned trajectory attribute feature Ti and the current trajectory attribute feature Si. Each set of trajectory attribute feature samples can update the trajectory attribute feature T once.

[0116] θ1=1-θ0.

[0117] θ0=α * 1 / (1+e -t ).

[0118] Wherein:

[0119] α is a preset value;

[0120] θ1 is the weighted value of the current trajectory attribute feature Si;

[0121] θ0 is the weighted value of the last learned trajectory attribute feature Ti;

[0122] t is the number of repeated learning of the same trajectory segment.

[0123] As the weight setting described above, the more the number of repeated learning, the less the useful data left, and thus the smaller the weight θ1.

[0124] In order to ensure effective learning and prevent over-learning or learning that cannot be terminated, several ways of terminating the number of repeated learning are provided as follows.

[0125] In an optional embodiment, the method can further include using the trajectory attribute feature set of the latest motion trajectory obtained after each update for single-point motion learning until the trajectory attribute feature set of the latest motion trajectory obtained after the last update is not available for single-point motion learning. In this optional embodiment, the sample is repeatedly iterated and single-point motion learning is repeatedly performed based on the iterated sample, so that more accurate results can be obtained, indirectly reducing the dependence of training on the amount of data. In addition, in order to ensure effective learning and prevent over-learning or learning that cannot be terminated, when the iterated sample is not suitable for learning, the iteration of the sample based on the current trajectory segment is terminated, and the trajectory attribute feature sample of the next trajectory segment is used as a new learning sample for single-point motion learning.

[0126] In an optional embodiment, the method can further include using the trajectory attribute feature set of the latest motion trajectory obtained after each update for single-point motion learning until the number of updates exceeds a preset number. The sample is repeatedly iterated and single-point motion learning is repeatedly performed based on the iterated sample, so that more accurate results can be obtained, indirectly reducing the dependence of training on the amount of data. In addition, in order to ensure effective learning and prevent over-learning or learning that cannot be terminated, when the number of repeated training is more than the preset number, the iteration of the sample based on the current trajectory segment is terminated, and the trajectory attribute feature sample of the next trajectory segment is used as a new learning sample for single-point motion learning.

[0127] In an optional embodiment, the method can further include using the trajectory attribute feature set of the latest motion trajectory obtained after each update for single-point motion learning until the learning effect is lower than a preset threshold. The learning effect can be manually evaluated or script evaluated. The sample is repeatedly iterated and single-point motion learning is repeatedly performed based on the iterated sample, so that more accurate results can be obtained, indirectly reducing the dependence of training on the amount of data. In addition, in order to ensure effective learning and prevent over-learning or learning that cannot be terminated, when the learning effect after a certain iteration learning is lower than the preset threshold, the iteration of the sample based on the current trajectory segment is terminated, and the trajectory attribute feature sample of the next trajectory segment is used as a new learning sample for single-point motion learning.

[0128] As Figure 5 shown, in order to make the technical solutions and technical effects of the present application better clear and easy to understand, an exemplary application is provided as follows.

[0129] S500, input a video, and extract a position set P = {P0, P1, P2, …, Pn} of a target single point from the video at a time interval T.

[0130] According to the chronological order, the position set can be a motion trajectory of the target single point in the video.

[0131] S502, calculate a slope Si between adjacent points Pi and Pi+1 in a unit time T to generate a trajectory attribute feature sample set S = {S0, S1, S2, …, Sn}.

[0132] S504, set a threshold X, and divide the trajectory attribute feature sample set S into n / X trajectory segments (each segment corresponds to a learning sample).

[0133] S506, determine whether the trajectory attribute feature sample of the input trajectory segment can be used for single point motion learning.

[0134] Specifically, whether the input Si is an expected normal value can be detected according to a Gaussian anomaly detection function.

[0135] For example, the trajectory attribute feature sample is S = {S0, S1, S2, …, Sn}, and each Si is sequentially traversed. First, a Gaussian distribution function f(x) = (x, μ, δ) of the target single point in the historical learning library is obtained, and whether Si is in the f(x) ≤ M region is detected. If not, the trajectory attribute feature sample corresponding to the trajectory segment is discarded. If all Si in the trajectory segment pass the detection, it is indicated that the trajectory attribute feature sample corresponding to the trajectory segment can be used for single point motion learning.

[0136] S508, update the slope T by weighted average according to the historical slope T and the slope S of the target single point, that is, generate a new learning trajectory.

[0137] For example, the input trajectory attribute feature sample is S = {S0, S1, S2, …, Sn}.

[0138] The learned and updated trajectory attribute feature set is T = {T0, T1, T2, …, Tn}.

[0139] Each trajectory attribute feature Ti learned and updated is Ti = (θ0 * Ti+θ1 * Si) / 2.

[0140] The trajectory attribute feature Ti is updated by weighted average according to the trajectory attribute feature Ti obtained by the target single point last time and the current trajectory attribute feature Si. Each group of trajectory attribute feature samples can update a round of trajectory attribute feature T.

[0141] S510, evaluate the learning effect of the trajectory segment, re-learn until no useful information can be extracted.

[0142] For example: according to user feedback to get learning score to determine whether the learning is effective. If effective, continue learning. If learning fails, determine whether the current trajectory segment has been sliced X times. If the number of slicing exceeds X times, the learning of the trajectory segment ends. If not, the trajectory segment is sliced again for learning until the learning ends.

[0143] The following provides an example of repeated learning:

[0144] The historical trajectory attribute features of the target single point T = {T0, T1, T2, …, Tn};

[0145] (1) First learning:

[0146] The input trajectory attribute feature sample is S = {S0, S1, S2, …, Sn}. When each element of the sample is preset normal, it is used as a learning sample for first learning. The learning process is: according to T = {T0, T1, T2, …, Tn} and S = {S0, S1, S2, …, Sn}, weighting is performed to obtain updated trajectory attribute features Ta.

[0147] That is: T and S → Ta.

[0148] (2) Second learning:

[0149] The input trajectory attribute feature sample is Ta. When each element of the sample is preset normal, it is used as a learning sample for second learning. The learning process is: according to T and Ta, weighting is performed to obtain updated trajectory attribute features Tb.

[0150] That is: T and Ta → Tb.

[0151] (2) Third learning:

[0152] The input trajectory attribute feature sample is Tb. When each element of the sample is preset normal, it is used as a learning sample for third learning. The learning process is: according to T and Tb, weighting is performed to obtain updated trajectory attribute features Tc.

[0153] That is: T and Tb → Tc.

[0154] By analogy, until the multiple repeated learning starting from the trajectory attribute sample S ends. If the trajectory attribute features obtained by multiple repeated learning are Tx, then Tx replaces T to be used as the historical trajectory attribute features for subsequent training.

[0155] The embodiment has the following advantages:

[0156] (1) In the embodiment, the motion trajectory of a single point is constantly modified based on a set of positions of the single point to achieve single-point motion learning, effectively reducing the learning data and avoiding model solving process to save computation.

[0157] (2) The set of trajectory attribute features S is divided into n / X trajectory segments (each segment corresponds to a learning sample), and different trajectory attribute feature samples correspond to different trajectory segments, so single-point motion learning is performed by trajectory segment. In implementation, it is necessary to determine whether the trajectory attribute feature sample of each trajectory segment can be used for single-point motion learning. Unlike determining whether the trajectory description set of the entire motion trajectory can be used for single-point motion learning at one time, judging by trajectory segment by segment can avoid the problem that "once there is an abnormal point (trajectory attribute feature) in the overall judgment of the entire motion trajectory, the trajectory of the entire video is discarded". In the embodiment, the trajectory segment splitting strategy is adopted, the trajectory attribute feature sample of the useful trajectory segment can be used for single-point motion learning, and the trajectory segment containing abnormal trajectory attribute features is discarded, and the useful data of the video is fully utilized as a sample.

[0158] (3) The learned and updated trajectory attribute feature T can be used as a sample for single-point motion learning again, and the single-point motion learning by the cyclic learning strategy can obtain more accurate results, indirectly reducing the dependence of learning on data volume.

[0159] The single-point motion learning of the embodiment can continuously learn and modify the trajectory without the need for manual editing of the motion trajectory.

[0160] (4) Based on the trajectory description feature (the position of the target single point in the frame) extracted by the unit time T, the trajectory has a constraint on the change rate in addition to the shape constraint (i.e., discarding time-scale variable trajectories, such as dance requiring fast and slow beats).

[0161] (5) By using the Gaussian distribution function, it can be effectively detected whether the current input trajectory attribute feature is a preset normal value. Specifically: by f(x)≤M judgment, M is the attenuation value. The more times the trajectory attribute feature corresponding to the same trajectory segment is updated and input as a sample, the lower the value of M. That is: the more times the same trajectory segment is repeatedly learned, the less useful data is left, so a strategy for discarding the trajectory segment needs to be developed, such as using the attenuation of the M value to discard the trajectory segment to continue single-point motion learning with the next trajectory segment.

[0162] Embodiment two

[0163] Figure 6A block diagram of a single-point motion learning device according to Embodiment Two of the present application is shown schematically. The single-point motion learning device can be divided into one or more program modules, which are stored in a storage medium and executed by one or more processors to complete the embodiments of the present application. The program module referred to in the embodiments of the present application refers to a series of computer program instruction segments capable of completing a specific function, and the functions of the program modules in the embodiments will be described in detail below. As shown in Figure 6 The single-point motion learning device 600 can include a first acquisition module 610, a second acquisition module 620, a determination module 630, and a learning module 640, wherein:

[0164] The first acquisition module 610 is configured to acquire a set of positions of a target single point in a video, the set of positions being used to constitute a motion trajectory sample of the target single point.

[0165] The second acquisition module 620 is configured to acquire, according to the set of positions, a plurality of sets of trajectory attribute feature samples corresponding to a plurality of trajectory segments in the motion trajectory sample.

[0166] The determination module 630 is configured to determine that a target trajectory attribute feature sample is available for single-point motion learning, the target trajectory attribute feature sample being any one of the plurality of sets of trajectory attribute feature samples.

[0167] The learning module 640 is configured to, in response to the target trajectory attribute feature sample being available for single-point motion learning, perform single-point motion learning according to the target trajectory attribute feature sample to update a latest motion trajectory of the target single point, wherein a set of trajectory attribute features of the latest motion trajectory is used as a sample for next single-point motion learning.

[0168] In an optional embodiment, the second acquisition module 620 is further configured to:

[0169] acquire, according to the set of positions, a set of trajectory attribute feature samples of the motion trajectory sample;

[0170] arrange in time sequence, and cut the set of trajectory attribute feature samples into the plurality of sets of trajectory attribute feature samples.

[0171] In an optional embodiment, the trajectory attribute feature includes a slope of adjacent positions.

[0172] The second acquisition module 620 is further configured to:

[0173] slice the set of trajectory attribute feature samples to obtain M sets of trajectory attribute feature samples, M being a natural number greater than 1;

[0174] Each set of trajectory attribute feature samples includes one or more slopes.

[0175] wherein, when the value of the slope is zero, the slope with the value of zero is cut into a set of separate trajectory attribute feature samples.

[0176] In an optional embodiment, the target trajectory attribute feature sample includes one or more trajectory attribute features; the determining module 630 is further configured to:

[0177] obtain a Gaussian distribution of the target single point in the historical learning library;

[0178] in response to each trajectory attribute feature in the target trajectory attribute feature sample falling within a preset range of the Gaussian distribution, determine that the target trajectory attribute feature sample is available for the single point motion learning;

[0179] in response to at least one trajectory attribute feature in the target trajectory attribute feature sample not falling within the preset range of the Gaussian distribution, determine that the target trajectory attribute feature sample is unavailable for the single point motion learning.

[0180] In an optional embodiment, the learning module 640 is configured to:

[0181] determine a historical trajectory attribute feature of the target single point; and

[0182] update a latest motion trajectory of the target single point according to the historical trajectory attribute feature and the trajectory attribute features in the target trajectory attribute feature sample; a trajectory attribute feature set of the latest motion trajectory includes the trajectory attribute features learned this time and is used as a sample for next single point motion learning.

[0183] In an optional embodiment, the apparatus further includes a repeating module configured to:

[0184] use the trajectory attribute feature set of the latest motion trajectory obtained each time for single point motion learning until the trajectory attribute feature set of the latest motion trajectory obtained the last time is unavailable for single point motion learning.

[0185] In an optional embodiment, the apparatus further includes a repeating module configured to:

[0186] use the trajectory attribute feature set of the latest motion trajectory obtained each time for single point motion learning until the number of updates exceeds a preset number of times.

[0187] In an optional embodiment, the apparatus further includes a repeating module configured to:

[0188] use the trajectory attribute feature set of the latest motion trajectory obtained each time for single point motion learning until the learning effect is lower than a preset threshold.

[0189] Embodiment Three

[0190] Figure 7 A schematic diagram of a hardware architecture of a computer device 10000 suitable for implementing the single-point motion learning method according to Embodiment Three of the present application is shown. The computer device 10000 can be the server 300 or a part of the server 300, or a terminal device. The computer device 10000 is a device capable of automatically performing numerical calculation and / or information processing according to pre-set or stored instructions. For example, it can be a smart phone, a tablet computer, a PC, a virtual reality device, etc. As shown, the computer device 10000 at least includes, but is not limited to, a memory 10010, a processor 10020, and a network interface 10030 which are communicatively linked through a system bus. Among them: Figure 7

[0191] The memory 10010 at least includes one type of computer readable storage medium, which includes a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 10010 can be an internal storage module of the computer device 10000, such as a hard disk or a memory of the computer device 10000. In other embodiments, the memory 10010 can also be an external storage device of the computer device 10000, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device 10000. Of course, the memory 10010 can also include both the internal storage module and the external storage device of the computer device 10000. In this embodiment, the memory 10010 is generally used to store the operating system and various application software installed on the computer device 10000, such as the program code of the single-point motion learning method, etc. In addition, the memory 10010 can also be used to temporarily store various data that have been output or will be output.

[0192] ​The processor 10020 may, in some embodiments, be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 10020 is generally used to control the overall operation of the computer device 10000, such as performing control and processing related to data interaction or communication of the computer device 10000, etc. In the present embodiment, the processor 10020 is used to run program codes or process data stored in the memory 10010.

[0193] The network interface 10030 can include a wireless network interface or a wired network interface, and is generally used to establish a communication link between the computer device 10000 and other computer devices. For example, the network interface 10030 is used to connect the computer device 10000 with an external user terminal through a network, establish a data transmission channel and a communication link between the computer device 10000 and the external user terminal, etc. The network can be an intranet, the Internet, a Global System for Mobile Communications (GSM), a Wideband Code Division Multiple Access (WCDMA), a 4G network, a 5G network, Bluetooth, Wi-Fi, or other wireless or wired networks.

[0194] It should be noted that, Figure 7 Only a computer device with components 10010-10030 is shown, but it should be understood that it is not required to implement all of the illustrated components, and more or fewer components can be implemented instead.

[0195] In the present embodiment, the single-point motion learning method stored in the memory 10010 can also be divided into one or more program modules and executed by one or more processors (the processor 10020 in the present embodiment) to complete the embodiments of the present application.

[0196] Embodiment Four

[0197] The present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the single-point motion learning method in Embodiment One.

[0198] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the computer-readable storage medium can be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device. Of course, the computer-readable storage medium can also include both the internal storage unit and the external storage device of the computer device. In this embodiment, the computer-readable storage medium is typically used to store the operating system and various application software installed on the computer device, such as the program code of the single-point motion learning method in this embodiment. In addition, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or will be output.

[0199] Obviously, those skilled in the art should understand that the modules or steps of the embodiments of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of this application are not limited to any particular combination of hardware and software.

[0200] It should be noted that the above are merely preferred embodiments of this application and do not limit the scope of patent protection of this application. Any equivalent structural or procedural changes made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of this application.

Claims

1. A single point motion learning method, characterized by, The method comprises the following steps: acquiring a position set of a target single point in a video, the position set being used to constitute a motion trajectory sample of the target single point; acquiring, according to the position set, a plurality of groups of trajectory attribute feature samples corresponding to a plurality of trajectory segments in the motion trajectory sample; determining that a target trajectory attribute feature sample can be used for single point motion learning, the target trajectory attribute feature sample being any one of the plurality of groups of trajectory attribute feature samples; and in response to the target trajectory attribute feature sample being usable for single point motion learning, performing single point motion learning according to the target trajectory attribute feature sample to update a latest motion trajectory of the target single point; wherein a trajectory attribute feature set of the latest motion trajectory is used as a sample for next single point motion learning. The acquiring, according to the position set, a plurality of groups of trajectory attribute feature samples corresponding to a plurality of trajectory segments in the motion trajectory sample comprises: acquiring a trajectory attribute feature sample set of the motion trajectory sample according to the position set; arranging the trajectory attribute feature sample set in time sequence and dividing the trajectory attribute feature sample set into the plurality of groups of trajectory attribute feature samples. The trajectory attribute feature comprises a slope of adjacent positions. The arranging the trajectory attribute feature sample set in time sequence and dividing the trajectory attribute feature sample set into the plurality of groups of trajectory attribute feature samples comprises: slicing the trajectory attribute feature sample set to obtain M groups of trajectory attribute feature samples, M being a natural number greater than 1; each group of trajectory attribute feature samples comprises one or more slopes; when the value of a slope is zero, the slope with the value of zero is divided into a group of separate trajectory attribute feature samples.

2. The method of claim 1, wherein, The target trajectory attribute feature sample comprises one or more trajectory attribute features; the determining that the target trajectory attribute feature sample can be used for single point motion learning comprises: acquiring a Gaussian distribution of the target single point in a historical learning library; in response to each trajectory attribute feature in the target trajectory attribute feature sample falling within a preset range of the Gaussian distribution, determining that the target trajectory attribute feature sample can be used for the single point motion learning; in response to at least one trajectory attribute feature in the target trajectory attribute feature sample not falling within the preset range of the Gaussian distribution, determining that the target trajectory attribute feature sample cannot be used for the single point motion learning.

3. The method of claim 1, wherein, The performing single point motion learning according to the target trajectory attribute feature sample to update a latest motion trajectory of the target single point comprises: determining a historical trajectory attribute feature of the target single point; and updating a latest motion trajectory of the target single point according to the historical trajectory attribute feature and the trajectory attribute features in the target trajectory attribute feature sample; a trajectory attribute feature set of the latest motion trajectory comprises the trajectory attribute features learned this time and is used as a sample for next single point motion learning.

4. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: using a trajectory attribute feature set of a latest motion trajectory obtained each time for single point motion learning until a trajectory attribute feature set of a latest motion trajectory obtained after the last time is not usable for single point motion learning.

5. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: The trajectory attribute feature set of the latest motion trajectory obtained each time of updating is used for single-point motion learning until the number of updates exceeds a preset number.

6. The method according to any one of claims 1 to 3, characterized in that, Further comprising: The trajectory attribute feature set of the latest motion trajectory obtained each time of updating is used for single-point motion learning until the learning effect is lower than a preset threshold.

7. A single point motion learning device characterized by, Comprising: The first acquisition module is configured to acquire a position set of the target single point in the video, and the position set is used to constitute a motion trajectory sample of the target single point. The second acquisition module is configured to acquire, according to the position set, a plurality of groups of trajectory attribute feature samples corresponding to a plurality of trajectory segments in the motion trajectory sample. The determination module is configured to determine that a target trajectory attribute feature sample can be used for single-point motion learning, the target trajectory attribute feature sample being any one of the plurality of groups of trajectory attribute feature samples. The learning module is configured to, in response to the target trajectory attribute feature sample being available for single-point motion learning, perform single-point motion learning according to the target trajectory attribute feature sample to obtain a latest motion trajectory of the target single point. The method comprises: According to the position set, a trajectory attribute feature sample set of the motion trajectory sample is acquired. The trajectory attribute feature sample set is divided into the plurality of groups of trajectory attribute feature samples in a time sequence. The trajectory attribute feature includes a slope of adjacent positions. The trajectory attribute feature sample set is sliced to obtain M groups of trajectory attribute feature samples, M being a natural number greater than 1. Each group of trajectory attribute feature samples includes one or more slopes. When the value of the slope is zero, the slope with the value of zero is divided into a separate group of trajectory attribute feature samples. The processor executes the computer program to implement the steps of the single-point motion learning method of any one of claims 1 to 6.

8. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The computer readable storage medium stores a computer program, which can be executed by at least one processor to enable the at least one processor to perform the steps of the single-point motion learning method of any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, ​

Images