Target posture recognition method and device, electronic equipment and readable storage medium

Abstract

The application discloses a target gesture recognition method and device, electronic equipment and a readable storage medium. The method comprises the following steps: acquiring a video to be recognized; wherein the video to be recognized comprises continuous video frames; performing target gesture prediction on the video to be recognized to obtain an initial gesture prediction result corresponding to each video frame; decomposing all the initial gesture prediction results to obtain at least two different frequency components; inputting each frequency component into a network model to obtain a sub-gesture prediction result corresponding to each frequency component; and obtaining a final gesture prediction result according to all the sub-gesture prediction results. In the foregoing manner, the accuracy of final target gesture recognition can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of pose recognition, in particular to a target pose recognition method and device, an electronic device and a readable storage medium. BACKGROUND

[0002] The related target pose estimation method is applied to a video scene, and the prediction result has significant jitter, and the reasons mainly include the following points:

[0003] 1) The target pose change in the video scene will be much faster than that in the static scene, which makes it difficult for the traditional target pose estimation model to accurately capture the rapid change in the video;

[0004] 2) The background noise in the video scene will affect the prediction result of the model;

[0005] 3) The target pose estimation itself lacks modeling capability for continuous motion, resulting in discontinuous time series prediction results. SUMMARY

[0006] The present application provides a target pose recognition method, device, electronic device and readable storage medium, which can improve the accuracy of final target pose recognition.

[0007] In a first aspect, the present application provides a target pose recognition method, which comprises: acquiring a to-be-recognized video; wherein the to-be-recognized video comprises continuous video frames; performing target pose prediction on the to-be-recognized video to obtain an initial pose prediction result corresponding to each video frame; decomposing all initial pose prediction results to obtain at least two different frequency components; inputting each frequency component into a network model to obtain a sub-pose prediction result corresponding to each frequency component; and obtaining a final pose prediction result according to all sub-pose prediction results.

[0008] In a second aspect, the present application provides a target pose recognition device, which comprises: an acquisition module configured to acquire a to-be-recognized video; wherein the to-be-recognized video comprises continuous video frames; a prediction module configured to perform target pose prediction on the to-be-recognized video to obtain an initial pose prediction result corresponding to each video frame; a decomposition module configured to decompose all initial pose prediction results to obtain at least two different frequency components; a processing module configured to input each frequency component into a network model to obtain a sub-pose prediction result corresponding to each frequency component; and obtain a final pose prediction result according to all sub-pose prediction results.

[0009] In a third aspect, the present application provides an electronic device, which comprises a processor, a memory and a display interface coupled to the processor; wherein the memory is configured to store a computer program, and the processor is configured to execute the computer program to implement the method provided in the first aspect.

[0010] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method provided in the first aspect is implemented.

[0011] The beneficial effects of the present application are as follows: Different from the prior art, the target gesture recognition method, device, electronic device and readable storage medium provided by the present application effectively suppress noise by decomposing all initial gesture prediction results into at least two different frequency components, and then inputting each frequency component into a network model to obtain a sub-gesture prediction result corresponding to each frequency component; the final gesture prediction result is obtained according to all sub-gesture prediction results, which can effectively remove the jitter of the initial gesture prediction result and improve the accuracy of the final target gesture recognition. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor. Among them:

[0013] Figure 1 is a flowchart of an embodiment of the target gesture recognition method provided by the present application;

[0014] Figure 2 is a flowchart of another embodiment of the target gesture recognition method provided by the present application;

[0015] Figure 3 is a flowchart of an embodiment of step 23 provided by the present application;

[0016] Figure 4 is a flowchart of another embodiment of the target gesture recognition method provided by the present application;

[0017] Figure 5 is a flowchart of an embodiment of the network model training method provided by the present application;

[0018] Figure 6 is a flowchart of another embodiment of the network model training method provided by the present application;

[0019] Figure 7 is a flowchart of another embodiment of the target gesture recognition method provided by the present application;

[0020] Figure 8 is a schematic diagram of an application scenario of the target gesture recognition method provided by the present application;

[0021] Figure 9 FIG. 1 is a structural schematic diagram of an embodiment of the target pose recognition device provided in the present application;

[0022] Figure 10 FIG. 2 is a structural schematic diagram of an embodiment of the electronic device provided in the present application;

[0023] Figure 11 FIG. 3 is a structural schematic diagram of another embodiment of the electronic device provided in the present application;

[0024] Figure 12 FIG. 4 is a structural schematic diagram of an embodiment of the computer readable storage medium provided in the present application. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the drawings in the embodiments of the present application. It can be understood that the specific embodiments described herein are only used to explain the present application, rather than limit the present application. In addition, it should be noted that, for the convenience of description, only the parts related to the present application are shown in the drawings, rather than all the structures. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0026] Reference to "embodiments" in this document means that the specific features, structures or characteristics described in conjunction with the embodiments can be included in at least one embodiment of the present application. The phrase appears at various places in the specification does not necessarily all refer to the same embodiment, nor is it necessarily independent or alternative embodiments to other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0027] The existing target pose estimation method is applied to a video scene, and the prediction result has significant jitter, which is mainly caused by the following points:

[0028] 1) The target pose change in the video scene will be much faster than that in the static scene, which makes it difficult for the traditional human pose estimation model to accurately capture the rapid change in the video;

[0029] 2) The background noise in the video scene will affect the prediction result of the model;

[0030] 3) The target pose estimation itself lacks modeling ability for continuous motion, resulting in discontinuous time series prediction results.

[0031] Therefore, it is necessary to de-jitter the prediction result of the target pose estimation model.

[0032] The existing de-jittering methods for timing target pose estimation mainly include traditional filter-based methods and deep learning-based methods. The traditional filter can effectively eliminate jitter, but it has limitations such as model assumption and locality; while the deep learning method can effectively simulate and suppress complex jitter, and learn the global rule from a large amount of timing data, thereby more effectively solving the jitter problem.

[0033] The traditional filter, such as Gaussian filter, SG filter, etc., is the most commonly used method for smoothing and de-jittering of continuous signals. The traditional filter de-jittering method is simple and intuitive, and the calculation amount is relatively small, but it needs to be customized and parameterized for specific business scenarios, and it is easy to appear over-smoothing. In addition, the traditional filter often requires collecting prediction results of a long time window for filtering, thereby limiting the de-jittering effect in real-time scenarios.

[0034] The end-to-end method based on deep learning can be directly de-jittered and smoothed, and the timing human pose estimation result is given, but the accuracy of human pose estimation is reduced. On the other hand, the end-to-end model requires more and higher quality training sets to train the model, which also leads to higher training cost of the end-to-end model and limited application scenarios.

[0035] Therefore, any one of the following embodiments provided by the present application is proposed to solve any one of the above technical problems, and specific reference is made to any one of the following embodiments.

[0036] Reference Figure 1 , Figure 1 is a flowchart of an embodiment of the target pose recognition method provided by the present application. The method comprises:

[0037] Step 11: obtaining a to-be-recognized video; wherein the to-be-recognized video comprises continuous video frames.

[0038] In some embodiments, the to-be-recognized video can be obtained by a monocular camera or a binocular camera, etc.

[0039] Specifically, the camera device can be a device applied to a monitoring scene. For example, the camera device is arranged in a campus to capture events occurring in the campus. For example, the camera device is arranged in a traffic scene to capture events in the traffic scene. For example, the camera device is arranged in a home scene to capture events in the home scene.

[0040] Step 12: performing target pose prediction on the to-be-recognized video to obtain an initial pose prediction result corresponding to each video frame.

[0041] It can be understood that in the video, a corresponding target is usually included, such as the target can be a person and / or an animal, etc. And the posture of the target is recognized, and according to the corresponding posture recognition result, a corresponding prompt can be performed. For example, in a classroom scene, whether the posture of a student meets the requirements can be recognized, and then a posture adjustment reminder can be performed according to the recognition result. For example, in a life scene, whether the target posture is a preset posture can be recognized, and then a warning prompt can be performed according to the recognition result.

[0042] In some embodiments, the posture prediction model can be trained to predict the target posture of the video to be recognized, and the initial posture prediction result corresponding to each video frame can be obtained. The posture prediction model can be constructed based on a residual connection network, an inception network, an MLP network, a convolutional neural network, and / or an SE network.

[0043] Step 13: decompose all initial posture prediction results to obtain at least two different frequency components.

[0044] Generally, posture recognition prediction is performed by recognizing key points of a target, and then the key points are connected in relevance to form a corresponding posture image. The posture image is the initial posture prediction result.

[0045] The present application considers that the target posture change in a video dynamic scene is much faster than that in a static scene, which makes it difficult for a traditional human posture estimation model to accurately capture the rapid change in the video, and the background noise in the video dynamic scene affects the prediction result of the model, and the target posture estimation itself lacks modeling ability for continuous motion, resulting in discontinuous time sequence prediction results, etc. Therefore, it is necessary to de-bounce the posture prediction result. The purpose of de-bouncing is to eliminate signal noise represented as key point jitter, so as to improve the accuracy and reliability of target posture estimation.

[0046] Based on this, all initial posture prediction results are decomposed to obtain at least two different frequency components. For example, all initial posture prediction results are decomposed to obtain high frequency components and low frequency components.

[0047] For another example, all initial posture prediction results are decomposed according to different frequency sections, a component with a frequency greater than a first frequency is taken as a first frequency component, a component with a frequency less than the first frequency and greater than a second frequency is taken as a second frequency component, and a component with a frequency less than the second frequency is taken as a third frequency component.

[0048] Step 14: input each frequency component into a network model to obtain a sub-posture prediction result corresponding to each frequency component.

[0049] In some embodiments, the network model can be a lightweight network model, and can be capable of performing pose prediction on each frequency component to obtain a sub-pose prediction result corresponding to each frequency component.

[0050] In some embodiments, the network model can include a plurality of sub-networks, and each sub-network corresponds to a frequency component. Therefore, the corresponding frequency component can be input into the corresponding sub-network to obtain the corresponding sub-pose prediction result.

[0051] Step 15: obtaining a final pose prediction result according to all the sub-pose prediction results.

[0052] Since the factor pose prediction result is obtained by decomposing the initial pose prediction result, all the sub-pose prediction results can be summed up, and the result of the summation is the final pose prediction result.

[0053] In the present embodiment, at least two different frequency components are obtained by decomposing all the initial pose prediction results, and the noise is effectively suppressed. Then, each frequency component is input into the network model to obtain a sub-pose prediction result corresponding to each frequency component. The final pose prediction result is obtained according to all the sub-pose prediction results, which can obtain an accurate pose prediction result, effectively remove the jitter of the initial pose prediction result, and improve the accuracy of the final target pose recognition.

[0054] Referring to Figure 2 , Figure 2 is a flowchart of an embodiment of the target pose recognition method provided in the present application. The method includes:

[0055] Step 21: obtaining a video to be recognized; wherein the video to be recognized includes continuous video frames.

[0056] Step 22: performing target pose prediction on the video to be recognized to obtain an initial pose prediction result corresponding to each video frame.

[0057] Steps 21-22 have the same or similar technical solutions as the above embodiments, and will not be described here.

[0058] Step 23: decomposing all the initial pose prediction results according to a time sequence to obtain at least two different frequency components.

[0059] Since the video has time continuity, the video frames also have a corresponding time sequence. The corresponding initial pose prediction result also has a corresponding time sequence. Based on this, all the initial pose prediction results can be decomposed according to a time sequence to obtain at least two different frequency components. For example, spectral analysis, time series analysis, and Fourier series analysis can be used to decompose all the initial pose prediction results.

[0060] In some embodiments, referring toFigure 3 Step 23 can be the following flow:

[0061] Step 231: determining the initial first frequency component from all initial pose prediction results.

[0062] Since the initial pose prediction results are represented in the form of tensors, the initial first frequency component can be determined from all initial pose prediction results.

[0063] Step 232: performing padding operation and mean operation on the initial first frequency component to obtain the first frequency component.

[0064] Then, padding operation and mean operation are performed on the initial first frequency component to obtain the first frequency component. It can be understood that when the initial pose prediction results corresponding to the tensors are decomposed, the format of the decomposed frequency component does not correspond to the preset format, and therefore padding operation is needed. For example, 0 padding operation is performed on the head and tail of the initial first frequency component. In other embodiments, the padding operation can fill the remaining content without special meaning. Then, mean operation is performed on all initial first frequency components after the padding operation to obtain the first frequency component.

[0065] Step 233: obtaining the second frequency component by using the initial pose prediction results and the first frequency component; wherein the frequency of the first frequency component is lower than the frequency of the second frequency component.

[0066] Since the initial pose prediction results are composed of the first frequency component and the second frequency component, the second frequency component can be obtained by subtracting the first frequency component from the initial pose prediction results.

[0067] Step 24: inputting each frequency component into the network model to obtain a sub-pose prediction result corresponding to each frequency component.

[0068] Step 25: obtaining the final pose prediction result according to all sub-pose prediction results.

[0069] Step 24 and step 25 have the same or similar technical solutions as any of the above embodiments, and will not be described here.

[0070] In this embodiment, at least two different frequency components are obtained by decomposing all initial pose prediction results, noise is effectively suppressed, and then each frequency component is input into the network model to obtain a sub-pose prediction result corresponding to each frequency component; the final pose prediction result is obtained according to all sub-pose prediction results. The way of obtaining accurate pose prediction results can effectively remove the jitter of the initial pose prediction results and improve the accuracy of the final target pose recognition.

[0071] Further, by using time series decomposition technology, noise can be effectively suppressed, and the stability of the system can be improved.

[0072] Referring to Figure 4 , Figure 4 is a flowchart of an embodiment of the target pose recognition method provided in the present application. The method comprises:

[0073] Step 41: obtaining a video to be recognized; wherein the video to be recognized comprises continuous video frames.

[0074] Step 42: performing target pose prediction on the video to be recognized to obtain an initial pose prediction result corresponding to each video frame.

[0075] Step 43: decomposing all initial pose prediction results according to time series to obtain a first frequency component and a second frequency component.

[0076] Steps 41-43 have the same or similar technical solutions as the above embodiments, and will not be described here.

[0077] The difference is that in the present embodiment, all initial pose prediction results are decomposed according to time series to obtain a first frequency component and a second frequency component. The second frequency component can be a high frequency component, and the second frequency component can be a low frequency component. In some embodiments, a frequency threshold can be set, and frequencies higher than the frequency threshold are high frequencies, and frequencies lower than the frequency threshold are low frequencies.

[0078] Step 44: inputting the first frequency component into a first subnetwork to obtain a first sub-pose prediction result corresponding to the first frequency component.

[0079] Wherein the first subnetwork is trained by using pose ground truth training data with the first frequency component.

[0080] Step 45: inputting the second frequency component into a second subnetwork to obtain a second sub-pose prediction result corresponding to the second frequency component.

[0081] Wherein the second subnetwork is trained by using pose ground truth training data with the second frequency component.

[0082] Step 46: obtaining a final pose prediction result according to the first sub-pose prediction result and the second sub-pose prediction result.

[0083] In the embodiment, the noise is effectively suppressed by decomposing all initial pose prediction results into at least two different frequency components, and then inputting each frequency component into the network model to obtain a sub-pose prediction result corresponding to each frequency component; and the accurate pose prediction result is obtained by the manner of obtaining the final pose prediction result according to all sub-pose prediction results, which can effectively remove the jitter of the initial pose prediction result and improve the accuracy of the final target pose recognition.

[0084] Further, the time series decomposition technology can effectively suppress the noise and improve the stability of the system.

[0085] Referring to Figure 5 , Figure 5 is a flowchart of an embodiment of the network model training method provided in the application. The network model includes at least two sub-networks, wherein the network model is trained in the following manner:

[0086] Step 51: obtaining a training video labeled with a real pose.

[0087] The labeling manner can be a manual labeling manner or a machine labeling manner. It can be understood that the accuracy of the training video labeled with the real pose can be considered as 100%.

[0088] Step 52: decomposing the training video to obtain at least two different frequency video components.

[0089] The training video is decomposed into at least two different frequency video components according to the time series.

[0090] Step 53: training a corresponding sub-network using each frequency video component to obtain a corresponding loss value.

[0091] The loss value can be obtained based on a corresponding loss function, and in the loss function, the position, velocity and acceleration related to the pose can be used as hyperparameters for constraint. The position, velocity and acceleration can be obtained in continuous video frames.

[0092] Step 54: adjusting the network parameters of each sub-network according to the loss value.

[0093] Since multiple loss values are obtained, the multiple loss values can be weighted and summed to obtain a total loss value. The network parameters of each sub-network are adjusted based on the total loss value until the training is completed. The trained network model is used to recognize the above frequency components.

[0094] In some application scenarios, the network model includes a first sub-network and a second sub-network. Therefore, the network model can be trained in the manner as shown in Figure 6

[0095] ​Step 61: Obtain a training video labeled with a real pose.

[0096] Step 62: Decompose the training video to obtain a first frequency video component and a second frequency video component.

[0097] The first frequency video component is a low frequency component, and the second frequency video component is a high frequency component.

[0098] Step 63: Train a first sub-network using the first frequency video component to obtain a first loss value.

[0099] Step 64: Train a second sub-network using the second frequency video component to obtain a second loss value.

[0100] Step 65: Adjust the network parameters of each sub-network according to the first loss value and the second loss value.

[0101] The network parameters of each sub-network are adjusted using the correlation of the first loss value and the second loss value. For example, the first loss value and the second loss value are summed, and the network parameters of each sub-network are adjusted based on the summed loss value until the training is completed.

[0102] The trained network model is used to recognize the frequency components described above.

[0103] The embodiments of the present application can be applied to the scene of human pose estimation to de-jitter the pose prediction results predicted by the front-end network model in real time, and obtain more accurate pose prediction results. As shown in Figure 7 includes:

[0104] Step 71: Obtain a human pose estimation prediction result with jitter on consecutive video frames.

[0105] The prediction result can be generated by an algorithm or a model on a monocular camera or a multi-sensor-based vision system.

[0106] The present application can support different human pose representation results at the same time, such as 2D coordinate system human key point representation (shape [J, 2], J represents the number of key points, and 2 represents two-dimensional coordinates), 3D coordinate system human key point representation (shape [J, 3], J represents the number of key points, and 3 represents three-dimensional coordinates), and human key point representation using SMPL format (shape [J, 3], J represents the number of key points, and 3 represents the corresponding rotation or translation in a three-dimensional coordinate system).

[0107] It can be understood that due to consecutive video frames, the human pose estimation prediction result has jitter, which may result in insufficient accuracy of the result. Therefore, in the present application, the human pose estimation prediction result is post-processed, and step 72 is performed.

[0108] Step 72: The obtained jitter prediction result is decomposed into low-frequency components and high-frequency components by time series decomposition.

[0109] The decomposition method can adopt the following method:

[0110] The original N tensors of shape [J, D] are decomposed using a time series decomposition method, and two sub-sequences corresponding to the low-frequency and high-frequency components of the motion are obtained. The time series decomposition method is as follows:

[0111]

[0112] wherein, represents the original N tensors of shape [J, D], and represents the low-frequency components and high-frequency components obtained after decomposition, and Padding represents the 0 padding operation at the beginning and end, and AvgPooling represents the average operation.

[0113] Step 73: The low-frequency components and high-frequency components are accepted by the deep learning model as model inputs, and the time series human pose estimation result after de-jittering is output.

[0114] The deep learning model accepts N tensors of shape [J, D] as input and outputs the de-jittering smoothed prediction result. Wherein, N represents the number of video frames, J represents the number of key points, and D represents the number of dimensions.

[0115] The above deep learning model can be a lightweight neural network model that supports solving human pose estimation de-jittering tasks including 2D, 3D, and grid model representation.

[0116] The model flow chart of the deep learning model is as shown in Figure 8 The high-frequency sub-network is used to process the high-frequency components, and the low-frequency sub-network is used to process the low-frequency components. Since the time series decomposition method separates the low-frequency components and high-frequency components based on the additive principle, the sum of the outputs of the high-frequency sub-network and the low-frequency sub-network can represent the human pose estimation result after de-jittering processing during inference. During training, the same time series decomposition operation is performed on the human pose ground truth to obtain the corresponding true low-frequency components and true high-frequency components, and then the low-frequency sub-network and the high-frequency sub-network are supervised learning. Specifically, the total loss function used for gradient descent can be represented as:

[0117] L = L low + L high ;

[0118]

[0119]

[0120] wherein, L low and L high denote the loss function of the low-frequency subnetwork and the high-frequency subnetwork, denotes the difference operator, and α, β and γ denote the hyperparameters that control the three motion descriptors of position, velocity and acceleration.

[0121] Specifically, the present application supports various tasks of 2D, 3D and SMPL representation, can effectively perform human pose estimation, improve the stability of human pose estimation in time sequence, support lightweight neural network model skeleton, and can be used for human pose estimation de-jittering task in real-time or quasi-real-time scene; and adopts time sequence decomposition technology, which can effectively suppress noise and improve the stability of the system.

[0122] Referring to Figure 9 , the target pose recognition device 90 comprises an acquisition module 91, a prediction module 92, a decomposition module 93 and a processing module 94.

[0123] The acquisition module 91 is configured to acquire a to-be-recognized video; wherein the to-be-recognized video comprises continuous video frames.

[0124] The prediction module 92 is configured to perform target pose prediction on the to-be-recognized video to obtain an initial pose prediction result corresponding to each video frame.

[0125] The decomposition module 93 is configured to decompose all the initial pose prediction results to obtain at least two different frequency components.

[0126] The processing module 94 is configured to input each frequency component into a network model to obtain a sub-pose prediction result corresponding to each frequency component; and obtain a final pose prediction result according to all the sub-pose prediction results.

[0127] It can be understood that the acquisition module 91, the prediction module 92, the decomposition module 93 and the processing module 94 cooperate to realize the method of any one of the above-mentioned embodiments.

[0128] Referring to Figure 10 , Figure 10 is a structural schematic diagram of an embodiment of an electronic device provided by the present application. The electronic device 110 comprises a processor 111 and a memory 112 coupled with the processor 111; wherein the memory 112 is configured to store a computer program, and the processor 111 is configured to execute the computer program to realize the following method:

[0129] Obtaining a to-be-identified video; wherein the to-be-identified video comprises continuous video frames; performing target pose prediction on the to-be-identified video to obtain an initial pose prediction result corresponding to each video frame; decomposing all the initial pose prediction results to obtain at least two different frequency components; inputting each frequency component into a network model to obtain a sub-pose prediction result corresponding to each frequency component; and obtaining a final pose prediction result according to all the sub-pose prediction results.

[0130] It can be understood that the processor 111 is configured to execute a computer program to implement the method of any of the above embodiments.

[0131] Referring to Figure 11 , Figure 11 is a structural schematic diagram of another embodiment of an electronic device provided by the present application. The electronic device 110, for example, can be a mobile electronic device, and the electronic device 110 can include a memory 112, a processor (Central Processing Unit, CPU) 111, a circuit board (not shown in the figure), a power supply circuit, and a microphone 123. The circuit board is disposed inside a space enclosed by a housing; the processor 111 and the memory 112 are arranged on the circuit board; the power supply circuit is configured to supply power to various circuits or devices of the electronic device; the memory 112 is configured to store executable program codes; and the processor 111 is configured to run a computer program corresponding to the executable program codes by reading the executable program codes stored in the memory 112, so as to implement the method of any of the above embodiments.

[0132] The electronic device can further include a peripheral interface 114, an RF (Radio Frequency, RF) circuit 116, an audio circuit 117, a speaker 122, a power management chip 119, an input / output (I / O) subsystem 120, other input / control devices 121, a display 124, and an external port 115, which communicate with each other through one or more communication buses or signal lines 118.

[0133] The memory 112 can be accessed by the processor 111, the peripheral interface 114, etc., and the memory 112 can include a high-speed random access memory and can further include a non-volatile memory, such as one or more disk storage devices, flash memory devices, or other volatile solid-state memory devices. The peripheral interface 114 can connect the input and output peripherals of the device to the processor 111 and the memory 112.

[0134] The I / O subsystem 120 can connect the input and output peripherals on the device, such as the display 124 and other input / control devices 121, with the peripherals interface 114. The I / O subsystem 120 can include a display controller 1201 and one or more input controllers 1202 to control other input / control devices 121. Where the other input / control devices 121 include a physical button (e.g., 203), a rocker switch, a touchpad, a physical dial, and / or other input / control devices. The input controller(s) 1202 receives and / or sends electrical signals from and / or to other input / control devices 121. The display 124 detects contact and any movement or release of the contact, which can be interpreted as a user input. The display controller 1201 receives and / or sends electrical signals from and / or to the display 124. The display 124 detects contact and any movement or release of the contact, which can be interpreted as a user input. The display controller 1201 converts the electrical signals from the display 124 into images to be displayed on the display 124. The display 124 is used to display visual output to the user of the electronic device. Such visual output can include graphics, text, icons, video, and any combination thereof. Some or all of the visual output can be generated electronically, e.g., on the display 124, or stored, e.g., on the storage 130, and retrieved and displayed on the display 124.

[0135] The display 124 is an input and output interface between the user and the electronic device, and displays visual output to the user, which can include graphics, text, icons, video, and the like.

[0136] The display controller 1201 in the I / O subsystem 120 receives and / or sends electrical signals from and / or to the display 124. The display 124 detects contact and any movement or release of the contact, which can be interpreted as a user input. The display controller 1201 converts the electrical signals from the display 124 into images to be displayed on the display 124. The display 124 is used to display visual output to the user of the electronic device. Such visual output can include graphics, text, icons, video, and any combination thereof. Some or all of the visual output can be generated electronically, e.g., on the display 124, or stored, e.g., on the storage 130, and retrieved and displayed on the display 124.

[0137] The RF circuitry 116 can be configured to facilitate wireless communications for the electronic device 100. For example, the RF circuitry 116 can include a receiver, a transmitter, a transmit / receive element, or a combination thereof, which can be configured to transmit and to receive wireless signals over one or more wireless communication networks, such as a network 170. In some embodiments, the RF circuitry 116 can include a plurality of switches, filters, amplifiers, or a combination thereof, which can be configured to facilitate the communication of signals between various components of the electronic device 100. In some embodiments, the RF circuitry 116 can include a pluraUty of antennas. In some embodiments, the RF circuitry 116 can be configured to transmit and to receive signals using different communication technologies or protocols, including but not limited to global system for mobile communications (GSM), code division multiple access (CDMA), CDMA2000, UMTS, LTE, WIMAX, Bluetooth, WiFi, Zigbee, or any other wireless communication technology or protocol.

[0138] The audio circuit 117 is mainly used to receive audio data from the peripheral interface 114, convert the audio data into an electrical signal, and send the electrical signal to the speaker 122. The speaker 122 is used to restore the voice signal received by the mobile phone from the wireless network through the RF circuit 116 into sound and play the sound to the user. The power management chip 119 is used to power and manage the power supply for the processor 111, the I / O subsystem 120, and the hardware connected to the peripheral interface 114.

[0139] The electronic device described above can be a mobile terminal, such as a smart phone, a tablet, or the like, or a terminal device, such as a personal computer.

[0140] Referring to Figure 12 , Figure 12 is a structural schematic diagram of an embodiment of the computer-readable storage medium provided in the present application. The computer-readable storage medium 150 stores a computer program 151, and the computer program 151, when executed by a processor, implements the following method:

[0141] obtaining a to-be-identified video; wherein the to-be-identified video includes continuous video frames; performing target pose prediction on the to-be-identified video to obtain an initial pose prediction result corresponding to each video frame; decomposing all the initial pose prediction results to obtain at least two different frequency components; inputting each frequency component into a network model to obtain a sub-pose prediction result corresponding to each frequency component; and obtaining a final pose prediction result according to all the sub-pose prediction results.

[0142] It can be understood that the computer program 151, when executed by a processor, can also implement the method of any of the above embodiments.

[0143] In summary, the target pose identification method, device, electronic device, and readable storage medium provided in the present application effectively suppress noise by decomposing all the initial pose prediction results to obtain at least two different frequency components, then inputting each frequency component into a network model to obtain a sub-pose prediction result corresponding to each frequency component, and obtaining a final pose prediction result according to all the sub-pose prediction results, thereby obtaining an accurate pose prediction result, effectively removing the jitter of the initial pose prediction result, and improving the accuracy of the final target pose identification.

[0144] Further, the present application decomposes the initial pose prediction result with jitter into a low-frequency component and a high-frequency component through time series decomposition, performs noise reduction on different motion components, and effectively filters jitter noise with uncertain amplitude and frequency.

[0145] Further, a deep learning model is adopted to receive the low-frequency component and the high-frequency component as model input, and output a time sequence target pose prediction result after de-jittering. On this basis, an innovative loss function is designed and applied to constrain the optimization iteration direction of the model from the aspects of position, velocity and acceleration.

[0146] Further, the real-time de-jittering method has low calculation amount and can be used for real-time processing, and has the advantages of high real-time performance and good noise resistance.

[0147] In several embodiments provided in the present application, it should be understood that the disclosed method and device can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0148] The integrated units in the above other embodiments, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part of the prior art that makes a contribution or the whole or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0149] The above description is only an embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation based on the content of the specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A target gesture recognition method, characterized by, The method comprises: acquiring a video to be recognized; wherein the video to be recognized comprises continuous video frames; performing target pose prediction on the video to be recognized to obtain an initial pose prediction result corresponding to each video frame; decomposing all the initial pose prediction results to obtain at least two different frequency components; inputting each frequency component into a corresponding subnetwork of a network model to obtain a sub-pose prediction result corresponding to each frequency component; obtaining a final pose prediction result according to all the sub-pose prediction results.

2. The method of claim 1, wherein, The decomposition of all the initial pose prediction results to obtain at least two different frequency components comprises: decomposing all the initial pose prediction results according to a time sequence to obtain at least two different frequency components.

3. The method of claim 2, wherein, The decomposition of all the initial pose prediction results according to a time sequence to obtain at least two different frequency components comprises: determining an initial first frequency component from all the initial pose prediction results; performing a padding operation and a mean operation on the initial first frequency component to obtain a first frequency component; obtaining a second frequency component by using the initial pose prediction result and the first frequency component; wherein the frequency of the first frequency component is lower than the frequency of the second frequency component.

4. The method of claim 3, wherein, The network model comprises a first subnetwork and a second subnetwork, and the inputting of each frequency component into a corresponding subnetwork of a network model to obtain a sub-pose prediction result corresponding to each frequency component comprises: inputting the first frequency component into the first subnetwork to obtain a first sub-pose prediction result corresponding to the first frequency component; and inputting the second frequency component into the second subnetwork to obtain a second sub-pose prediction result corresponding to the second frequency component.

5. The method of claim 4, wherein, The obtaining of a final pose prediction result according to all the sub-pose prediction results comprises: obtaining a final pose prediction result according to the first sub-pose prediction result and the second sub-pose prediction result.

6. The method of claim 1, wherein, The network model comprises at least two subnetworks, and the network model is trained in the following manner: acquiring a training video labeled with a real pose; decomposing the training video to obtain at least two different frequency video components; training a corresponding subnetwork by using each frequency video component to obtain a corresponding loss value; adjusting network parameters of each subnetwork according to the loss value.

7. The method of claim 6, wherein, The decomposition of the training video to obtain at least two different frequency video components comprises: decomposing the training video to obtain a first frequency video component and a second frequency video component. The training of a corresponding subnetwork by using each frequency video component to obtain a corresponding loss value comprises: training the first subnetwork by using the first frequency video component to obtain a first loss value; training the second subnetwork by using the second frequency video component to obtain a second loss value; The adjustment of network parameters of each subnetwork according to the loss value comprises: adjusting network parameters of each subnetwork according to the first loss value and the second loss value.

8. A target pose recognition apparatus characterized by comprising: The target pose recognition device comprises: an acquisition module configured to acquire a video to be recognized; wherein the video to be recognized comprises continuous video frames; A prediction module is configured to perform target pose prediction on the video to be recognized to obtain an initial pose prediction result corresponding to each video frame. A decomposition module is configured to decompose all the initial pose prediction results to obtain at least two different frequency components. A processing module is configured to input each frequency component into a corresponding subnetwork of a network model to obtain a sub pose prediction result corresponding to each frequency component, and obtain a final pose prediction result according to all the sub pose prediction results.

9. An electronic device, comprising: The electronic device includes a processor and a memory coupled to the processor; The memory is configured to store a computer program, and the processor is configured to execute the computer program to implement the method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program, when executed by a processor, implements the method of any one of claims 1-7.

Images