A variable frame rate human motion analysis processing method, device and equipment

Abstract

The present application provides a variable frame rate human motion analysis processing method, device and equipment, the method comprises: acquiring a preset frame rate motion video file; each video frame image in the motion video file is recognized to obtain a three-dimensional motion data model set based on time domain; the motion video file is divided according to preset frame rate distribution rule, to determine the target frame rate corresponding to the motion video file in different time periods; the frame rate of the motion video file in different time periods is converted into the corresponding target frame rate, to determine the frame rate set of the motion video file; the three-dimensional motion data model set and the frame rate set are encapsulated to obtain an encapsulation file, so that the user can analyze the motion data through the encapsulation file, the present application adopts a variable frame rate processing mode, dynamically analyzes and processes according to the actual motion type, and ensures the accuracy of model data and the timeliness of processing as far as possible.

Description

A method, apparatus, and device for analyzing and processing human motion with a variable frame rate. Technical Field

[0001] This article belongs to the field of computer technology, specifically relating to a variable frame rate human motion analysis and processing method, device, and system. Background Technology

[0002] With the rise of healthy living concepts, people are increasingly aware of the importance of home fitness, leading to the emergence of the smart fitness mirror market and a variety of related products. As a product that extends the reach of offline gyms to end users, the smart fitness mirror is constantly being iterated and improved. From initially lacking AI video analysis support, it is now developing towards intelligent terminal functionality, supporting not only voice recognition input but also motion sensing input and AI video motion-assisted analysis.

[0003] The following problems exist in the process of capturing and analyzing videos of actual athletes' movements: Analysis and processing of motion videos can only be performed using a fixed frame rate. When the movement frequency is high, the video analysis and processing may far exceed the actual processing capacity of the equipment, resulting in inaccurate movement analysis and significant delays. For slower movements, a large amount of processing power is spent on relatively inefficient and repetitive analysis, leading to a significant waste of equipment resources. Therefore, improving the efficiency of human movement analysis has become an urgent technical problem to be solved. Summary of the Invention

[0004] In view of the above-mentioned problems in the prior art, the purpose of this paper is to provide a variable frame rate human motion analysis and processing method, apparatus and device, which can improve the efficiency of human motion processing and analysis.

[0005] To solve the above-mentioned technical problems, the specific technical solution presented in this paper is as follows:

[0006] On the one hand, this paper provides a variable frame rate human motion analysis and processing method, the method including:

[0007] Obtain a motion video file with a preset frame rate;

[0008] Joint point recognition is performed on each video frame image in the motion video file to obtain a set of time-domain-based three-dimensional motion data models;

[0009] The motion video files are divided according to a preset frame rate allocation rule to determine the target frame rate corresponding to the motion video files in different time periods;

[0010] The frame rates of motion video files from different time periods are converted into corresponding target frame rates to determine the frame rate set of the motion video files.

[0011] The three-dimensional motion data model set and the frame rate set are encapsulated to obtain an encapsulated file, which allows users to perform motion data analysis through the encapsulated file.

[0012] Further, the step of performing joint point recognition on each video frame image in the motion video file to obtain a set of time-domain-based three-dimensional motion data models includes:

[0013] Joint point identification is performed on each video frame image in the motion video file to determine the information of each joint point of the human body;

[0014] Based on the information of each joint point in each video image, a three-dimensional human body model corresponding to each video frame image is obtained;

[0015] Based on the temporal sequence of video frame images, all human 3D models are integrated to obtain a set of temporal-domain-based 3D motion data models.

[0016] Further, the step of dividing the motion video file according to a preset frame rate allocation rule to determine the target frame rate corresponding to the motion video file in different time periods includes:

[0017] Obtain the motion type corresponding to the motion video file;

[0018] Based on the type of motion, the motion video file is divided into multiple sub-videos, and the importance of each sub-video is different from that of its adjacent sub-videos;

[0019] The target frame rate for each sub-video is determined based on its importance, with higher-importance sub-videos having higher target frame rates.

[0020] Furthermore, dividing the motion video file into multiple sub-videos according to the motion type further includes:

[0021] Obtain the attribute features corresponding to the motion type, whereby the attribute features characterize the motion target of the motion type;

[0022] Based on the attribute characteristics corresponding to the movement type, the movement actions of the movement type are divided into multiple beats, and the movement targets focused on by two adjacent beats are different;

[0023] The motion video file is divided into multiple sub-videos based on multiple beats, wherein the sub-videos corresponding to the beats of the motion target that are closer to the motion type are of higher importance.

[0024] Furthermore, the step of dividing the motion video file according to a preset frame rate allocation rule to determine the target frame rate corresponding to the motion video file in different time periods may further include:

[0025] Calculate the similarity between each video frame and the next video frame to obtain a temporal similarity set;

[0026] Determine the video frame image corresponding to the similarity exceeding the preset value in the similarity set, and define the video frame image as the end frame, wherein the video frame images at both ends of the motion video file are also defined as end frames;

[0027] Based on the end frames, the motion video file is divided into multiple time-segment motion video files;

[0028] Determine if the video frame images in the motion video file for each time period meet the target frame rate for video detection.

[0029] Furthermore, the frame rates of motion video files from different time periods are converted into corresponding target frame rates to determine the frame rate set of the motion video files;

[0030] Obtain the frequency of changes in human movements in motion video files at different time periods;

[0031] Based on the frequency of change, determine the encoding method corresponding to each time period;

[0032] Based on the encoding method corresponding to each time period, the frame rate of the motion video files in different time periods is converted into the corresponding target frame rate to determine the frame rate set of the motion video files.

[0033] Furthermore, enabling users to perform motion data analysis through the packaged file includes:

[0034] In response to the user's fitness playback command, the packaged file is decoded to obtain the three-dimensional motion data model set and the frame rate set, and the motion video file is played through the display device;

[0035] The video capture device captures video of the user's motion process according to the frame rate set, and the video capture device includes at least the display device;

[0036] Based on the data collected by the video acquisition device, a real-time three-dimensional motion data model of the user is established.

[0037] Real-time comparison of the actual 3D motion data model with the synchronous model in the time-domain-based 3D motion data model set;

[0038] If the comparison result exceeds the preset range, an action adjustment prompt will be generated on the display device.

[0039] Furthermore, when the video acquisition device includes the display device and at least one terminal device, the viewing angles of the display device and each of the terminal devices are different;

[0040] The step of establishing a real-time three-dimensional motion data model of the user based on the data acquired by the video acquisition device further includes:

[0041] Real-time acquisition of network parameters for each video capture device;

[0042] Calculate the network processing capability of each video acquisition device based on the network parameters;

[0043] Based on the network processing capabilities, the master control device is determined from all the video acquisition devices;

[0044] The main control device determines the data processing sequence of each video acquisition device based on the network processing capability and preset allocation strategy of each video acquisition device;

[0045] According to the data processing sequence, the video processing device processes the acquired data and sends the processing result to the main control device;

[0046] The main control device integrates the received processing results to obtain the user's actual three-dimensional motion data model.

[0047] On the other hand, this paper also provides a variable frame rate human motion analysis and processing device, the device comprising:

[0048] The acquisition module is used to acquire motion video files with a preset frame rate;

[0049] The three-dimensional motion data model set determination module is used to perform joint point recognition on each video frame image in the motion video file to obtain a time-domain-based three-dimensional motion data model set.

[0050] The target frame rate determination module is used to divide the motion video file according to a preset frame rate allocation rule in order to determine the target frame rate corresponding to the motion video file in different time periods.

[0051] The frame rate set determination module is used to convert the frame rate of motion video files in different time periods into the corresponding target frame rate in order to determine the frame rate set of the motion video files.

[0052] The encapsulation module is used to encapsulate the three-dimensional motion data model set and the frame rate set to obtain an encapsulated file, so that users can perform motion data analysis through the encapsulated file.

[0053] Finally, this document also provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method described above.

[0054] Using the above technical solution, this paper describes a variable frame rate human motion analysis and processing method, apparatus, and device. The method includes: acquiring motion video files with a preset frame rate; performing joint point recognition on each video frame image in the motion video file to obtain a time-domain-based three-dimensional motion data model set; dividing the motion video file according to a preset frame rate allocation rule to determine the target frame rate corresponding to motion video files in different time periods; converting the frame rates of motion video files in different time periods into corresponding target frame rates to determine the frame rate set of the motion video files; and encapsulating the three-dimensional motion data model set and the frame rate set to obtain an encapsulated file, enabling users to perform motion data analysis through the encapsulated file. This paper adopts a variable frame rate processing method, performing dynamic analysis and processing according to the actual motion type, and ensuring the accuracy of the model data and the timeliness of processing as much as possible.

[0055] To make the above and other objects, features and advantages of this document more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments or prior art described herein, the accompanying drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this article. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 illustrates the steps of a variable frame rate human motion analysis and processing method provided in this embodiment.

[0058] Figure 2 shows a schematic diagram of the human body structure model in the embodiments of this paper;

[0059] Figure 3 shows a schematic diagram of the structure of a variable frame rate human motion analysis and processing device provided in the embodiments of this article;

[0060] Figure 4 shows a schematic diagram of the structure of the computer device provided in the embodiments of this article.

[0061] Explanation of symbols in the attached drawings:

[0062] 100. Acquisition Module;

[0063] 200. Module for determining the set of three-dimensional motion data models;

[0064] 300. Target frame rate determination module;

[0065] 400. Frame Rate Set Determination Module;

[0066] 500. Packaging module;

[0067] 402. Computer equipment;

[0068] 404, Processor;

[0069] 406. Memory;

[0070] 408. Drive mechanism;

[0071] 410. Input / output module;

[0072] 412. Input devices;

[0073] 414. Output devices;

[0074] 416. Presentation equipment;

[0075] 418. Graphical User Interface;

[0076] 420. Network interface;

[0077] 422. Communication link;

[0078] 424. Communication bus. Detailed Implementation

[0079] The technical solutions in the embodiments described below will be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments described herein, and not all of the embodiments. Based on the embodiments described herein, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this document.

[0080] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings herein are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0081] In existing technologies, the following problems exist in the process of video acquisition and analysis of actual athletes' movements: motion videos can only be analyzed and processed using a fixed frame rate. When the movement frequency is high, the video analysis and processing may far exceed the actual processing capacity of the equipment, resulting in inaccurate movement analysis and significant delays. For slower movements, a large amount of processing power is spent on relatively inefficient and repetitive analysis, leading to a significant waste of equipment resources.

[0082] To address the aforementioned issues, this embodiment provides a variable frame rate human motion analysis and processing method that can adapt to different motion videos in a timely manner, thereby matching different video frame rates and improving the efficiency of human motion processing and analysis. Figure 1 is a schematic diagram of the steps of a variable frame rate human motion analysis and processing method provided in this embodiment. This specification provides the operation steps of the method as described in the embodiments or flowcharts, but based on conventional or non-creative labor, more or fewer operation steps may be included. The order of steps listed in the embodiments is merely one of many possible execution orders and does not represent the only execution order. In actual system or device products, the method can be executed sequentially or in parallel according to the embodiments or figures. Specifically, as shown in Figure 1, the method may include:

[0083] S101: Obtain a motion video file with a preset frame rate;

[0084] S102: Perform joint point recognition on each video frame image in the motion video file to obtain a set of time-domain-based three-dimensional motion data models;

[0085] S103: Divide the motion video file according to a preset frame rate allocation rule to determine the target frame rate corresponding to the motion video file in different time periods;

[0086] S104: Convert the frame rate of motion video files from different time periods into the corresponding target frame rate to determine the frame rate set of the motion video files;

[0087] S105: The three-dimensional motion data model set and the frame rate set are encapsulated to obtain an encapsulated file, so that the user can perform motion data analysis through the encapsulated file.

[0088] This specification can be understood as follows: by dividing motion video files acquired at a fixed frequency into time periods, different target frame rates are determined for different time periods. Then, multiple different video frame rates are obtained from the same motion video file through frame rate conversion. Combined with the identified three-dimensional motion data model, targeted processing of video files is achieved. While ensuring accuracy and reliability, energy consumption is reduced and the processing efficiency of video files is improved.

[0089] To ensure the reliability of video data acquisition, the preset frame rate can be a high frame rate. High-quality motion video files can be obtained by shooting at a high frame rate. For example, the preset frame rate can be greater than 30 frames, such as 60 frames, 120 frames, 240 frames, etc. The frame rate is not limited in the embodiments of this specification.

[0090] It should be noted that the exercise video file can be video data collected by the coach during exercise, and the posture during the exercise can be the standard posture of the exercise, so that the exerciser or user can retrieve the exercise video file for follow-up practice and correction of the movement.

[0091] In the embodiments of this specification, the video files are encoded and optimized according to the frame rate set, compressing high frame rate fixed frame rate videos into variable frame rate videos, thereby reducing the video file size and the required network transmission bandwidth.

[0092] In the embodiments of this specification, the step of performing joint point recognition on each video frame image in the motion video file to obtain a set of time-domain-based three-dimensional motion data models includes:

[0093] Joint point identification is performed on each video frame image in the motion video file to determine the information of each joint point of the human body;

[0094] Based on the information of each joint point in each video image, a three-dimensional human body model corresponding to each video frame image is obtained;

[0095] Based on the temporal sequence of video frame images, all human 3D models are integrated to obtain a set of temporal-domain-based 3D motion data models.

[0096] This can be understood as follows: each video frame image can be recognized by a pre-trained convolutional neural network to determine the joint information in each image. The joint information can be the attributes and positions of each joint. For example, the convolutional neural network can be used to identify the joints of the user's limbs in each video frame, such as the left ear, left eye, tip of the nose, right eye, right ear, left shoulder, left elbow, left wrist, left fingertip, right shoulder, right elbow, right wrist, right fingertip, left hip, left knee, left ankle, left toe, right hip, right knee, right ankle, and right toe (see Figure 2 for specific point numbers).

[0097] Furthermore, by analyzing the time-varying states of the same joint point within consecutive video frames, a set of time-domain 3D motion data models of the user can be generated. This set of 3D motion data models can represent the standard motion posture for that movement, which the athlete can imitate and learn by playing the video.

[0098] In some embodiments of this specification, dividing the motion video file according to a preset frame rate allocation rule to determine the target frame rate corresponding to the motion video file in different time periods includes:

[0099] Obtain the motion type corresponding to the motion video file;

[0100] Based on the type of motion, the motion video file is divided into multiple sub-videos, and the importance of each sub-video is different from that of its adjacent sub-videos;

[0101] The target frame rate for each sub-video is determined based on its importance, with higher-importance sub-videos having higher target frame rates.

[0102] This can be understood as follows: the embodiments of this specification split the motion video file into sub-videos of different importance. The importance can be the degree of correlation between the sub-video and the type of motion. The sub-video with a better correlation is more important, and thus its corresponding target frame rate is higher. In this way, when the motion video file is played, the sub-videos can have higher video quality, resulting in a better experience for following along and learning.

[0103] The duration of each sub-video can be consistent or inconsistent. For example, the motion video file can be split into multiple sub-videos according to a preset duration. In other embodiments, the motion video file can also be divided into multiple sub-videos according to the similarity of the user's (i.e., the coach's) actions. The duration of each sub-video is determined by the set of consecutive actions with high similarity. The specific division method will be further described below.

[0104] The relevance between a sub-video and a sport type can be determined based on the theme of the sport type. For example, if the theme of sport type A is leg exercises, then sub-video B, which contains more leg exercises, is more relevant to sport type A. Consequently, sub-video B is more important, and therefore, a higher target frame rate is assigned to sub-video B.

[0105] In the embodiments of this specification, a mapping table between importance and target frame rate can be set. In this way, when determining the importance of a sub-video, the target frame rate corresponding to the sub-video can be determined by looking up the table. In some other embodiments, the importance can also be digitized (for example, it can be defined as five importance levels from high to low: 1, 2, 3, 4, and 5), thereby establishing a correspondence between importance, sub-video duration, and target frame rate (such as calculation formulas), so as to obtain a more accurate and reliable target frame rate and achieve efficient utilization of device performance.

[0106] In some embodiments of this specification, dividing the motion video file into multiple sub-videos according to the motion type further includes:

[0107] Obtain the attribute features corresponding to the motion type, whereby the attribute features characterize the motion target of the motion type;

[0108] Based on the attribute characteristics corresponding to the movement type, the movement actions of the movement type are divided into multiple beats, and the movement targets focused on by two adjacent beats are different;

[0109] The motion video file is divided into multiple sub-videos based on multiple beats, wherein the sub-videos corresponding to the beats of the motion target that are closer to the motion type are of higher importance.

[0110] This can be understood as follows: the attribute features can be the theme of the exercise type. Through this theme, the exercise target of the exercise type can be determined. For example, kickboxing focuses more on arm swings, while aerobics focuses more on limb movements. In actual exercise, the movements of each type of exercise are pre-arranged or designed. Therefore, the pre-designed set of movements can be determined through the theme of the exercise type. The pre-designed set of movements is then divided into multiple beats, where the movements or objects of attention of two adjacent beats are different, or the movements of two adjacent beats are significantly different. The divided beats are then mapped to the exercise video file to obtain the sub-videos corresponding to each beat. Furthermore, the sub-videos corresponding to beats that are closer to the exercise target are of higher importance, thereby achieving targeted processing of local videos in the exercise video file.

[0111] In some embodiments of this specification, the step of dividing the motion video file according to a preset frame rate allocation rule to determine the target frame rate corresponding to the motion video file in different time periods may further include:

[0112] Calculate the similarity between each video frame and the next video frame to obtain a temporal similarity set;

[0113] Determine the video frame image corresponding to the similarity exceeding the preset value in the similarity set, and define the video frame image as the end frame, wherein the video frame images at both ends of the motion video file are also defined as end frames;

[0114] Based on the end frames, the motion video file is divided into multiple time-segment motion video files;

[0115] Determine if the video frame images in the motion video file for each time period meet the target frame rate for video detection.

[0116] In other words, the embodiments of this specification can also segment motion video files according to the similarity of video frame images to obtain multiple corresponding sub-videos, and aggregate video frame images with high similarity and continuity to form a single sub-video, so that motion video files of multiple time periods can be divided.

[0117] In the embodiments of this specification, the target frame rate can also be determined by the following steps: determining the minimum frame rate required for video detection and analysis in each sub-video, and determining the target frame rate of the sub-video based on this frame rate. The minimum frame rate for video detection and analysis can be the frame rate that meets preset video playback requirements, such as ensuring smooth video playback without stuttering under existing device hardware and network parameters.

[0118] In some embodiments of this specification, the frame rate of motion video files from different time periods is converted into a corresponding target frame rate to determine the frame rate set of the motion video files;

[0119] Obtain the frequency of changes in human movements in motion video files at different time periods;

[0120] Based on the frequency of change, determine the encoding method corresponding to each time period;

[0121] Based on the encoding method corresponding to each time period, the frame rate of the motion video files in different time periods is converted into the corresponding target frame rate to determine the frame rate set of the motion video files.

[0122] This can be understood as follows: different encoding methods have different encoding efficiencies and capabilities, and their adaptability varies for complex tasks. For example, methods with strong encoding capabilities place a greater load on the equipment and have higher requirements, but can encode complex video data; methods with low encoding capabilities place a less load on the equipment, but can only encode simple video data. Therefore, by determining the frequency of human motion changes, the encoding difficulty of motion video files in different time periods can be determined, and then a suitable encoding method can be selected. This can reduce energy consumption and improve resource utilization while ensuring the encoding task is completed.

[0123] In some embodiments of this specification, enabling the user to perform motion data analysis through the packaged file includes:

[0124] In response to the user's fitness playback command, the packaged file is decoded to obtain the three-dimensional motion data model set and the frame rate set, and the motion video file is played through the display device;

[0125] The video capture device captures video of the user's motion process according to the frame rate set, and the video capture device includes at least the display device;

[0126] Based on the data collected by the video acquisition device, a real-time three-dimensional motion data model of the user is established.

[0127] Real-time comparison of the actual 3D motion data model with the synchronous model in the time-domain-based 3D motion data model set;

[0128] If the comparison result exceeds the preset range, an action adjustment prompt will be generated on the display device.

[0129] Furthermore, when the video acquisition device includes the display device and at least one terminal device, the viewing angles of the display device and each of the terminal devices are different;

[0130] The step of establishing a real-time three-dimensional motion data model of the user based on the data acquired by the video acquisition device further includes:

[0131] Real-time acquisition of network parameters for each video capture device;

[0132] Calculate the network processing capability of each video acquisition device based on the network parameters;

[0133] Based on the network processing capabilities, the master control device is determined from all the video acquisition devices;

[0134] The main control device determines the data processing sequence of each video acquisition device based on the network processing capability and preset allocation strategy of each video acquisition device;

[0135] According to the data processing sequence, the video processing device processes the acquired data and sends the processing result to the main control device;

[0136] The main control device integrates the received processing results to obtain the user's actual three-dimensional motion data model.

[0137] For example, the aforementioned temporal 3D motion data model set M0 and frame rate set F are respectively regarded as two independent data streams: data stream S M0 and data stream S FIt is packaged together with the motion video file using the Matroska Multimedia Container format;

[0138] When a fitness enthusiast (i.e., an exerciser) operates the fitness mirror to play fitness videos, the mirror's built-in player separates these two independent data streams, sending the data stream S... F Used to control the timing of sampling for motion analysis of fitness enthusiasts, transmitting data stream S M0 Used for comparing the exerciser's movements with standard movements at that time point;

[0139] When only a fitness mirror device is available, the data stream S is used directly. F Control the timing of motion analysis sampling for fitness participants to collect, analyze, and process data;

[0140] When multiple devices are performing motion analysis and processing, the data stream S is... F The data is forwarded to the main control device, which then controls different smart devices to perform collaborative processing according to the allocation strategy.

[0141] Furthermore, when there are video playback operations such as pause / resume, fast forward / rewind / slow down during video playback, the data stream S... F The frame synchronization method ensures that the sampling frame rate and time are synchronized;

[0142] Based on the time-domain three-dimensional motion data model set M1 collected by the fitness mirror and multiple smart terminal devices, the exerciser is given relevant motion prompts by comparing M1 and M0.

[0143] The variable frame rate human motion analysis and processing method provided in the embodiments of this specification optimizes video files by encapsulating multiple data streams with a variable frame rate; it encapsulates data sampling control data streams, human motion data streams, and multiple data streams such as video, audio, and subtitles to realize multiple application scenarios such as motion analysis control, human motion comparison, and VR virtual character synthesis.

[0144] Based on the methods provided above, this specification also provides a variable frame rate human motion analysis and processing device, as shown in Figure 3. The device includes:

[0145] Acquisition module 100 is used to acquire motion video files with a preset frame rate;

[0146] The three-dimensional motion data model set determination module 200 is used to perform joint point recognition on each video frame image in the motion video file to obtain a time-domain-based three-dimensional motion data model set.

[0147] The target frame rate determination module 300 is used to divide the motion video file according to a preset frame rate allocation rule in order to determine the target frame rate corresponding to the motion video file in different time periods.

[0148] The frame rate set determination module 400 is used to convert the frame rate of motion video files in different time periods into the corresponding target frame rate in order to determine the frame rate set of the motion video files.

[0149] The encapsulation module 500 is used to encapsulate the three-dimensional motion data model set and the frame rate set to obtain an encapsulated file, so that the user can perform motion data analysis through the encapsulated file.

[0150] The beneficial effects achieved by the device are consistent with the beneficial effects of the method provided above, and will not be described in detail in the embodiments of this specification.

[0151] As shown in Figure 4, a computer device is provided in an embodiment of this document. The apparatus described herein can be the computer device in this embodiment, performing the methods described above. The computer device 402 may include one or more processors 404, such as one or more central processing units (CPUs), each of which can implement one or more hardware threads. The computer device 402 may also include any memory 406 for storing information of any kind, such as code, settings, data, etc. Without limitation, for example, memory 406 may include any type of RAM, any type of ROM, flash memory device, hard disk, optical disk, etc. More generally, any memory can use any technology to store information. Further, any memory can provide volatile or non-volatile retention of information. Further, any memory can represent a fixed or removable component of the computer device 402. In one case, when processor 404 executes associated instructions stored in any memory or combination of memories, the computer device 402 can perform any operation of the associated instructions. The computer device 402 also includes one or more drive mechanisms 408 for interacting with any memory, such as hard disk drive mechanisms, optical disk drive mechanisms, etc.

[0152] Computer device 402 may also include an input / output module 410 (I / O) for receiving various inputs (via input device 412) and providing various outputs (via output device 414). A specific output mechanism may include a presentation device 416 and an associated graphical user interface (GUI) 418. In other embodiments, the input / output module 410 (I / O), input device 412, and output device 414 may be omitted, and the device may function solely as a computer device within a network. Computer device 402 may also include one or more network interfaces 420 for exchanging data with other devices via one or more communication links 422. One or more communication buses 424 couple the components described above together.

[0153] Communication link 422 can be implemented in any way, such as via a local area network, a wide area network (e.g., the Internet), a point-to-point connection, or any combination thereof. Communication link 422 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc., governed by any protocol or combination of protocols.

[0154] Corresponding to the method in Figure 1, this embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the above-described method.

[0155] This embodiment also provides a computer-readable instruction, wherein when a processor executes the instruction, the program therein causes the processor to perform the method shown in FIG1.

[0156] It should be understood that in the various embodiments of this document, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this document.

[0157] It should also be understood that, in the embodiments herein, the term "and / or" is merely a description of the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following associated objects have an "or" relationship.

[0158] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this document.

[0159] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0160] In the embodiments provided herein, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, devices, or units, or they may be electrical, mechanical, or other forms of connection.

[0161] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments described herein, depending on actual needs.

[0162] Furthermore, the functional units in the various embodiments of this document can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0163] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this paper, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this paper. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0164] This document uses specific embodiments to illustrate the principles and implementation methods of this document. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of this document. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this document. Therefore, the content of this specification should not be construed as a limitation of this document.

Claims

1. A method for analyzing and processing human motion at a variable frame rate, characterized in that, The method includes: acquiring a motion video file with a preset frame rate; performing joint point recognition on each video frame image in the motion video file to obtain a time-domain-based three-dimensional motion data model set; acquiring the motion type corresponding to the motion video file; dividing the motion video file into multiple sub-videos according to the motion type, with each sub-video having a different importance than its adjacent sub-videos; determining the target frame rate corresponding to each sub-video based on its importance, wherein sub-videos with higher importance have higher target frame rates; converting the frame rates of motion video files from different time periods into corresponding target frame rates to determine the frame rate set of the motion video file; and encapsulating the three-dimensional motion data model set and the frame rate set to obtain an encapsulated file, allowing users to perform motion data analysis through the encapsulated file.

2. The method according to claim 1, characterized in that, The step of performing joint point recognition on each video frame image in the motion video file to obtain a set of time-domain-based three-dimensional motion data models includes: performing joint point recognition on each video frame image in the motion video file to determine the joint point information of the human body; obtaining the human body three-dimensional model corresponding to each video frame image based on the joint point information in each video image; and integrating all the human body three-dimensional models according to the temporal sequence of the video frame images to obtain a set of time-domain-based three-dimensional motion data models.

3. The method according to claim 1, characterized in that, The step of dividing the motion video file into multiple sub-videos according to the motion type further includes: obtaining attribute features corresponding to the motion type, wherein the attribute features characterize the motion target of the motion type; dividing the motion action of the motion type into multiple beats according to the attribute features corresponding to the motion type, wherein adjacent beats focus on different motion targets; dividing the motion video file into multiple sub-videos according to the multiple beats, wherein the sub-videos corresponding to beats whose motion targets are closer to the motion target of the motion type have higher importance.

4. The method according to claim 1, characterized in that, The process involves converting the frame rates of motion video files from different time periods into corresponding target frame rates to determine the frame rate set of the motion video files; obtaining the frequency of human motion changes in the motion video files during different time periods; determining the encoding method corresponding to each time period based on the frequency of changes; and converting the frame rates of motion video files from different time periods into corresponding target frame rates based on the encoding method corresponding to each time period to determine the frame rate set of the motion video files.

5. The method according to claim 1, characterized in that, The step of enabling users to perform motion data analysis through the encapsulated file includes: responding to the user's fitness playback command, decoding the encapsulated file to obtain the three-dimensional motion data model set and the frame rate set, and playing the motion video file through a display device; a video acquisition device, including at least the display device, capturing video of the user's motion process according to the frame rate set; establishing the user's actual three-dimensional motion data model in real time based on the data acquired by the video acquisition device; comparing the actual three-dimensional motion data model with the synchronous model in the time-domain-based three-dimensional motion data model set in real time; and generating motion adjustment prompts on the display device if the comparison result exceeds a preset range.

6. The method according to claim 5, characterized in that, When the video acquisition device includes the display device and at least one terminal device, the viewing angles of the display device and each terminal device are different; the step of establishing a real-time actual three-dimensional motion data model of the user based on the acquisition data of the video acquisition device further includes: acquiring the network parameters of each video acquisition device in real time; calculating the network processing capability of each video acquisition device based on the network parameters; determining a master control device from all the video acquisition devices based on the network processing capability; the master control device determining the data processing sequence of the video acquisition devices based on the network processing capability of each video acquisition device and a preset allocation strategy; the video processing device processing the acquisition data according to the data processing sequence and sending the processing result to the master control device; the master control device integrating the received processing result to obtain the user's actual three-dimensional motion data model.

7. A variable frame rate human motion analysis and processing device, characterized in that, The device includes: an acquisition module for acquiring motion video files with a preset frame rate; a 3D motion data model set determination module for performing joint point recognition on each video frame image in the motion video file to obtain a time-domain-based 3D motion data model set; a target frame rate determination module for acquiring the motion type corresponding to the motion video file; dividing the motion video file into multiple sub-videos according to the motion type, with each sub-video having a different importance than its adjacent sub-videos; determining the target frame rate corresponding to each sub-video based on its importance, wherein sub-videos with higher importance have higher target frame rates; a frame rate set determination module for converting the frame rates of motion video files from different time periods into corresponding target frame rates to determine the frame rate set of the motion video file; and a packaging module for packaging the 3D motion data model set and the frame rate set to obtain a packaged file, allowing users to perform motion data analysis through the packaged file.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 6.

Images