Method and device for automatically generating AI training course

Abstract

The invention discloses a method and device for automatically generating an AI training course, and relates to the field of physical education. A first video is acquired, skeleton points are acquired on human body parts in the first video, and a skeleton point movement track is formed in the first video; obtaining an invalid action interval in the first video; performing slicing processing on a time axis of the first video based on the invalid action interval to obtain a plurality of segmented sections; dividing the segmentation section into a valid section and an invalid section according to an invalid action interval; and judging the type of the effective section, matching the effective section with a preset AI identification module, and adding the matched AI identification module into the effective section to obtain a second video for generating the AI. According to the invention, the recorded or live-broadcast course video can be quickly and efficiently converted into the training course with the AI recognition effect.

Description

Technical Field

[0001] The present invention relates to the field of intelligent fitness, and in particular to a method and device for automatically generating an AI training course. Background Art

[0002] Traditional sports teaching or coaching methods mainly involve sports coaches verbally describing the basic concepts and action steps of sports skills or personally demonstrating sports skills to coach people during sports. However, traditional teaching or coaching methods are highly dependent on sports coaches, which may lead to low teaching or coaching efficiency.

[0003] In the era of information-based education, video classrooms have been increasingly widely used in teaching. Video classrooms are a new way of teaching and learning that uses the Internet for online video learning. Currently, video classroom models mainly include two forms: one is video recording, that is, classroom teaching through pre-recorded teaching videos; the other is video live broadcast, that is, classroom teaching by transmitting the video and audio of the teacher's lecture to students in real time. However, both video recording and video live broadcast methods only receive the teacher's teaching content unilaterally.

[0004] Therefore, in the relevant technology, a large number of monitoring equipment, 3D cameras, display screens and wearable sensors are generally chosen to detect people's posture during sports to determine whether the movement posture during exercise is standard. However, when these devices are used for motion analysis or motion correction, the complexity of the detection process may be enhanced due to the integration of multiple devices, which may lead to a slower reaction speed in identifying movement posture, and may not meet people's needs for efficient sports guidance.

[0005] Based on this, in existing physical education teaching, by capturing and analyzing the user's motion data in real time to evaluate whether the user's motion posture meets the standards, the time for relevant instructors to observe and correct the user's motion posture one by one is greatly reduced, thereby facilitating improving the efficiency of the sports teaching process.

[0006] However, converting existing videos into videos with recognition functions usually requires extracting and processing the actions in the course. Such operations require a lot of manpower and material resources, and the conversion efficiency is low, and errors are prone to occur during the conversion process. Summary of the invention

[0007] One object of the present invention is to provide a method and device for automatically generating AI training courses, which can quickly and efficiently convert recorded or live course videos into training courses with AI recognition effects.

[0008] This purpose is achieved by adopting the following technical solutions:

[0009] A method and device for automatically generating an AI training course, the method comprising:

[0010] Acquire a first video, identify a human body part in the first video, acquire a skeleton point on the human body part in the first video, and form a motion trajectory of the skeleton point in the first video;

[0011] Obtaining an invalid action interval in the first video;

[0012] Slicing the time axis of the first video based on the invalid action interval to obtain a plurality of slice segments;

[0013] The segmentation segments are divided into valid segments and invalid segments according to the invalid action intervals;

[0014] Determine the type of the valid segment, match the valid segment with a preset AI recognition module, add the matching AI recognition module to the valid segment, and obtain a second video that generates AI.

[0015] The type of valid segment is determined, specifically including:

[0016] The motion trajectory of the skeleton points in the effective segment is matched with the motion trajectory of the skeleton points of several actions in the preset first action library. If the match is successful, the effective segment is an action recognition segment; if the match is unsuccessful, the motion trajectory of the skeleton points in the effective segment is compared with the motion trajectory of the skeleton points of several actions in the preset second action library to obtain the matching similarity; if the matching similarity is greater than or equal to the preset value, the effective segment is a key frame recognition segment; if the matching similarity is less than the preset value, the effective segment is adjusted to an invalid segment.

[0017] Several actions in the preset first action library all have AI recognition modules for corresponding actions. The AI ​​recognition module for the actions matched in the preset first action library is added to the action recognition segment, and the AI ​​recognition module for the actions in the second action library with the highest matching similarity is added to the key frame recognition segment to obtain a second video with generated AI.

[0018] The present invention converts the human body parts appearing in the first video into skeleton points, forms the skeleton point motion trajectory in the first video, determines whether the skeleton point motion trajectory is continuous, and obtains the discontinuous segment of the skeleton point motion trajectory; the discontinuous segment of the skeleton point motion trajectory is the invalid action interval. The time axis of the first video is sliced ​​based on the invalid action interval to obtain a valid segment, and finally the valid segment is divided into a key frame recognition segment or an action recognition segment, wherein the action recognition segment includes a plurality of actions, and the action matches a plurality of actions in the preset first action library. Finally, a matching AI recognition module is added to the key frame recognition segment and the action recognition segment to obtain a second video generating AI.

[0019] The first action library includes push-ups, squats, high leg raises, plank support, jumping jacks, etc. The second action library includes stretching movements, sports dance, etc. Compared with the movements in the first action library, the movements in the second action library are mainly dance movements. The dance movements referred to in the present invention include: dance, martial arts, rhythmic gymnastics, etc.

[0020] In physical education teaching, the teacher's teaching video is more inclined to the actions of the first action library and the second action library. Therefore, the setting of the present invention can quickly add the matching AI recognition module to the corresponding position and quickly obtain the second video of generating AI.

[0021] Furthermore, when obtaining the invalid action interval in the first video, in addition to judging whether the motion trajectory of the skeleton point is continuous, further judgment can be made through the voice or text in the first video, specifically including: obtaining the voice or text in the first video; comparing the obtained voice or text in the first video with the preset intermittent voice or preset intermittent text, if they match, the segment corresponding to the voice or text in the first video is the invalid action interval. Among them, the preset intermittent voice or preset intermittent text includes rest, pause and demonstration completion.

[0022] Furthermore, a method for automatically generating an AI training course is provided, the method being executed by a terminal device, the method comprising:

[0023] Presenting a video interface, wherein the video interface is used to display the second video;

[0024] In response to the AI ​​recognition module being triggered for displaying the second video, obtaining video data of the user;

[0025] Compare the user's video data with the skeletal point motion trajectory of the action corresponding to the AI ​​recognition module to obtain the number of actions and action time of the user's video data.

[0026] Compare the user's video data with the skeleton point motion trajectory of the action corresponding to the key frame recognition segment, and obtain the similarity between the skeleton point motion trajectory of the user's video data and the skeleton point motion trajectory of the action corresponding to the key frame recognition segment; based on the similarity, present the similarity score corresponding to the key frame recognition segment on the video interface.

[0027] The method for automatically generating an AI training course of the present invention is executed by a terminal device. After obtaining a second video trigger, the terminal device displays the second video. During the display of the second video, when an action recognition segment or a key frame recognition segment in the second video is played, an AI recognition module is triggered. The AI ​​recognition module obtains the user's video data, and by comparing the skeletal point motion trajectory of the user's video data, similarity data or the number of actions and action time of the user's video data are presented on the terminal device, which is more conducive to the implementation of physical education.

[0028] On the other hand, a device for automatically generating an AI training course, wherein the device includes a processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform operations according to any of the methods described above.

[0029] According to another aspect of the present application, a computer-readable medium storing instructions is provided, and when the instructions are executed, the system performs the operation according to any of the above methods.

[0030] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0031] The present invention provides a method and device for automatically generating AI training courses, which can quickly and efficiently convert recorded or live course videos into training courses with AI recognition effects. In addition, during the conversion process, the conversion is more accurate and more suitable for the rapid conversion of sports teaching videos. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0033] Figure 1 Schematic diagram of the process of this method;

[0034] Figure 2 A schematic diagram of a first video timeline showing a motion trajectory of a skeleton point formed in the first video;

[0035] Figure 3 A schematic diagram of performing slicing processing on the first video time axis;

[0036] Figure 4 is a schematic diagram of a valid segment and an invalid segment in a first video;

[0037] Figure 5 A schematic diagram of a second video for generating AI is obtained by adding a matching AI recognition module. DETAILED DESCRIPTION

[0038] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0039] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those within the scope of this description. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0040] Those skilled in the art should understand that, in the disclosure of the present invention, the orientation or position relationship indicated by the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc. are based on the orientation or position relationship shown in the drawings, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the above terms should not be understood as limiting the present invention.

[0041] It is to be understood that the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" should not be understood as a limitation on the quantity.

[0042] Example 1

[0043] Please refer to Figure 1 , Figure 1 The following is a flow chart of the method, which includes:

[0044] Step 1, obtain a first video, identify human body parts in the first video, obtain skeleton points on the human body parts in the first video, and form skeleton point motion trajectories in the first video, such as Figure 2 As shown;

[0045] Step 2, determine whether the motion trajectory of the skeleton point is continuous, obtain discontinuous segments of the motion trajectory of the skeleton point, and the discontinuous segments of the motion trajectory of the skeleton point are invalid action intervals; preferably, when obtaining the invalid action intervals, the invalid action intervals can be further determined based on the voice or text in the first video, wherein the voice or text in the first video is obtained; the voice or text in the first video is compared with the preset intermittent voice or preset intermittent text, if they match, the segment corresponding to the voice or text in the first video is the invalid action interval; the preset intermittent voice or preset intermittent text includes rest, pause and demonstration completion;

[0046] Step 3, based on the invalid action interval, the time axis of the first video is sliced ​​to obtain a plurality of slice segments, such as Figure 3 As shown;

[0047] Step 4: divide the segment into valid segments and invalid segments according to the invalid action interval, such as Figure 4 As shown;

[0048] Step 5, determine the type of the valid segment, match the valid segment with the preset AI recognition module, add the matching AI recognition module to the valid segment, and obtain the second video generated by AI, such as Figure 5 As shown;

[0049] Step 51, matching the motion trajectory of the skeleton points in the effective section with the motion trajectory of the skeleton points of several actions in the preset first action library. If the match is successful, the effective section is the action recognition segment; if the match is unsuccessful, step 53 is triggered;

[0050] Step 52, a plurality of actions in the preset first action library all have AI recognition modules corresponding to the actions, and the AI ​​recognition modules of the actions matched in the preset first action library are added to the action recognition segment;

[0051] Step 53, if the motion trajectory of the skeleton points in the effective section does not match the motion trajectory of the skeleton points of the plurality of actions in the preset first action library, then the motion trajectory of the skeleton points in the effective section is compared with the motion trajectory of the skeleton points of the plurality of actions in the preset second action library to obtain a matching similarity;

[0052] If the matching similarity is greater than or equal to the preset value, the valid segment is a key frame identification segment;

[0053] If the matching similarity is less than the preset value, the valid segment is adjusted to an invalid segment;

[0054] Adding an AI recognition module that matches the action in the second action library with the highest similarity to the key frame recognition segment to obtain a second video that generates AI; in some embodiments, the preset value is 95%;

[0055] Step 6: This method is executed by the terminal device. When in use,

[0056] Step 61, presenting a video interface, where the video interface is used to display the second video;

[0057] Step 62, in response to the AI ​​recognition module being triggered in response to displaying the second video, obtaining the video data of the user;

[0058] Step 63, comparing the user's video data with the motion trajectory of the skeleton points of the action corresponding to the AI ​​recognition module to obtain the number of actions and action time of the user's video data;

[0059] Step 64, compare the user's video data with the skeleton point motion trajectory of the action corresponding to the key frame recognition segment, obtain the similarity between the skeleton point motion trajectory of the user's video data and the skeleton point motion trajectory of the action corresponding to the key frame recognition segment; and present the similarity score corresponding to the key frame recognition segment on the video interface based on the similarity.

[0060] Among them, the movements in the preset first movement library are training movements, including squats, jumping jacks, etc.;

[0061] The actions in the preset second action library are dance actions, including stretching actions, sports dance, etc.;

[0062] The second video includes at least one key frame recognition segment and at least one action recognition segment.

[0063] In some embodiments, the device is a terminal device, which can be a fitness mirror, or other devices with interactive functions. The device with interactive functions includes an AI recognition module. When the AI ​​recognition module is triggered, the user is located in front of the acquisition device of the device with interactive functions. When the user moves according to the action in the second video, the acquisition device acquires the video image, acquires the user image in the video image, extracts the user's skeleton points according to the user image, and extracts the user's skeleton points according to the extracted user skeleton points.

[0064] The devices referred to in this application include but are not limited to user devices, network devices, or devices formed by integrating user devices and network devices through a network. The user devices include but are not limited to any mobile electronic products that can interact with users (for example, interact with users through a touchpad), such as smart phones, tablet computers, etc., and the mobile electronic products can use any operating system, such as Android operating system, iOS operating system, etc. Among them, the network device includes an electronic device that can automatically perform numerical calculations and information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits, programmable logic devices, field programmable gate arrays, digital signal processors, embedded devices, etc. The network devices include but are not limited to computers, network hosts, single network servers, multiple network server sets or multiple servers. The cloud consists of a large number of computers or network servers based on cloud computing, wherein cloud computing is a type of distributed computing, a virtual supercomputer consisting of a group of loosely coupled computer sets. The network includes but is not limited to the Internet, wide area network, metropolitan area network, local area network, VPN network, wireless self-organizing network (Ad Hoc Network), etc. Preferably, the device may also be a program running on the user device, the network device, or a device formed by integrating the user device and the network device, the network device and the touch terminal, or the network device and the touch terminal via a network.

[0065] In some embodiments, when obtaining skeleton points, the coordinates of the skeleton points of the user in each image frame are obtained, and the detected skeleton points may include: any skeleton point among the top of the head, tip of the nose, left ear, right ear, chin, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left metacarpophalangeal joint, right metacarpophalangeal joint, left fingertips, right fingertips, left hip, right hip, left knee, right knee, left ankle, right ankle, left toe tip, right toe tip, left heel, right heel, etc.

[0066] In some embodiments, the number of actions, action time, and similarity score of the user's video data are displayed on the terminal device. When displaying the number of actions, action time, and similarity score, it can be a dynamic image, a static image, or a control including an image, or text for operation instructions.

[0067] Example 2

[0068] The present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method and device for automatically generating an AI training course are implemented.

[0069] The processor may be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0070] The memory can be used to store the computer program and / or module, and the processor realizes various functions of a device for obtaining a ranking list in the invention by running or executing the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0071] Example 3

[0072] The present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the method and device for automatically generating an AI training course are implemented.

[0073] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a device or a device or used in combination with it.

[0074] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0075] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for automatically generating AI training courses, characterized in that: The method comprises: Acquire a first video, identify a human body part in the first video, acquire a skeleton point on the human body part in the first video, and form a motion trajectory of the skeleton point in the first video; Obtaining an invalid action interval in the first video; Slicing the time axis of the first video based on the invalid action interval to obtain a plurality of slice segments; The segmentation segments are divided into valid segments and invalid segments according to the invalid action intervals; Determine the type of the valid segment, match the valid segment with a preset AI recognition module, add the matching AI recognition module to the valid segment, and obtain a second video that generates AI.

2. The method for automatically generating an AI training course according to claim 1, characterized in that: Determine the type of valid segment, including: The motion trajectory of the skeleton points in the effective section is matched with the motion trajectory of the skeleton points of several actions in the preset first action library. If the match is successful, the effective section is an action recognition segment.

3. The method for automatically generating an AI training course according to claim 2, characterized in that: Several actions in the preset first action library all have AI recognition modules for corresponding actions. The AI ​​recognition modules for the actions matched in the preset first action library are added to the action recognition clip to obtain a second video for generating AI.

4. The method for automatically generating an AI training course according to claim 3, characterized in that: If the motion trajectory of the skeleton points in the effective section does not match the motion trajectory of the skeleton points of several actions in the preset first action library, then the motion trajectory of the skeleton points in the effective section is compared with the motion trajectory of the skeleton points of several actions in the preset second action library to obtain the matching similarity; If the matching similarity is greater than or equal to the preset value, the valid segment is a key frame identification segment; If the matching similarity is less than the preset value, the valid segment is adjusted to an invalid segment; An AI recognition module that matches the action in the second action library with the highest similarity is added to the key frame recognition segment to obtain a second video that generates AI.

5. The method for automatically generating an AI training course according to claim 1, characterized in that: The step of obtaining the invalid action interval in the first video specifically includes: Determine whether the motion trajectory of the skeleton point is continuous, and obtain the discontinuous fragments of the motion trajectory of the skeleton point; The discontinuous segments of the skeleton point motion trajectory are invalid action intervals.

6. The method for automatically generating an AI training course according to claim 1, characterized in that: The step of obtaining the invalid action interval in the first video specifically includes: Acquire the voice or text in the first video; The acquired voice or text in the first video is compared with the preset intermittent voice or preset intermittent text. If they match, the segment corresponding to the voice or text in the first video is an invalid action interval.

7. The method for automatically generating an AI training course according to claim 4, characterized in that: The method is performed by a terminal device, and the method includes: Presenting a video interface, wherein the video interface is used to display the second video; In response to the AI ​​recognition module being triggered for displaying the second video, obtaining video data of the user; Compare the user's video data with the skeletal point motion trajectory of the action corresponding to the AI ​​recognition module to obtain the number of actions and action time of the user's video data.

8. The method for automatically generating an AI training course according to claim 7, characterized in that: Compare the user's video data with the skeleton point motion trajectory of the action corresponding to the key frame recognition segment, and obtain the similarity between the skeleton point motion trajectory of the user's video data and the skeleton point motion trajectory of the action corresponding to the key frame recognition segment; based on the similarity, present the similarity score corresponding to the key frame recognition segment on the video interface.

9. The method for automatically generating an AI training course according to claim 1, characterized in that: The preset intermittent voice or preset intermittent text includes rest, pause and presentation completion.

10. A device for automatically generating AI training courses, characterized in that: The device is used to execute the method according to any one of claims 1 to 9.

Images