A training intelligence test method and device

Abstract

The present invention provides a training intelligent testing method and device. The training intelligent testing method of the present invention includes: using face recognition technology to verify personnel, and then using visual model technology to perform human posture recognition, action standard judgment, and automatic timing and counting on the collected training video; using big data technology to establish a training data model for the judgment result for training improvement guidance. The training intelligent testing method of the present invention can reduce the amount of manual input and improve the accuracy of the results by successfully applying machine vision technology in the training and testing process, and also avoid human cheating and unfair situations.

Description

Technical Field

[0001] The present invention relates to the field of sports training, and in particular to a training intelligence testing method and device. Background Art

[0002] Machine vision technology is widely used in food and beverage, cosmetics, pharmaceuticals, building materials and chemicals, metal processing, electronic manufacturing, packaging, automobile manufacturing and other industries, replacing artificial vision to achieve detection, measurement and control; machine vision technology mainly uses computers to simulate human visual functions, extract information from the images of objective objects, process and understand it, and ultimately use it for actual detection, measurement and control; machine vision includes digital image processing technology, deep learning technology, analog recognition technology, light source lighting technology, optical imaging technology, sensor technology, analog and digital video technology, computer software and hardware technology, human-computer interface technology, etc.

[0003] However, the successful application of machine vision technology in the field of sports training is still relatively rare nowadays, especially in the parallel bars (bar end flexion and swing flexion) training process. Most of the training tests use manual identity verification, manual judgment of movement standardization, and manual statistics of results. This manual method often leads to problems such as cheating, inaccurate and unfair results, and large amount of manual input.

[0004] In view of this, the present invention is proposed. Summary of the invention

[0005] In view of this, the present invention discloses a training intelligent testing method and device, which can reduce the amount of manual input, improve the accuracy of the results, and avoid artificial cheating and unfair situations by successfully applying machine vision technology in the training and testing process.

[0006] Specifically, the present invention is achieved through the following technical solutions:

[0007] In a first aspect, the present invention discloses a training intelligent testing method, comprising the following steps:

[0008] Use face recognition technology to verify personnel, and then use visual model technology to perform human posture recognition, action standard judgment, action comments, and automatic timing and counting on the collected training videos;

[0009] A training data model is established based on the judgment results using big data technology to be used for training evaluation and to assist in guiding training.

[0010] In a second aspect, the present invention discloses a training intelligent testing device, comprising:

[0011] Testing module: used to verify personnel using face recognition technology, and then evaluate the collected training videos and identify human postures through visual model technology to compare and judge with standard postures;

[0012] Guidance module: used to establish a training data model for the judgment result using big data technology for training guidance.

[0013] In a third aspect, the present invention discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the training intelligent testing method as described in the first aspect.

[0014] In a fourth aspect, the present invention discloses a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the training intelligent testing method as described in the first aspect are implemented.

[0015] In summary, the training intelligent testing method and intelligent testing device proposed in the present invention only require the training personnel to verify their identity by scanning their faces on a tablet and enter the preparation area according to the voice prompt of the tablet. The operator clicks the start command on the tablet, and the voice broadcast prompts the start. The personnel start training. Cameras are arranged on the vertical side of the parallel bars, and machine vision video analysis technology and deep learning training technology are used to record and collect videos of the training process and perform sports posture analysis and movement standard judgment. If the shoulder joints are higher than the elbow joints when the arms are flexed or the elbow joints are not straightened when the arms are extended, the action count fails and a voice prompt is given. Otherwise, the action count is successful and a voice prompt is given. If the feet touch the ground or the feet touch the column, the test is over, the voice prompt is given and the results are broadcast. The software platform evaluates and grades the collected sports data, and at the same time gives targeted training improvement guidance suggestions in combination with the evaluation model. The operation is convenient and the accuracy is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0017] Figure 1 A flowchart of a training intelligent testing method provided by an embodiment of the present invention;

[0018] Figure 2 A schematic diagram of the principle of evaluating and recognizing human posture provided by an embodiment of the present invention;

[0019] Figure 3 A network structure diagram for evaluating and recognizing human posture provided by an embodiment of the present invention;

[0020] Figure 4-5 The effect diagram of the training intelligent testing method provided by the embodiment of the present invention;

[0021] Figure 6 A coordinate diagram of limb joint points provided by an embodiment of the present invention;

[0022] Figure 7 A structural diagram of a training intelligent system provided by an embodiment of the present invention;

[0023] Figure 8 A flowchart of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0025] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0026] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0027] Reference Figure 1 As shown, the present invention discloses a training intelligent testing method, comprising the following steps:

[0028] S1. Use face recognition technology to verify personnel, and then use visual model technology to perform human posture recognition, action standard judgment, action comments, and automatic timing and counting on the collected training videos;

[0029] S2. Use big data technology to establish a training data model for the judgment results, so as to be used for training evaluation and to assist in guiding training.

[0030] Among them, in the S1 step, when verifying personnel, face recognition technology is used. By pre-collecting the facial feature information of the athletes, face modeling is performed, and a unified face information database is constructed. By swiping the face on the tablet, the facial feature data is extracted and transferred to the model library for facial feature comparison, and the comparison results and personnel information are fed back to complete the personnel identity verification.

[0031] After personnel verification, it is necessary to use visual model technology to conduct specific evaluation and identification of the training video. By locating, marking, and linking the joints of the human body in the video, and using image processing technology, the video frames are decomposed into pictures. The posture and joint movement range of the movement process are analyzed and identified through pictures, and the joint movement range data of each complete movement is collected and recorded, and compared and judged through the range data and standard movement rules.

[0032] Specifically, the human body posture recognition method includes:

[0033] Scale the original bitmap object of the training video, call the function from the PoseNet library to get the Person object, and scale the bitmap back to the screen size;

[0034] Draw a new bitmap on the Canvas object, draw the skeleton on the canvas using the keypoint locations obtained from the Person object, display the keypoints whose confidence exceeds a certain threshold, and use a single SurfaceView to display the output;

[0035] SurfaceView displays on the screen by acquiring, locking, and drawing on the View canvas.

[0036] The specific principle is to obtain the confidence map of the joint points and the vector map of the limbs through a series of convolutional neural networks, and then combine the confidence map and the vector map to obtain a new skeleton. The principle diagram of the convolutional neural network can be seen in Figure 2 As shown:

[0037] 1. Figure a is the input data, which is an RGB image. After passing through a series of convolutional neural networks, b and c are obtained at the same time.

[0038] 2. Figure b shows the joint point confidence map. There are as many corresponding confidence maps as the number of joint points that need to be detected. An image may have the same joint point for multiple people.

[0039] As shown in Figure b above, the confidence map on the left detects the left elbows of two people at the same time, and the confidence map on the right detects the left shoulders of two people.

[0040] Figure 3.c shows the vector diagram of the limbs, with two vector diagrams corresponding to each joint.

[0041] 4. Combining b and c gives d, which is the connection of a joint point of a limb.

[0042] 5. All d are combined to get e, which is the connection result of all the limbs we want to detect.

[0043] The specific network structure diagram during the convolution process is as follows Figure 3 As shown. Figure 3 It can be seen that F is the feature map obtained after the original image passes through the first 10 layers of the VGG19 network. The network of Branch1 obtains the confidence map S of the joint points, and the network of Branch2 obtains the vector map L of the limbs. F passes through Stage1 to obtain S1 and L1. Starting from Stage2, the input of the Stage t network consists of the confidence map S and vector map L obtained by the previous network Stage t-1 plus the feature map F. The expression is as follows:

[0044]

[0045]

[0046] The first 10 layers of the VGG19 network are circled in Table 1 below:

[0047] Table 1 Grid results

[0048]

[0049] Each Stage network can get two losses. The loss function of the Staget network is as follows:

[0050]

[0051]

[0052] in, and In order to accurately annotate the confidence map and vector map through the data set, W(p) is a binary mask. When training the model, when the data set does not annotate a certain joint point or the joint point cannot form a limb, set W(p) = 0 to avoid the increase of the "error" of this loss. The total loss function is as follows:

[0053]

[0054] Confidence map used to calculate loss during model training It is generated based on the data of the COCO dataset. Represents the confidence map of a joint point j. Since we need to detect 19 joint points, 19 confidence maps need to be generated for each image. The effect after training is as follows: Figure 4 As shown, Figure 4 The left picture in the figure is the original image, and the right picture is the confidence map of all the joint points generated. In order to facilitate comparison, the zoomed image of the original image is used as the background. The confidence map of the real joint points (all joint points) is as follows Figure 5 As shown in the right figure.

[0055] Assuming that the joint point (x, y) is the coordinate of the joint point j of character k, then A (x0, y0) is the coordinate of the upper left corner, B (x1, y1) is the coordinate of the lower right corner, A and B form a rectangle Z, and the joint point coordinate is the center of rectangle Z. Let width and height be the width and height of the image, then the definitions of x0, y0, x1, y1 are as follows,

[0056] x0 = int(max(0, x-β))

[0057] y0 = int(max(0,y-β))

[0058] x1 = int(min(width, x + β))

[0059] y1 = int(min(height, y+β))

[0060] Then, traverse each coordinate point p in rectangle Z and find the value of point p and x. j，k Gaussian kernel function to obtain the value of the coordinate point p in the confidence map The formula is as follows:

[0061]

[0062] All the joint points j of character k are composed of This constitutes the confidence map of the joint point

[0063]

[0064] During model training, vector diagram of loss function It is also generated based on the COCO dataset.

[0065] like Figure 6 As shown, assuming and represents the coordinates of the joint points j1 and j2 of the limb c of person k in the dataset image. If point p is on the limb, then is the unit vector from j1 to j2, otherwise, is 0, the formula is as follows:

[0066]

[0067] Point P meets the conditions,

[0068]

[0069] Among them, l c，k is the length of the limb, σ l is the width of the limb, v ⊥ is the perpendicular vector to the unit vector v.

[0070] In actual calculation, a specific threshold needs to be added, so a constant th is set first:

[0071]

[0072] In actual calculations, assume that the coordinates of the joints at both ends of the limb are (x1, y1) and (x2, y2), respectively. Subtract or add a threshold th from the coordinates of the joints at both ends of the limb to obtain two new coordinates (x1-th, y1-th) and (x2+th, y2+th). Take these two coordinates as the upper left corner and the lower right corner to obtain a matrix Z. Traverse each pixel point p of the matrix and calculate the distance dist from point p to the limb. If dist < th, it is considered that point p is on the limb, and point p saves the direction cosine and direction sine of limb c. Each joint point corresponds to a joint point confidence map, and each limb corresponds to two vector maps, corresponding to the direction cosine and direction sine of the limb. The formula is as follows:

[0073]

[0074] Assumptions and These are the two candidate joint points detected. These two joint points are connected to form the limb c. Now we sample between these two joint points and then use the vector graph L c Along the line segments, we measure the probability of their connection, and the calculation formula is as follows:

[0075]

[0076] Among them, p(u) is and The coordinate points inserted between are generally approximated by sampling at equal distances:

[0077]

[0078] The larger the E obtained, the greater the probability that the two joint points should be connected.

[0079] Following the above step S1, in the step S2, a training data model is established for the judgment result using big data technology for training guidance.

[0080] In the step S2, before the big data technology analysis is performed, a method for detecting human body touching the line is also involved. The specific method includes: drawing lines on the ground or poles with a camera, acquiring real-time video streams using machine vision video analysis technology, decomposing the video frames and generating images for temporary storage, using image processing technology to analyze whether the generated images have touched the line, and if so, taking photos, pushing data to the software platform, and when the training is over, giving voice prompts, and the platform counting the number of successes as the final score.

[0081] After completing the above steps, big data analysis technology is used to establish a sports training data model, a basic training knowledge base model, and an expert guidance model. By comparing and analyzing the training data and patterns, targeted suggestions for improving training guidance are comprehensively drawn.

[0082] Finally, video streaming technology is used to record the entire process of camera movement, locate, mark, and link the human body joints on the video, realize video clip cutting and storage, support video playback, and play back the video to see the human body joint lines.

[0083] The software program required for the solution of the present invention in actual application only needs an intelligent training platform and AI algorithm. The specific methods for different mobile terminal applications are slightly different, as follows:

[0084] 1. Mobile APP: Combined with the smart training management platform, it uses intelligent perception and modern scientific and technological means to carry out training evaluation, performance recording, and data analysis. Through the mobile APP, it can realize rapid support for scenarios such as daily training plan viewing, mobile training, sports circle, performance query, ranking query, evaluation guidance, and data analysis.

[0085] 2. Tablet APP: Used at the training site, it provides face recognition identity authentication, starting orders, timing, scoring, and recording. It also provides daily training reports and assessment report score inquiries, score rankings, and can view training details, playback videos, diagnose, and provide guidance suggestions.

[0086] 3. Smart training management platform: connects to front-end hardware equipment and algorithm systems, uniformly collects and records relevant training data, and provides software management and data analysis functions for basic training, including: training plans, training files, training assessments, training results, evaluation guidance, system management and other functions.

[0087] 4. AI algorithm: Utilize machine vision and machine learning technology to customize algorithms and iterate training for different training scenarios, providing capabilities such as human posture recognition, big data evaluation models, personnel marking, personnel tracking, touch line recognition, rule analysis, and data collection.

[0088] The specific software operation process can be followed as follows:

[0089] 1. Athletes scan their faces for verification and registration. If the verification fails, a voice prompt will be given. Otherwise, the tablet will display the person’s information and a voice prompt will be given to the athlete to enter the designated area.

[0090] 2. The operator gives the command, the voice broadcasts the command information, and the athlete starts training. During the process, if the shoulder joints are higher than the elbow joints when the arms are bent or the elbow joints are not straightened when the arms are extended, the action count fails and the voice prompts. Otherwise, the action counts successfully and the voice prompts.

[0091] 3. If your feet touch the ground or the pillar, the test is over and the voice prompts you and announces the result.

[0092] In summary, the training intelligent testing method of the present invention adopts machine learning intelligent recognition to replace manual assessment, reduce fraud, improve efficiency and fairness, reduce a large amount of manpower organization, and accumulate training assessment data, which can assist and guide more scientific training.

[0093] In addition, the present invention also provides a training intelligent testing device, such as Figure 7 As shown, specifically including:

[0094] Testing module 101: used to verify personnel using face recognition technology, and then evaluate and identify human postures in the collected training videos using visual model technology to compare and judge with standard postures;

[0095] Guidance module 102: used to establish a training data model for the judgment result using big data technology for training guidance.

[0096] The system is mainly composed of the above two modules. Through the construction of this device, it is easy to use machine learning to evaluate the entire training process and provide guidance and suggestions, which is convenient and accurate.

[0097] In specific implementation, the above modules can be implemented as independent entities, or can be arbitrarily combined and implemented as the same or several entities. The specific implementation of the above units can refer to the previous method embodiments, which will not be repeated here.

[0098] Figure 8 This is a schematic diagram of the structure of a computer device disclosed in the present invention. Figure 8As shown, the computer device 400 includes at least a memory 402 and a processor 401; the memory 402 is connected to the processor via a communication bus 403, and is used to store computer instructions executable by the processor 401, and the processor 401 is used to read computer instructions from the memory 402 to implement the steps of the training intelligent testing method described in any of the above embodiments.

[0099] For the above-mentioned device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0100] Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM and flash memory devices), magnetic disks (e.g., internal disks or removable disks), magneto-optical disks, and CD ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuits.

[0101] Finally, it should be noted that although this specification contains many specific implementation details, these should not be interpreted as limiting the scope of any invention or the scope of protection claimed, but are mainly used to describe the features of specific embodiments of specific inventions. Certain features described in multiple embodiments within this specification may also be implemented in combination in a single embodiment. On the other hand, the various features described in a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. In addition, although features may work in certain combinations as described above and even initially claimed as such, one or more features from the claimed combination may be removed from the combination in some cases, and the claimed combination may point to a sub-combination or a variation of a sub-combination.

[0102] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that these operations be performed in the particular order shown or performed sequentially, or requiring that all illustrated operations be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

[0103] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the particular order or sequential order shown to achieve the desired results. In some implementations, multitasking and parallel processing may be advantageous.

[0104] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. A training intelligence test method, characterized in that: It includes the following steps: Use face recognition technology for personnel verification, and then use visual model technology to perform human pose recognition, action standard evaluation, action commentary, automatic timing and counting on the collected training videos; Use big data technology to establish a training data model for the judgment results for training evaluation and assist in guiding training; The method for recognizing the human pose includes: Scale the original bitmap object of the training video, call a function from the PoseNet library to obtain a Person object, and scale the bitmap back to the screen size; Draw a new bitmap on the Canvas object, draw a skeleton on the canvas at the key point positions obtained from the Person object, display the key points with a confidence level exceeding a specific threshold, and use a single SurfaceView to display the output; The SurfaceView is displayed on the screen by obtaining, locking, and drawing on the View canvas; After passing through a series of convolutional neural networks, the original bitmap object obtains a confidence map of joint points and a vector map of limb trunks, and combines the confidence map and the vector map to obtain a new drawn skeleton; The method of the series of convolutional neural networks includes: The original bitmap passes through the network of Branch1 to obtain the confidence map S of the joint point t , the vector graph L of the limb is obtained through the network of Branch2 t , F obtains S1 and L1 through Stage 1. Starting from Stage 2, the input of the Stage t network is the confidence map S obtained by the previous network Stage t-1. t and vector L t Add the feature map F, and the expression is as follows: Each Stage network can obtain two losses. The loss function of the Staget network is as follows: In the above formula, and is the accurate confidence map and vector map annotated by the dataset data, W(p) is the binary mask, and the total loss function is as follows: Among them, the confidence map S used to calculate the loss t And the vector graph Lt is generated based on the data of the COCO dataset; Set the joint point (x, y) as the coordinates of joint point j of person k. Then, with A(x0, y0) as the upper left corner coordinate and B(x1, y1) as the lower right corner coordinate, a rectangle Z is formed by A and B, and the joint point coordinates are the center of rectangle Z. Let width and height be the width and height of the image, then the definitions of x0, y0, x1, and y1 are as follows: x0 = int(max(0, x - β)) y0 = int(max(0, y - β)) x1 = int(min(width, x + β)) y1 = int(min(height, y + β)). Then, traverse each coordinate point p in rectangle Z and find the value of point p and x. j，k Gaussian kernel function to obtain the value of the coordinate point p in the confidence map The formula is as follows: All the joint points j of character k are composed of This constitutes the confidence map of the joint point The following formula: Assumptions and represents the coordinates of the joint points j1 and j2 of the limb c of person k in the dataset image. If point p is on the limb, then is the unit vector from j1 to j2, otherwise, is 0, the formula is as follows: Where point P satisfies the following formula: Among them, l c，k is the length of the limb, σ l is the width of the limb, V ⊥ is the perpendicular vector to the unit vector v.

2. The training intelligent testing method according to claim 1, characterized in that: For the confidence map And vector graphics Add a specific threshold value in the calculation process. The specific calculation process is as follows: Assume that th in the above formula is a constant. The joint point coordinates at both ends of the limb trunk are (x1, y1) and (x2, y2) respectively. Subtract or add a threshold th to the joint point coordinates at both ends of the limb trunk to obtain two new coordinate points (x1 - th, y1 - th) and (x2 + th, y2 + th); Use the above two coordinate points as the upper left corner and the lower right corner to obtain a matrix Z. Traverse each pixel point p of the matrix Z, calculate the distance dist from point p to the limb trunk. If dist < th, it is considered that point p belongs to the limb trunk. Point p saves the direction cosine and direction sine of limb trunk c. Each joint point corresponds to a joint point confidence map, and each limb trunk corresponds to two vector maps, corresponding to the direction cosine and direction sine of the limb trunk respectively. The formula is as follows: Assumptions and are the two detected candidate joint points, which are connected to form a limb c. Samples are taken between the two joint points, and then the vector graph L is used. c Along the line segments, we measure the probability of their connection, and the calculation formula is as follows: Among them, p(u) is and The coordinate points inserted between are approximated by sampling at equal distances: The larger the obtained E, the greater the possibility that these two joint points should be connected.

3. An intelligent testing device using the training intelligent testing method according to any one of claims 1 to 2, characterized in that: It includes: Test module: used to perform personnel verification using face recognition technology, and then evaluate and recognize human poses on the collected training videos through visual model technology for comparison and judgment with the standard poses; Guidance module: used to establish a training data model for the judgment result using big data technology for training guidance.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed, the steps of the training intelligent testing method described in any one of claims 1-2 are implemented.

5. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the training intelligent testing method as described in any one of claims 1-2 are implemented.

Images