Smart sports examination method and related device based on AI technology

Abstract

The present invention discloses a smart physical examination method and related devices based on AI technology, wherein the system includes: the examinee logs in and selects a physical examination item according to face recognition information; loads a 3D standard action skeleton model and a 3D action deduction skeleton model corresponding to the physical examination item, wherein the 3D standard action skeleton model includes the motion trajectory of key skeleton points corresponding to each preset time series; collects the physical examination video of the examinee, extracts the 3D skeleton points corresponding to the examination time according to each key frame image of the video, and constructs the 3D physical examination item skeleton model of the examinee; calculates the physical examination item score of the examinee according to the 3D standard action skeleton model, the 3D action deduction skeleton model and the 3D physical examination item skeleton model. The present invention performs physical fitness tests on examinees according to the 3D standard action skeleton model, the 3D action deduction skeleton model and the 3D physical examination item skeleton model, and has a high level of scientific rationalization and low cost and high efficiency.

Description

Technical Field

[0001] The present invention relates to the field of smart sports, and in particular to a smart sports examination method and related devices based on AI technology. Background Art

[0002] In recent years, more and more attention has been paid to cultivating students' good physical qualities, and physical fitness testing is an important part of it. Traditional physical fitness test items require teachers to personally test and score, and use manual evaluation methods, which are relatively inefficient and have a large workload. At the same time, there are many human interference factors and there are many disadvantages.

[0003] At present, the normative analysis and recognition of sports movements is an important branch of smart sports. However, it is difficult for related technologies to combine AI technology with high quality to achieve normative analysis and processing of sports movement data. Although some smartphones or bracelets contain sports modes such as running and skipping rope, which can measure relevant data, including counting, exercise heart rate, calories consumed, and exercise time, they lack normative analysis and processing of sports movement data, making it difficult to meet the physical fitness tests of various schools.

[0004] At present, there are some smart sports examination systems, but they are mainly aimed at indoor scenes (mainly sports classrooms), and require the pre-installation of a large number of monitoring equipment, 3D cameras, various large screens (such as LED screens, TVs or class signs, etc.), wearable sensors, etc. to detect the posture and movement status of the human body. The price of the above equipment is relatively high, and the maintenance and update costs are also relatively high. At the same time, the reaction speed of identifying the user's movement status is also slow during video analysis and sports tests.

[0005] Therefore, the problems of high cost, low efficiency and low level of scientific rationality in sports evaluation need to be solved urgently. Summary of the invention

[0006] In view of the above problems, the present invention is proposed to provide an intelligent sports examination method, device, computing equipment and computer storage medium based on AI technology to overcome the problems of high cost, low efficiency and low scientific rationality level of the above sports assessment.

[0007] According to one aspect of the present invention, a smart sports examination method based on AI technology is provided, comprising:

[0008] The examinee logs in and selects a physical test item according to the face recognition information, wherein the physical test item is a single person sport, which includes at least one of the following: gymnastics, sit-ups, pull-ups, dips, jumping jacks, hip bridges, squats, and martial arts routines;

[0009] Loading the 3D standard action skeleton model and the 3D action deduction skeleton model corresponding to the physical examination item, wherein the 3D standard action skeleton model includes the motion trajectory of the key skeleton points corresponding to each preset time series;

[0010] Collect the physical examination video of the examinee, extract the 3D skeleton points corresponding to the examination time according to each key frame image of the video, and construct the 3D physical examination item skeleton model of the examinee;

[0011] The physical examination item scores of the examinee are calculated based on the 3D standard action skeleton model, the 3D action deduction skeleton model and the 3D physical examination item skeleton model.

[0012] In an optional manner, extracting 3D skeleton points corresponding to the test time according to each key frame image of the video further includes:

[0013] Convert the 2D images of each key frame of the video into ITK data format;

[0014] Convert the 2D image in ITK data format to a DICOM format image;

[0015] According to the skeleton_3D method, the 3D skeleton points of the DICOM format image corresponding to the examination time are extracted.

[0016] In an optional manner, the step of extracting 3D skeleton points of the DICOM format image corresponding to the examination time according to the skeleton_3D method further comprises:

[0017] Extract the 2D skeleton points of the DICOM format image corresponding to the examination time according to the skeleton_3D method;

[0018] For the key frame 2D skeleton points of any time series t, the 3D skeleton points corresponding to the key frame 2D skeleton points of the time series t are calculated according to the key frame 2D skeleton points, 2D skeleton point contour ratio and 2D skeleton point contour displacement information of the previous time series t-1.

[0019] In an optional manner, the calculating the sports test item score of the examinee according to the 3D standard action skeleton model, the 3D action deduction skeleton model and the 3D sports test item skeleton model further includes:

[0020] For a key frame 3D skeleton point of any time sequence in the 3D sports test item skeleton model, calculate a first spatial distance mean square error between the key frame 3D skeleton point coordinates and the key frame 3D skeleton point coordinates in the 3D standard action skeleton model corresponding to the time sequence, and a second spatial distance mean square error between the key frame 3D skeleton point coordinates and the key frame 3D skeleton point coordinates in the 3D action deduction skeleton model corresponding to the time sequence;

[0021] Calculate the total first spatial distance mean square error and the second spatial distance mean square error of all key frame 3D skeleton points of the examinee according to the first spatial distance mean square error and the second spatial distance mean square error respectively;

[0022] If the first spatial distance mean square error total value and the second spatial distance mean square error total value are respectively between the first preset range and the second preset range, then it is determined that the candidate's physical examination score is qualified.

[0023] In an optional manner, the method further includes:

[0024] For any key frame 3D skeleton point in the 3D sports test item skeleton model, the Dunn index of the key frame 3D skeleton point and each key frame 3D skeleton point in the 3D standard action skeleton model and each key frame 3D skeleton point in the 3D action deduction skeleton model is calculated according to the DBSCAN clustering algorithm;

[0025] If the Dunn index is greater than a preset threshold, the key frame 3D skeleton point is determined to be qualified;

[0026] The physical examination score of the candidate is calculated based on the proportion of the number of 3D skeleton points that meet the preset threshold.

[0027] In an optional manner, the 3D standard action skeleton model also includes the number of repeated actions and the test time.

[0028] According to another aspect of the present invention, there is provided a smart sports examination device based on AI technology, comprising:

[0029] An examination selection module, used for examinees to log in and select physical examination items according to face recognition information, wherein the physical examination items are individual sports, including at least one of the following: gymnastics, sit-ups, pull-ups, dips, jumping jacks, hip bridges, squats, and martial arts routines;

[0030] A model loading module, used to load the 3D standard action skeleton model and the 3D action deduction skeleton model corresponding to the physical examination items, wherein the 3D standard action skeleton model includes the motion trajectory of the key skeleton points corresponding to each preset time series;

[0031] The test acquisition module is used to collect the physical test video of the examinee, extract the 3D skeleton points corresponding to the test time according to each key frame image of the video, and construct the 3D physical test item skeleton model of the examinee;

[0032] The score calculation module is used to calculate the examinee's sports test item score based on the 3D standard action skeleton model, the 3D action deduction skeleton model and the 3D sports test item skeleton model.

[0033] According to another aspect of the present invention, there is provided a computing device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0034] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the above-mentioned smart sports examination method based on AI technology.

[0035] According to another aspect of the present invention, a computer storage medium is provided, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to perform operations corresponding to the above-mentioned intelligent sports examination method based on AI technology.

[0036] According to the scheme provided by the present invention, it includes: the examinee logs in and selects a physical examination item according to face recognition information, wherein the physical examination item is a single-person sport, which includes at least one of the following: gymnastics, sit-ups, pull-ups, dips, jump jacks, hip bridges, squats, and martial arts routines; loads the 3D standard action skeleton model and 3D action deduction skeleton model corresponding to the physical examination item, wherein the 3D standard action skeleton model includes the motion trajectory of the key skeleton points corresponding to each preset time sequence; collects the physical examination video of the examinee, extracts the 3D skeleton points corresponding to the examination time according to each key frame image of the video, and constructs the 3D physical examination item skeleton model of the examinee; calculates the physical examination item score of the examinee according to the 3D standard action skeleton model, the 3D action deduction skeleton model, and the 3D physical examination item skeleton model. The present invention performs physical fitness tests on examinees according to the 3D standard action skeleton model, the 3D action deduction skeleton model, and the 3D physical examination item skeleton model, and has a high level of scientific rationalization and low cost and high efficiency.

[0037] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0039] Figure 1 A schematic diagram showing a flow chart of a smart sports examination method based on AI technology according to an embodiment of the present invention;

[0040] Figure 2 A schematic diagram of 3D skeleton rendering according to an embodiment of the present invention is shown;

[0041] Figure 3 A schematic diagram of a sports test key frame according to an embodiment of the present invention is shown;

[0042] Figure 4 A schematic diagram of 3D skeleton points of a student cheating in an embodiment of the present invention is shown;

[0043] Figure 5 A schematic diagram of a skeleton trajectory according to an embodiment of the present invention is shown;

[0044] Figure 6 A schematic diagram showing the displacement change of a candidate according to an embodiment of the present invention is shown;

[0045] Figure 7 A structural framework diagram of a smart sports examination device based on AI technology according to an embodiment of the present invention is shown;

[0046] Figure 8 A schematic diagram of the structure of a computing device according to an embodiment of the present invention is shown.

[0047] Figure numerals: 1. Initial state; 2. Matching successful; 3. State missing; 4. Initial state of the next cycle. DETAILED DESCRIPTION

[0048] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0049] Figure 1 The flowchart of the smart physical examination method of AI technology in an embodiment of the present invention is shown. The method calculates the physical examination item scores of the examinee based on the 3D standard action skeleton model, the 3D action deduction skeleton model and the 3D physical examination item skeleton model. Specifically, Figure 1As shown, the following steps are included:

[0050] Step S101, the examinee logs in according to the face recognition information and selects a physical examination item, wherein the physical examination item is an individual sport, which includes at least one of the following: gymnastics, sit-ups, pull-ups, dips, jumping jacks, hip bridges, squats, and martial arts routines.

[0051] In the present embodiment, only one mobile phone is needed to carry out sports evaluation. During evaluation, the examinee opens the sports test APP in the mobile phone, fixes the mobile phone on one side of the venue, sets the angle of the mobile phone appropriately, adjusts the distance between the body and the mobile phone, until the human body is completely located in the identification frame, and then the sports evaluation can be started. According to the type of sports, such as sit-ups, rope skipping, etc., sports evaluation can be carried out at the same time. Each examinee opens the sports test APP in his mobile phone and selects a venue of about 3 square meters, and then the sports evaluation can be carried out. By using the examinee's smart phone to carry out sports evaluation, it is only necessary to provide the corresponding venue, which solves the problem of the high price of traditional sports evaluation equipment, and also does not need on-site assistants (coaches, examiners or relatives and friends) to monitor, score and time the examinee in real time, which not only improves the efficiency of sports evaluation, but also saves a lot of human resource costs.

[0052] The examinee registers and logs in to the physical examination system APP according to the face recognition information, and selects the corresponding physical examination items. In this embodiment, the physical examination items are single-person sports, including at least one of the following: gymnastics, sit-ups, pull-ups, dips, jumping jacks, hip bridges, squats, and martial arts routines. The above-mentioned single-person sports are usually limited to a certain displacement space, and the distance can be changed in the front, back, left, and right directions, but the distance of the change should be within the shooting and action recognition range of the mobile phone camera, usually about 2 meters to 8 meters for better recognition effect.

[0053] Step S102, loading the 3D standard action skeleton model and the 3D action deduction skeleton model corresponding to the physical examination item, wherein the 3D standard action skeleton model includes the motion trajectory of the key skeleton points corresponding to each preset time series.

[0054] In order to facilitate the rapid calculation of the physical education scores of the examination, after the examinee logs in and uses the examinee's preparation time, the system pre-loads the 3D standard action skeleton model and the 3D action deduction skeleton model corresponding to the physical education examination items. In this embodiment, the 3D standard action skeleton model includes the motion trajectory of the key skeleton points corresponding to each preset time series, for example, including the time series information of the model, the skeleton point coordinate information of each key frame, the examination time, the standard and the corresponding index information, etc. The 3D action deduction skeleton model includes the common incorrect motion trajectory of the key skeleton points corresponding to each preset time series, for example, including the time series information of the model, the skeleton point coordinate information of each incorrect key frame, the cheating action information and the corresponding index information, etc.

[0055] Optionally, the method of pre-building a 3D standard action skeleton model and / or a 3D action deduction skeleton model includes:

[0056] Step 1: Modeling through 3D production software or 3D animation software (such as 3Dmax, CAD, Maya and other tools), for example, constructing human skeleton points by drawing points, lines and triangles.

[0057] Step 2: According to the preset key frame actions, the human skeleton points are stretched, sectioned, rotated, synthesized, and other operations are performed to construct the key frame skeleton points for the physical fitness test.

[0058] Step three, setting the motion trajectory and corresponding time series of each physical fitness test key frame skeleton point, generating a 3D standard action skeleton model and / or a 3D action deduction skeleton model.

[0059] Step 4: Export various parameters of the 3D standard action skeleton model and / or the 3D action deduction skeleton model, such as the skeleton point coordinate information of each key frame and the corresponding time series.

[0060] Step 5: Associate the test time and other information with the database or file corresponding to the above model, and create corresponding index information.

[0061] Optionally, the method of pre-building a 3D standard action skeleton model and a 3D action deduction skeleton model further includes:

[0062] Draw a 3D animation model for the standard action and / or deduction action through the wearable device, and export the coordinate information of the skeleton points of each key frame and the corresponding time series and other information, or establish the above 3D model through a deep learning neural network model.

[0063] Optionally, for individual sports such as gymnastics and martial arts routines, corresponding preset time sequences are set according to the prescribed movements in each section; for individual sports with standardized counting such as sit-ups, pull-ups, dips, jumping jacks, hip bridges, squats, etc., corresponding preset time sequences are set only according to the repeated movements.

[0064] Step S103, collecting the physical examination video of the examinee, extracting the 3D skeleton points corresponding to the examination time according to each key frame image of the video, and constructing the 3D physical examination item skeleton model of the examinee.

[0065] Specifically, the physical examination video of the examinee is collected, each key frame image is segmented according to the video, the 3D skeleton points corresponding to the examination time in each key frame image are extracted, and a 3D physical examination item skeleton model of the examinee is constructed.

[0066] In an optional manner, extracting 3D skeleton points corresponding to the test time according to each key frame image of the video further includes:

[0067] Convert the 2D images of each key frame of the video into ITK data format;

[0068] Convert the 2D image in the ITK data format to the DICOM format image;

[0069] According to the skeleton_3D method, the 3D skeleton points of the DICOM format image corresponding to the examination time are extracted.

[0070] Specifically, the 2D images of each key frame of the video are converted into ITK (Insight Segmentation and Registration Toolkit) data format. ITK can be well used for 3D segmentation, matching and statistics, and supports toolboxes including multiple functions, such as threshold filters, edge detectors, neighborhood filters, mathematical morphology, smoothing filters and other functions.

[0071] Convert the 2D image in ITK data format to DICOM format image. The key bones in a single DICOM can include multiple continuous motion sets, which can store dynamic key bone motion trajectories and can be compressed for use in other formats.

[0072] According to a skeleton_3D method, such as the skeleton_3D method of the scikit image processing library in the Python programming language, which simplifies a binary object into a 1-pixel wide representation that can be used for feature extraction and / or representing the topology of an object, border pixels are identified and removed if they do not disconnect the corresponding objects to extract 3D skeleton points of the DICOM format image corresponding to the examination time.

[0073] In an optional manner, the step of extracting 3D skeleton points of the DICOM format image corresponding to the examination time according to the skeleton_3D method further comprises:

[0074] Extract the 2D skeleton points of the DICOM format image corresponding to the examination time according to the skeleton_3D method;

[0075] For the key frame 2D skeleton points of any time series t, the 3D skeleton points corresponding to the key frame 2D skeleton points of the time series t are calculated according to the key frame 2D skeleton points, 2D skeleton point contour ratio and 2D skeleton point contour displacement information of the previous time series t-1.

[0076] In some physical fitness tests such as frog jumps and gymnastics, candidates move forward, backward, left and right to a large extent, such as Figure 6 As shown, the left figure is the skeleton point at the current time t, and the right figure is the skeleton point at the next time t+1. The contour ratio and displacement information of the 2D skeleton point at the next time t+1 have undergone significant changes. For the key frame 2D skeleton point of any time series t, according to the key frame 2D skeleton point, 2D skeleton point contour ratio and 2D skeleton point contour displacement information of the previous time series t-1, the three-dimensional coordinate transformation of the key frame 2D skeleton point of the time series t-1 is calculated according to the contour ratio and contour displacement information. After correcting the three-dimensional coordinates, the corresponding key frame 2D skeleton point of the time series t is calculated as follows Figure 2 The 3D skeleton points shown.

[0077] Step S104, calculating the physical examination item score of the examinee according to the 3D standard action skeleton model, the 3D action deduction skeleton model and the 3D physical examination item skeleton model.

[0078] Specifically, according to the matching 3D physical examination item skeleton model between the 3D standard action skeleton model and the 3D action deduction skeleton model, the physical examination item score of the examinee is calculated.

[0079] For example, according to the motion trajectories of the key skeleton points corresponding to each preset time series in the 3D standard action skeleton model, for example, including the time series information of the model, the coordinate information of the skeleton points of each key frame, the test time, the standard and the corresponding index information, the 3D sports test item skeleton model is matched to calculate the score of the physical test item of the examinee. And, according to the 3D action deduction skeleton model, including the common incorrect motion trajectories of the key skeleton points corresponding to each preset time series, for example, including the time series information of the model, the coordinate information of the skeleton points of each incorrect key frame, the cheating action information and the corresponding index information, the 3D sports test item skeleton model is matched to calculate the deduction score of the physical test item of the examinee.

[0080] In an optional manner, the calculating the sports test item score of the examinee according to the 3D standard action skeleton model, the 3D action deduction skeleton model and the 3D sports test item skeleton model further includes:

[0081] For a key frame 3D skeleton point of any time sequence in the 3D sports test item skeleton model, calculate a first spatial distance mean square error between the key frame 3D skeleton point coordinates and the key frame 3D skeleton point coordinates in the 3D standard action skeleton model corresponding to the time sequence, and a second spatial distance mean square error between the key frame 3D skeleton point coordinates and the key frame 3D skeleton point coordinates in the 3D action deduction skeleton model corresponding to the time sequence;

[0082] Calculate the total first spatial distance mean square error and the second spatial distance mean square error of all key frame 3D skeleton points of the examinee according to the first spatial distance mean square error and the second spatial distance mean square error respectively;

[0083] If the first spatial distance mean square error total value and the second spatial distance mean square error total value are respectively between the first preset range and the second preset range, then it is determined that the candidate's physical examination score is qualified.

[0084] Specifically, for the key frame 3D skeleton point of any time sequence in the 3D sports test item skeleton model, the first spatial distance mean square error between the coordinates of the key frame 3D skeleton point and the coordinates of the key frame 3D skeleton point in the 3D standard action skeleton model corresponding to the time sequence is calculated. For example, the key frame 3D skeleton point coordinates are ((1,1,1), (1,2,1), (1,1,2), (8,2,1), (8,9,0), (8,9,2)), and the key frame 3D skeleton point coordinates in the 3D standard action skeleton model corresponding to the time series are ((1.4,1,1), (1,2,1.1), (1,1.2,2), (8,2,1), (8.3,9,0), (8,9.2,2)). The spatial distance of each skeleton point is calculated according to the spatial Euclidean distance (called the first spatial distance), and the mean square error of the first spatial distance and the second spatial distance between the key frame 3D skeleton point coordinates and the key frame 3D skeleton point coordinates in the 3D action deduction skeleton model corresponding to the time series are calculated.

[0085] The first spatial distance mean square error and the second spatial distance mean square error of all key frame 3D skeleton points of the examinee are calculated respectively according to the first spatial distance mean square error and the second spatial distance mean square error. The mean square error can be used to determine the degree to which the test skeleton point data deviates from the standard skeleton point data.

[0086] Determine whether the total mean square error of the first spatial distance and the total mean square error of the second spatial distance are within a preset range. If the total mean square error of the first spatial distance and the total mean square error of the second spatial distance are respectively between the first preset range and the second preset range, then determine that the candidate's physical examination score is qualified.

[0087] In an optional manner, the method further includes:

[0088] For any key frame 3D skeleton point in the 3D sports test item skeleton model, the Dunn index of the key frame 3D skeleton point and each key frame 3D skeleton point in the 3D standard action skeleton model and each key frame 3D skeleton point in the 3D action deduction skeleton model is calculated according to the DBSCAN clustering algorithm;

[0089] If the Dunn index is greater than a preset threshold, the key frame 3D skeleton point is determined to be qualified;

[0090] The physical examination score of the candidate is calculated based on the proportion of 3D skeleton points that meet the preset threshold.

[0091] Specifically, for any key frame 3D skeleton point in the 3D sports test item skeleton model, the Dunn index of the key frame 3D skeleton point and each key frame 3D skeleton point in the 3D standard action skeleton model and each key frame 3D skeleton point in the 3D action deduction skeleton model is calculated according to the DBSCAN clustering algorithm. Among them, the DBSCAN clustering algorithm is an unsupervised machine learning algorithm based on density clustering (DBSCAN). If the mutual distance of the data points is less than or equal to the specified epsilon, then they are of the same class, and the neighborhood of the minPts number within the radius of a neighborhood is considered to be a cluster. DBSCAN can determine whether two points are similar and the distance between them belonging to the same class. The Dunn index refers to the minimum value of the closest distance between any two clusters, divided by the maximum value of the distance between the two points farthest from each other in any cluster. If the minimum value of the closest distance between any two clusters is larger (that is, the sample distances between clusters are very far from each other), the Dunn index is larger; if the maximum value of the distance between the two points farthest from each other in any cluster is smaller (that is, the sample distances within the cluster are very close), the Dunn index is larger. If the Dunn index is greater than a preset threshold, the key frame 3D skeleton point is determined to be qualified. Then, the physical examination score of the examinee is calculated based on the proportion of the number of 3D skeleton points that meet the preset threshold.

[0092] In an optional manner, the method further includes:

[0093] For the 3D skeleton points of any key frame in the 3D sports test item skeleton model, calculate the number of the 3D skeleton points of the key frame;

[0094] If the number of 3D skeleton points in the key frame is greater than the preset number of skeleton points, the candidate's score is judged to be unqualified.

[0095] Specifically, for the 3D skeleton points of any key frame in the skeleton model of the 3D sports test item, the number of 3D skeleton points of the key frame is calculated. For example, if the number of 3D skeleton points of the key frame is 27 (the fixed identification number of skeleton points in the 3D model is 30), it may indicate that the test is in a sideways state. If the number of 3D skeleton points of the key frame is greater than the preset number of skeleton points, the number of 3D skeleton points of each person under normal circumstances cannot be greater than the fixed identification number of 3D skeleton points, indicating that the 3D skeleton points of other people appear in the key frame of the examinee. For example, the examinee may complete the pull-up physical fitness test with the help of other people, and then the examinee's score is judged to be unqualified. Optionally, whether the proportion of the number of key frames greater than the preset number of skeleton points exceeds the preset proportion, if it exceeds, the examinee's score is judged to be unqualified.

[0096] In an optional manner, the 3D standard action skeleton model also includes the number of repeated actions and the test time. For example, for single-person sports such as pull-ups and rope skipping, only a repetitive cycle action trajectory model is established, such as Figure 3 As shown, by examining the state of each test cycle of the examinee, the number of repeated actions that meet the standards is judged. It can be seen from the figure that the examinee starts the cyclical repeated action from the initial state 1, starting from the initial state 1, where the action of the fourth key frame is recorded as successful match 2, and the action of the seventh key frame is missing, which is recorded as state missing 3 (the action is unqualified and not included in the score), and then enters the initial state 4 of the next cycle, and the counting is repeated.

[0097] The solution provided by the above embodiment of the present invention is that the examinee logs in and selects a physical examination item according to face recognition information; loads a 3D standard action skeleton model and a 3D action deduction skeleton model corresponding to the physical examination item, wherein the 3D standard action skeleton model includes the motion trajectory of key skeleton points corresponding to each preset time sequence; collects the physical examination video of the examinee, extracts the 3D skeleton points corresponding to the examination time according to each key frame image of the video, and constructs the 3D physical examination item skeleton model of the examinee; calculates the physical examination item score of the examinee according to the 3D standard action skeleton model, the 3D action deduction skeleton model and the 3D physical examination item skeleton model. The present invention performs physical fitness tests on examinees according to the 3D standard action skeleton model, the 3D action deduction skeleton model and the 3D physical examination item skeleton model, and has a high level of scientific rationalization and low cost and high efficiency.

[0098] Figure 7The structure diagram of the smart sports test device based on AI technology according to an embodiment of the present invention is shown. The smart sports test device based on AI technology 700 includes: a test selection module 710 , a model loading module 720 , a test collection module 730 and a score calculation module 740 .

[0099] The test selection module 710 is used for the examinee to log in and select a physical test item according to the face recognition information, wherein the physical test item is a single person sport, which includes at least one of the following: gymnastics, sit-ups, pull-ups, dips, jumping jacks, hip bridges, squats, and martial arts routines;

[0100] The model loading module 720 is used to load the 3D standard action skeleton model and the 3D action deduction skeleton model corresponding to the sports test items, wherein the 3D standard action skeleton model includes the motion trajectory of the key skeleton points corresponding to each preset time series;

[0101] The test acquisition module 730 is used to collect the physical test video of the examinee, extract the 3D skeleton points corresponding to the test time according to each key frame image of the video, and construct the 3D physical test item skeleton model of the examinee;

[0102] The score calculation module 740 is used to calculate the sports test item score of the examinee based on the 3D standard action skeleton model, the 3D action deduction skeleton model and the 3D sports test item skeleton model.

[0103] In an optional manner, the test collection module 730 is further used to:

[0104] Convert the 2D images of each key frame of the video into ITK data format;

[0105] Convert the 2D image in ITK data format to a DICOM format image;

[0106] According to the skeleton_3D method, the 3D skeleton points of the DICOM format image corresponding to the examination time are extracted.

[0107] In an optional manner, the test collection module 730 is further used to:

[0108] Extract the 2D skeleton points of the DICOM format image corresponding to the examination time according to the skeleton_3D method;

[0109] For the key frame 2D skeleton points of any time series t, the 3D skeleton points corresponding to the key frame 2D skeleton points of the time series t are calculated according to the key frame 2D skeleton points, 2D skeleton point contour ratio and 2D skeleton point contour displacement information of the previous time series t-1.

[0110] In an optional manner, the score calculation module 740 is further used to:

[0111] For a key frame 3D skeleton point of any time sequence in the 3D sports test item skeleton model, calculate a first spatial distance mean square error between the key frame 3D skeleton point coordinates and the key frame 3D skeleton point coordinates in the 3D standard action skeleton model corresponding to the time sequence, and a second spatial distance mean square error between the key frame 3D skeleton point coordinates and the key frame 3D skeleton point coordinates in the 3D action deduction skeleton model corresponding to the time sequence;

[0112] Calculate the total first spatial distance mean square error and the second spatial distance mean square error of all key frame 3D skeleton points of the examinee according to the first spatial distance mean square error and the second spatial distance mean square error respectively;

[0113] If the first spatial distance mean square error total value and the second spatial distance mean square error total value are respectively between the first preset range and the second preset range, then it is determined that the candidate's physical examination score is qualified.

[0114] In an optional manner, the score calculation module 740 is further used to:

[0115] For any key frame 3D skeleton point in the 3D sports test item skeleton model, the Dunn index of the key frame 3D skeleton point and each key frame 3D skeleton point in the 3D standard action skeleton model and each key frame 3D skeleton point in the 3D action deduction skeleton model is calculated according to the DBSCAN clustering algorithm;

[0116] If the Dunn index is greater than a preset threshold, the key frame 3D skeleton point is determined to be qualified;

[0117] The physical examination score of the candidate is calculated based on the proportion of the number of 3D skeleton points that meet the preset threshold.

[0118] In an optional manner, the score calculation module 740 is further used to:

[0119] For the 3D skeleton points of any key frame in the 3D sports test item skeleton model, calculate the number of the 3D skeleton points of the key frame;

[0120] If the number of 3D skeleton points in the key frame is greater than the preset number of skeleton points, the candidate's score is judged to be unqualified.

[0121] In an optional manner, the 3D standard action skeleton model also includes the number of repeated actions and the test time.

[0122] The solution provided by the above embodiment of the present invention is that the examinee logs in and selects a physical examination item according to face recognition information; loads a 3D standard action skeleton model and a 3D action deduction skeleton model corresponding to the physical examination item, wherein the 3D standard action skeleton model includes the motion trajectory of key skeleton points corresponding to each preset time sequence; collects the physical examination video of the examinee, extracts the 3D skeleton points corresponding to the examination time according to each key frame image of the video, and constructs the 3D physical examination item skeleton model of the examinee; calculates the physical examination item score of the examinee according to the 3D standard action skeleton model, the 3D action deduction skeleton model and the 3D physical examination item skeleton model. The present invention performs physical fitness tests on examinees according to the 3D standard action skeleton model, the 3D action deduction skeleton model and the 3D physical examination item skeleton model, and has a high level of scientific rationalization and low cost and high efficiency.

[0123] Figure 8 The schematic diagram of the structure of the computing device embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the computing device.

[0124] like Figure 8 As shown, the computing device may include: a processor (processor) 802 , a communication interface (Communications Interface) 804 , a memory (memory) 806 , and a communication bus 808 .

[0125] The processor 802, the communication interface 804, and the memory 806 communicate with each other via the communication bus 808. The communication interface 804 is used to communicate with other devices such as a client or other server network elements. The processor 802 is used to execute the program 810, which can specifically execute the relevant modules and steps in the embodiment of the smart sports examination method based on AI technology.

[0126] Specifically, the program 810 may include program codes, which include computer operation instructions.

[0127] The processor 802 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the computing device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0128] The memory 806 is used to store the program 810. The memory 806 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0129] An embodiment of the present invention provides a non-volatile computer storage medium, which stores at least one executable instruction, and the computer executable instruction can execute the above-mentioned smart sports examination method based on AI technology.

[0130] The algorithm or display provided herein is not inherently related to any particular computer, virtual system or other equipment. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious to construct the structure required for this type of system. In addition, the embodiment of the present invention is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description made to specific languages ​​above is for disclosing the best mode of the present invention.

[0131] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0132] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, the various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the following intention: that the claimed invention requires more features than the features explicitly recited in each claim. More specifically, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present invention.

[0133] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0134] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0135] The various component embodiments of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some or all components according to embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., computer program and computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0136] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

Claims

1. A smart sports examination method based on AI technology, It is characterized in that include: The examinee logs in and selects a physical test item according to the face recognition information, wherein the physical test item is a single person sport, which includes at least one of the following: gymnastics, sit-ups, pull-ups, dips, jumping jacks, hip bridges, squats, and martial arts routines; Loading the 3D standard action skeleton model and the 3D action deduction skeleton model corresponding to the physical examination item, wherein the 3D standard action skeleton model includes the motion trajectory of the key skeleton points corresponding to each preset time series; Collect the physical examination video of the examinee, extract the 3D skeleton points corresponding to the examination time according to each key frame image of the video, and construct the 3D physical examination item skeleton model of the examinee; Calculate the physical examination item score of the examinee according to the 3D standard action skeleton model, the 3D action deduction skeleton model and the 3D physical examination item skeleton model; The calculating the sports test item score of the examinee according to the 3D standard action skeleton model, the 3D action deduction skeleton model and the 3D sports test item skeleton model further comprises: For a key frame 3D skeleton point of any time sequence in the 3D sports test item skeleton model, calculate a first spatial distance mean square error between the key frame 3D skeleton point coordinates and the key frame 3D skeleton point coordinates in the 3D standard action skeleton model corresponding to the time sequence, and a second spatial distance mean square error between the key frame 3D skeleton point coordinates and the key frame 3D skeleton point coordinates in the 3D action deduction skeleton model corresponding to the time sequence; Calculate the total first spatial distance mean square error and the second spatial distance mean square error of all key frame 3D skeleton points of the examinee according to the first spatial distance mean square error and the second spatial distance mean square error respectively; If the first spatial distance mean square error total value and the second spatial distance mean square error total value are respectively between the first preset range and the second preset range, then the physical examination item score of the examinee is determined to be qualified; The 3D motion deduction skeleton model includes common erroneous motion trajectories of key skeleton points corresponding to each preset time series.

2. The smart sports examination method based on AI technology according to claim 1, It is characterized in that The step of extracting 3D skeleton points corresponding to the test time according to each key frame image of the video further includes: Convert the 2D images of each key frame of the video into ITK data format; Convert the 2D image in ITK data format to a DICOM format image; According to the skeleton_3D method, the 3D skeleton points of the DICOM format image corresponding to the examination time are extracted.

3. The intelligent sports examination method based on AI technology according to claim 2, It is characterized in that The step of extracting the 3D skeleton points of the DICOM format image corresponding to the examination time according to the skeleton_3D method further comprises: Extract the 2D skeleton points of the DICOM format image corresponding to the examination time according to the skeleton_3D method; For the key frame 2D skeleton points of any time series t, the 3D skeleton points corresponding to the key frame 2D skeleton points of the time series t are calculated according to the key frame 2D skeleton points, 2D skeleton point contour ratio and 2D skeleton point contour displacement information of the previous time series t-1.

4. The intelligent sports examination method based on AI technology according to claim 1, It is characterized in that The method further comprises: For any key frame 3D skeleton point in the 3D sports test item skeleton model, the Dunn index of the key frame 3D skeleton point and each key frame 3D skeleton point in the 3D standard action skeleton model and each key frame 3D skeleton point in the 3D action deduction skeleton model is calculated according to the DBSCAN clustering algorithm; If the Dunn index is greater than a preset threshold, the key frame 3D skeleton point is determined to be qualified; The physical examination score of the candidate is calculated based on the proportion of the number of 3D skeleton points that meet the preset threshold.

5. The intelligent sports examination method based on AI technology according to claim 1, It is characterized in that The method further comprises: For the 3D skeleton points of any key frame in the 3D sports test item skeleton model, calculate the number of the 3D skeleton points of the key frame; If the number of 3D skeleton points in the key frame is greater than the preset number of skeleton points, the candidate's score is judged to be unqualified.

6. The intelligent sports examination method based on AI technology according to claim 1, It is characterized in that The 3D standard action skeleton model also includes the number of repeated actions and the test time.

7. A smart sports examination device based on AI technology, It is characterized in that include: An examination selection module, used for examinees to log in and select physical examination items according to face recognition information, wherein the physical examination items are individual sports, including at least one of the following: gymnastics, sit-ups, pull-ups, dips, jumping jacks, hip bridges, squats, and martial arts routines; A model loading module, used to load the 3D standard action skeleton model and the 3D action deduction skeleton model corresponding to the physical examination items, wherein the 3D standard action skeleton model includes the motion trajectory of the key skeleton points corresponding to each preset time series; The test acquisition module is used to collect the physical test video of the examinee, extract the 3D skeleton points corresponding to the test time according to each key frame image of the video, and construct the 3D physical test item skeleton model of the examinee; A score calculation module, used for calculating the physical examination item score of the examinee according to the 3D standard action skeleton model, the 3D action deduction skeleton model and the 3D physical examination item skeleton model; For a key frame 3D skeleton point of any time sequence in the 3D sports test item skeleton model, calculate a first spatial distance mean square error between the key frame 3D skeleton point coordinates and the key frame 3D skeleton point coordinates in the 3D standard action skeleton model corresponding to the time sequence, and a second spatial distance mean square error between the key frame 3D skeleton point coordinates and the key frame 3D skeleton point coordinates in the 3D action deduction skeleton model corresponding to the time sequence; Calculate the total first spatial distance mean square error and the second spatial distance mean square error of all key frame 3D skeleton points of the examinee according to the first spatial distance mean square error and the second spatial distance mean square error respectively; If the first spatial distance mean square error total value and the second spatial distance mean square error total value are respectively between the first preset range and the second preset range, then the physical education test item score of the examinee is determined to be qualified; The 3D motion deduction skeleton model includes common erroneous motion trajectories of key skeleton points corresponding to each preset time series.

8. A computing device, include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the intelligent sports examination method based on AI technology as described in any one of claims 1-6.

9. A computer storage medium, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to perform operations corresponding to the intelligent sports examination method based on AI technology as described in any one of claims 1-6.

Images