A method for accurately evaluating a motion state based on a modern machine learning model

Abstract

The application provides a movement state accurate evaluation method based on a modern machine learning model, and relates to the field of fitness. Through training and learning of face image samples and limb action video samples in different fatigue states in corresponding convolutional neural network models respectively, a face fatigue state detection model and a limb action fatigue detection model are obtained. Then, the movement video of a mover is acquired, and the face image and limb action information of the mover in the recognized movement video are sent into the face fatigue state detection model and the limb action fatigue detection model respectively, and then, comprehensive judgment is performed according to the face fatigue detection result and the limb fatigue detection result, so that the fatigue state of the mover can be accurately and quickly recognized. The method is simple and fast, can be based on face images and limb action information, combined with target detection and super-resolution reconstruction technologies, and utilizes a convolutional neural network model to accurately evaluate the fatigue state of the mover.

Description

Technical Field

[0001] This invention relates to the field of fitness, and more specifically, to a method for accurately assessing exercise status based on modern machine learning models. Background Technology

[0002] With societal development, more and more people are placing greater emphasis on exercise. Exercise can improve physical fitness, relax the mind, and boost immunity. However, strenuous exercise not only fails to provide positive benefits but also becomes a burden on the body, harming health. Therefore, accurately assessing the public's exercise status is of significant practical value.

[0003] Traditional methods can assess motor state, but they often require redundant auxiliary equipment and cannot maintain high accuracy. In fact, facial expressions and body movements can both serve as important references for judging fatigue levels. Therefore, fully integrating facial expressions and body movements with modern machine learning models to assess motor state is a very meaningful and urgent problem to be solved. Summary of the Invention

[0004] The purpose of this invention is to provide a method for accurately assessing motion state based on modern machine learning models. This method can accurately assess the fatigue state of an athlete by combining facial images and limb movement information with techniques such as target detection and super-resolution reconstruction, using a convolutional neural network model.

[0005] The embodiments of the present invention are implemented as follows:

[0006] In a first aspect, embodiments of this application provide a method for accurate motion state evaluation based on a modern machine learning model, which includes the following steps:

[0007] Acquire facial image samples under different fatigue states;

[0008] Face image samples under different fatigue states are input into a pre-set convolutional neural network model for training and learning to obtain a face fatigue state detection model.

[0009] Obtain video samples of limb movements under different fatigue states;

[0010] The limb movement video samples of different fatigue states are input into a pre-set convolutional neural network model for training and learning to obtain a limb movement fatigue detection model.

[0011] Obtain the video to be tested;

[0012] The target detection model is used at multiple scales to detect the video to be detected, and the first moving target region video is obtained.

[0013] Super-resolution reconstruction technology is used to reconstruct the video of the first moving target region to obtain the video of the second moving target region.

[0014] The video of the second motion target area is input into the limb movement fatigue detection model to obtain the limb fatigue detection results;

[0015] Acquire facial images from the video of the second moving target region and input them into the facial fatigue state detection model to obtain facial fatigue detection results;

[0016] The final fatigue test result is obtained by analyzing the results of limb fatigue detection and facial fatigue detection.

[0017] Based on the first aspect, in some embodiments of the present invention, the step of performing super-resolution reconstruction on the video of the first moving target region using super-resolution reconstruction technology to obtain the video of the second moving target region specifically includes:

[0018] Sequentially acquire each frame of the video of the first moving target region;

[0019] The first image vector is obtained by sampling features from each frame using a 5x5 convolution.

[0020] The first image vector is compressed using a 1x1 convolution to obtain the second image vector;

[0021] The second image vector is non-linearly mapped to the third image vector using a 3x3 convolution.

[0022] The third image vector is filtered and reconstructed using a 9x9 convolution to obtain new images for each frame.

[0023] Each new frame is synthesized to obtain the video of the second moving target region.

[0024] In some embodiments of the present invention, the steps of acquiring a face image in a video of a second moving target region and inputting it into a face fatigue state detection model to obtain a face fatigue detection result specifically include:

[0025] For each frame in the video of the second moving target region, capture optical flow images in two adjacent frames;

[0026] The captured optical flow image and its corresponding RGB video sequence are fed into a deep convolutional neural network model;

[0027] Deep convolutional neural network models automatically generate keyframe images of the second moving target region video;

[0028] The keyframe images are input into the face fatigue state detection model to obtain the face fatigue detection results.

[0029] In some embodiments of the present invention, the step of automatically generating keyframe images of the second moving target region video using the above-mentioned deep convolutional neural network model specifically includes:

[0030] The appearance and motion information of each frame is extracted by the input optical flow image and its corresponding RGB video sequence;

[0031] The appearance and motion features of each frame are combined to form a fused feature;

[0032] The fused features are reduced in dimensionality using the LDA algorithm to form low-dimensional features;

[0033] Generate labels for each frame using low-dimensional features;

[0034] After obtaining the label of each frame, the captured optical flow image and its corresponding RGB video sequence are used to train a deep convolutional neural network model to automatically generate the location prediction of key frames in the video of the second moving target region.

[0035] The trained deep convolutional neural network model automatically generates keyframe images of the second moving target region video.

[0036] In some embodiments of the present invention, after the step of performing super-resolution reconstruction of the moving target region using super-resolution reconstruction technology to obtain a second moving target region video, the following step is further included:

[0037] Image processing is performed on the video of the second moving target area.

[0038] In some embodiments of the present invention, the following step is further included after the above-described image processing step of the video of the second moving target region:

[0039] The first edge detection is performed on the video of the second moving target region after image processing.

[0040] In some embodiments of the present invention, after the above-described step of performing the first edge detection on the video of the second moving target region after image processing, the following step is further included:

[0041] A second edge detection is performed on the video of the second moving target region after the first edge detection.

[0042] In some embodiments of the present invention, the first edge detection adopts the Canny Edge Detection algorithm, and the second edge detection adopts the Local Binary Pattern algorithm.

[0043] Secondly, embodiments of this application provide an electronic device including a memory for storing one or more programs; and a processor. When the one or more programs are executed by the processor, the methods described in any of the first aspects above are implemented.

[0044] Thirdly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any of the first aspects above.

[0045] Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects:

[0046] First, facial image samples and limb movement video samples under different fatigue states are fed into corresponding convolutional neural network models for training, resulting in facial fatigue detection models and limb movement fatigue detection models. Then, motion videos of athletes are acquired, and the facial images and limb movement information identified in the videos are fed into the facial fatigue detection models and limb movement fatigue detection models, respectively. Finally, a comprehensive judgment is made based on the facial fatigue detection results and limb fatigue detection results, thus accurately and quickly identifying the athlete's fatigue state. This method is simple and fast, and can accurately assess the athlete's fatigue state based on facial images and limb movement information, combined with techniques such as object detection and super-resolution reconstruction, using convolutional neural network models. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 A detailed flowchart of a method for accurate motion state evaluation based on a modern machine learning model provided in this embodiment of the invention;

[0049] Figure 2 This is a flowchart illustrating the process of using super-resolution reconstruction technology to perform super-resolution reconstruction on a first moving target region video to obtain a second moving target region video in an embodiment of the present invention.

[0050] Figure 3 The flowchart below shows the process of obtaining a face image in a video of a second moving target region and inputting it into a face fatigue state detection model to obtain the face fatigue detection result in this embodiment of the invention.

[0051] Figure 4This is a flowchart illustrating the specific process of a deep convolutional neural network model automatically generating keyframe images of a second moving target region video in an embodiment of the present invention.

[0052] Figure 5 This is another specific flowchart of a method for accurate motion state evaluation based on a modern machine learning model, provided in an embodiment of the present invention.

[0053] Figure 6 This is a structural block diagram of an electronic device provided in an embodiment of the present invention.

[0054] Icons: 1. Memory; 2. Processor; 3. Communication interface. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0056] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0057] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0058] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the various embodiments and features described below can be combined with each other.

[0059] Example

[0060] Please see Figure 1-5 , Figure 1 The diagram shown is a detailed flowchart of a method for accurate motion state evaluation based on a modern machine learning model, provided by an embodiment of the present invention. Figure 2 The diagram shown is a flowchart illustrating the process of using super-resolution reconstruction technology to perform super-resolution reconstruction on a first moving target region video to obtain a second moving target region video in an embodiment of the present invention. Figure 3The diagram shows a flowchart illustrating the process of acquiring a face image from a video of a second moving target region, inputting it into a face fatigue detection model, and obtaining the face fatigue detection result in an embodiment of the present invention. Figure 4 The diagram shown is a flowchart illustrating the process of automatically generating keyframe images of the second moving target region video using a deep convolutional neural network model in an embodiment of the present invention. Figure 5 The diagram shows another specific flowchart of a motion state accurate evaluation method based on a modern machine learning model provided by an embodiment of the present invention. This motion state accurate evaluation method based on a modern machine learning model includes the following steps:

[0061] Step S1: Obtain facial image samples of people in different fatigue states.

[0062] Step S2: Input facial image samples of different fatigue states into a pre-set convolutional neural network model for training and learning to obtain a facial fatigue state detection model.

[0063] In steps S1-S2 above, we can usually tell a person's fatigue state by observing their facial expressions and movements. For example, if a person has a blank facial expression and frequently yawns or blinks, we can determine that they are fatigued. By acquiring facial image samples of different fatigue states, we label the fatigue state category information in the image samples and then feed them into a pre-set convolutional neural network model. The convolutional neural network can first extract features from the facial contours and facial features in the facial image samples, then train and classify them to obtain an accurate facial fatigue state detection model.

[0064] Step S3: Obtain video samples of limb movements under different fatigue states.

[0065] Step S4: Input the video samples of limb movements under different fatigue states into the pre-set convolutional neural network model for training and learning to obtain the limb movement fatigue detection model.

[0066] In steps S3-S4 above, we can usually judge a person's fatigue state by observing their body movements. For example, if a person's body movements are uncoordinated, they are unsteady on their feet, or they are breathing heavily, it can be determined that they are fatigued. By acquiring video samples of body movements in different fatigue states, we label the fatigue state category information of the body movements in the video samples, and then feed them into a pre-set convolutional neural network model. The convolutional neural network can first extract features from the body contours and body movements in the body movement video samples, and then perform training and classification to obtain an accurate body movement fatigue detection model.

[0067] In steps S1-S4 above, the principle is to train and learn classification rules using given categories or known training data, and then classify and predict unknown data. Specifically, the obtained facial image samples and body movement video samples of different fatigue states are first manually or otherwise labeled to indicate the fatigue state of the target person in the samples. These labels are then used to train and learn classification rules, resulting in corresponding facial fatigue state detection models and body movement fatigue detection models. After obtaining these models, the fatigue state of the person moving in the video to be detected can be classified and predicted. It is important to note that there is no specific order between steps S1 and S3; step S3 can precede step S1 or be performed simultaneously.

[0068] Step S5: Obtain the video to be tested.

[0069] In the above steps, the video to be tested can be obtained by using the camera devices already installed in the sports venue to track and film the exerciser, or by installing camera devices on the fitness equipment to record the user's exercise status video, or by uploading videos independently.

[0070] Step S6: Detect the video to be detected using a target detection model at multiple scales to obtain the video of the first moving target region.

[0071] In the above steps, the object detection model consists of a base network and a detection head. The former acts as a feature extractor, providing representations of each frame of the video to be detected at different sizes and levels of abstraction; the latter learns category and location associations based on these representations and supervised information. The two tasks of category prediction and location regression handled by the detection head are often performed in parallel, forming a multi-task loss for joint training. Finally, each frame containing the predicted bounding box is synthesized to obtain the first moving target region video.

[0072] Step S7: Use super-resolution reconstruction technology to perform super-resolution reconstruction on the video of the first moving target region to obtain the video of the second moving target region.

[0073] Specifically, please refer to Figure 2 , Figure 2 The diagram shows a flowchart illustrating the process of using super-resolution reconstruction technology to perform super-resolution reconstruction on a first moving target region video to obtain a second moving target region video in an embodiment of the present invention. The steps described above for using super-resolution reconstruction technology to perform super-resolution reconstruction on a first moving target region video to obtain a second moving target region video specifically include:

[0074] Step S7-1: Sequentially acquire each frame of the video of the first moving target region;

[0075] Step S7-2: Use a 5x5 convolution to sample features from each frame of the image to obtain the first image vector;

[0076] Step S7-3: Compress the first image vector using a 1x1 convolution to obtain the second image vector;

[0077] Step S7-4: Use a 3x3 convolution to non-linearly map the second image vector into a third image vector;

[0078] Step S7-5: Use a 9x9 convolution to filter and reconstruct the third image vector to obtain new images for each frame;

[0079] Step S7-6: Synthesize each new frame image to obtain the second moving target region video.

[0080] In the above steps, after sequentially acquiring each frame of the video of the first moving target region, feature extraction, compression, mapping, expansion, and deconvolution are performed on each frame sequentially to obtain new frames with higher resolution. Finally, these new frames are synthesized to obtain a high-resolution video of the second moving target region, which can provide higher-quality video information for subsequent detection.

[0081] Step S11: Input the video of the second motion target area into the limb movement fatigue detection model to obtain the limb fatigue detection result.

[0082] Step S12: Obtain the face image in the video of the second moving target area and input it into the face fatigue state detection model to obtain the face fatigue detection result.

[0083] Specifically, please refer to Figure 3 , Figure 3 The diagram illustrates the specific flowchart of an embodiment of the present invention for acquiring a face image from a video of a second moving target region, inputting it into a face fatigue state detection model, and obtaining a face fatigue detection result. The steps for acquiring a face image from a video of a second moving target region, inputting it into a face fatigue state detection model, and obtaining a face fatigue detection result specifically include:

[0084] Step S12-1: Capture optical flow images in two adjacent frames for each frame in the video of the second moving target region;

[0085] In the above steps, capturing optical flow images in two adjacent frames for each frame in the video of the second moving target region can provide detailed information for subsequent processing and ensure that information in each frame is not missed, thereby improving the accuracy of processing.

[0086] Step S12-2: Feed the captured optical flow image and its corresponding RGB video sequence to the deep convolutional neural network model;

[0087] In the above steps, the learning of the deep convolutional neural network model can effectively process the input optical flow image and its corresponding RGB video sequence, improving processing efficiency and accuracy.

[0088] Step S12-3: The deep convolutional neural network model automatically generates keyframe images of the second moving target region video;

[0089] In the above steps, the keyframe images of the second moving target area video are automatically generated using a deep convolutional neural network model. Users do not need to care about the specific processing steps. The keyframe images can be automatically extracted with high efficiency and high accuracy, providing a basis for subsequent processing.

[0090] Step S12-4: Input the keyframe image into the face fatigue state detection model to obtain the face fatigue detection result.

[0091] Specifically, please refer to Figure 4 , Figure 4 The diagram shows a flowchart illustrating the process of a deep convolutional neural network model automatically generating keyframe images of a second moving target region video in an embodiment of the invention. The steps involved in this process include:

[0092] Step S12-3-1: Extract the appearance and motion information of each frame from the input optical flow image and its corresponding RGB video sequence;

[0093] In the above steps, since the subsequent detection focuses on event information such as the activities and behaviors of people in the second moving target area video, by extracting the optical flow image of each frame of the second moving target area video, and then recognizing and processing it, the human actions and frames in the second moving target area video can be well identified.

[0094] Step S12-3-2: Fuse the appearance and motion features of each frame to form a fused feature;

[0095] In the above steps, by fusing the appearance and motion features of each frame to form fused features, the representational power of each frame is enhanced, and the features in each frame can be identified better and more accurately.

[0096] Step S12-3-3: Use the LDA algorithm to reduce the dimensionality of the fused features to form low-dimensional features;

[0097] In the above steps, the fused features are reduced in dimensionality using the LDA algorithm, shrinking them to a low-dimensional space to form a low-dimensional feature representation. The LDA algorithm is used here because it can compress complex fused features into a low-dimensional space, effectively reducing computational load. Furthermore, the choice of LDA algorithm ensures optimal separability of the low-dimensional features. Applying the LDA algorithm for classification and dimensionality reduction in complex videos of second moving target regions not only effectively classifies the data but also reduces its dimensionality while maintaining accuracy.

[0098] Step S12-3-4: Generate labels for each frame using low-dimensional features;

[0099] In the above steps, using low-dimensional features to generate labels for each frame can greatly reduce the computation and storage memory of labels, thus reducing the system burden.

[0100] Step S12-3-5: After obtaining the label of each frame, the captured optical flow image and its corresponding RGB video sequence are used to train a deep convolutional neural network model to automatically generate the position prediction of key frames in the video of the second moving target region.

[0101] In the above steps, after obtaining the label for each frame, a deep convolutional neural network model is trained using the captured optical flow image and its corresponding RGB video sequence to automatically generate keyframes in the video of the second moving target region for location prediction. The trained deep convolutional neural network model will be able to automatically generate keyframes in the video of the second moving target region. This achieves automatic and accurate generation of keyframes in the video of the second moving target region, improving detection efficiency and accuracy.

[0102] Step S12-3-6: The trained deep convolutional neural network model automatically generates keyframe images of the second moving target region video.

[0103] Step S13: Analyze the limb fatigue detection results and the face fatigue detection results to obtain the final fatigue detection results.

[0104] In the above steps, if either the limb fatigue detection result or the facial fatigue detection result indicates that the exerciser is in a state of fatigue, then the exerciser can be determined to be in a state of fatigued exercise, and a notification will be issued to remind the exerciser to rest. If both the limb fatigue detection result and the facial fatigue detection result indicate that the exerciser is not in a state of fatigue, then the exerciser will be notified that they can continue with appropriate exercise.

[0105] Please see Figure 5 , Figure 5The diagram shows another specific flowchart of a method for accurate motion state evaluation based on a modern machine learning model provided by an embodiment of the invention. Following the step of using super-resolution reconstruction technology to perform super-resolution reconstruction of the moving target region to obtain a second moving target region video, the method further includes the following steps:

[0106] Image processing is performed on the video of the second moving target area.

[0107] In the above steps, image processing of the video of the second moving target region can provide higher-quality video information for subsequent detection and analysis of the moving target. For example, image processing methods for the video of the second moving target region include one or more of color transformation, noise reduction, or image quality adjustment. Typically, images may be affected by imaging equipment and the external environment during digitization and transmission, resulting in potentially poor image quality. Depending on actual needs, image processing can be performed, such as color transformation, noise reduction, or image quality adjustment. This processing will improve the image quality, enabling better analysis and judgment, and improving processing efficiency and accuracy.

[0108] See Figure 5 , Figure 5 The diagram shows another specific flowchart of a method for accurate motion state assessment based on a modern machine learning model, provided by an embodiment of the invention. Following the above steps of image processing of the video of the second moving target region, the method further includes the following steps:

[0109] The first edge detection is performed on the video of the second moving target region after image processing.

[0110] In the above steps, since the athlete's position in the second motion target area video is not fixed, performing a first edge detection on the second motion target area video can distinguish the athlete from other video areas. Once the athlete's location is identified, subsequent analysis of the athlete's information only requires calculation of that area, greatly reducing unnecessary computation. This first edge detection can employ the Canny Edge Detection algorithm. Using the Canny Edge Detection algorithm effectively distinguishes the athlete from other video areas without compromising the edge information of the athlete's location.

[0111] See Figure 5 , Figure 5 The diagram shows another specific flowchart of a method for accurate motion state assessment based on a modern machine learning model, provided by an embodiment of the invention. Following the step of performing the first edge detection on the processed second moving target region video, the method further includes the following steps:

[0112] A second edge detection is performed on the video of the second moving target region after the first edge detection.

[0113] In the above steps, a second edge detection is performed on the video of the second moving target region after the first edge detection. This allows for better calculation of the outline information included in the athlete's body movements and facial images, improving the accuracy of the calculation. This second edge detection can employ the Local Binary Pattern algorithm. Using the Local Binary Pattern algorithm for secondary edge detection avoids the influence of lighting changes on the outline information in the athlete's body movements and facial images, thus enabling more accurate analysis of the athlete's body movements and facial images.

[0114] Please see Figure 6 , Figure 6 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. The electronic device includes a memory 1, a processor 2, and a communication interface 3. The memory 1, processor 2, and communication interface 3 are electrically connected directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The memory 1 can be used to store software programs and modules, such as the program instructions / modules corresponding to the motion state accurate evaluation method based on a modern machine learning model provided in this application embodiment. The processor 2 executes the software programs and modules stored in the memory 1 to perform various functional applications and data processing. The communication interface 3 can be used for signaling or data communication with other node devices.

[0115] The memory 1 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0116] Processor 2 can be an integrated circuit chip with signal processing capabilities. Processor 2 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0117] Understandable. Figure 6 The structure shown is for illustrative purposes only; the electronic device may also include components that are more advanced than those shown. Figure 6 The more or fewer components shown, or having the same Figure 6 The different configurations shown. Figure 6 The components shown can be implemented using hardware, software, or a combination thereof.

[0118] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0119] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0120] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0121] In summary, the present application provides a method for accurate motion state assessment based on modern machine learning models. First, facial image samples and limb movement video samples under different fatigue states are fed into corresponding convolutional neural network models for training, resulting in a facial fatigue state detection model and a limb movement fatigue detection model. Then, motion videos of the athlete are acquired, and the identified facial images and limb movement information from the videos are fed into the facial fatigue state detection model and the limb movement fatigue detection model, respectively. Finally, a comprehensive judgment is made based on the facial fatigue detection results and the limb fatigue detection results, thus enabling accurate and rapid identification of the athlete's fatigue state.

[0122] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0123] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within this application. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for precise evaluation of the state of motion based on modern machine learning models, characterized in that, The method comprises the following steps: obtaining face image samples of different fatigue states; inputting the face image samples of different fatigue states into a preset convolutional neural network model for training and learning to obtain a face fatigue state detection model; obtaining limb action video samples of different fatigue states; inputting the limb action video samples of different fatigue states into the preset convolutional neural network model for training and learning to obtain a limb action fatigue detection model; obtaining a video to be detected; detecting the video to be detected by using a target detection model under multiple scales to obtain a first moving target region video; performing super-resolution reconstruction on the first moving target region video by using a super-resolution reconstruction technology to obtain a second moving target region video; inputting the second moving target region video into the limb action fatigue detection model to obtain a limb fatigue detection result; obtaining a face image in the second moving target region video and inputting the face image into the face fatigue state detection model to obtain a face fatigue detection result; analyzing the limb fatigue detection result and the face fatigue detection result to obtain a final fatigue detection result; wherein: the step of obtaining the face image in the second moving target region video and inputting the face image into the face fatigue state detection model to obtain the face fatigue detection result specifically comprises: capturing a flow image in two adjacent frames for each frame in the second moving target region video; sending the captured flow image and a corresponding RGB video sequence to a deep convolutional neural network model; the deep convolutional neural network model automatically generates a key frame image of the second moving target region video; inputting the key frame image into the face fatigue state detection model to obtain the face fatigue detection result; the step of the deep convolutional neural network model automatically generating the key frame image of the second moving target region video specifically comprises: extracting appearance and motion information of each frame through the sent flow image and the corresponding RGB video sequence; fusing the appearance and motion features of each frame to form a fused feature; reducing the fused feature by using an LDA algorithm to form a low-dimensional feature; generating a label of each frame by using the low-dimensional feature; after obtaining the label of each frame, the deep convolutional neural network model is trained by using the captured flow image and the corresponding RGB video sequence to automatically generate a position prediction of the key frame in the second moving target region video; the trained deep convolutional neural network model automatically generates the key frame image of the second moving target region video.

2. The method of claim 1, wherein the method is based on a modern machine learning model. the step of performing super-resolution reconstruction on the first moving target region video by using the super-resolution reconstruction technology to obtain the second moving target region video specifically comprises: obtaining each frame image of the first moving target region video in sequence; performing feature sampling on each frame image by using a 5x5 convolution to obtain a first image vector; compressing the first image vector by using a 1x1 convolution to obtain a second image vector; nonlinearly mapping the second image vector to a third image vector by using a 3x3 convolution; filtering and reconstructing the third image vector by using a 9x9 convolution to obtain a new each frame image; synthesizing the new each frame image to obtain the second moving target region video.

3. The method of claim 1, wherein the method is characterized by, The step of performing super-resolution reconstruction on the moving target region using a super-resolution reconstruction technique to obtain a second moving target region video further comprises the following steps: performing image processing on the second moving target region video.

4. The method of claim 3, wherein the method is characterized by, The step of performing image processing on the second moving target region video further comprises the following steps: performing first edge detection on the second moving target region video after image processing.

5. The method of claim 4, wherein the method is characterized by, The step of performing first edge detection on the second moving target region video after image processing further comprises the following steps: performing second edge detection on the second moving target region video after the first edge detection.

6. The method of claim 5, wherein the method is based on a modern machine learning model. The first edge detection uses a Canny Edge Detection algorithm, and the second edge detection uses a Local Binary Pattern algorithm.

7. An electronic device, comprising: comprising: a memory for storing one or more programs; a processor; when the one or more programs are executed by the processor, the method as claimed in any one of claims 1-6 is implemented.

8. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method as claimed in any one of claims 1-6.

Images