A Motion State Evaluation Method and System Based on a Low-Consumption Machine Learning Model

Abstract

The present invention proposes a method and system for evaluating a motion state based on a low-power machine learning model, which relates to the technical field of motion state recognition. The method includes: using object detection technology to detect a motion video to be detected, obtaining a face image to be discriminated and a motion video to be discriminated. Inputting the face image to be discriminated into a low-power face fatigue detection model. If it is at the edge of the separation plane, calculate the first similarity and the second similarity to achieve the effect of secondary detection and improve the accuracy of evaluating the motion state using the face image to be discriminated. If the face image to be discriminated is a non-fatigued face image, the motion state of the user is more accurately evaluated by combining limb movements. Using a low-power limb fatigue detection model to detect the motion video to be discriminated reduces the computational consumption. If it is at the edge of the separation plane, calculate the third similarity and the fourth similarity to achieve the purpose of secondary detection of the motion video to be discriminated.

Description

Technical Field

[0001] The present invention relates to the technical field of motion state recognition. Specifically, it relates to a motion state evaluation method and system based on a low-consumption machine learning model. Background Art

[0002] With the progress of society, people pay more and more attention to physical health. As an effective way, exercise can significantly improve people's health level. Appropriate exercise can play a positive role, but excessive exercise often becomes a burden on the body and instead has a negative impact on health. Therefore, how to evaluate the motion state in real time has good practical application value.

[0003] In response to the above problems, although some classical methods can evaluate the motion state, they require huge computing resources. Therefore, how to effectively reduce the computing consumption and evaluate the motion state has very important application value. Summary of the Invention

[0004] The purpose of the present invention is to provide a motion state evaluation method and system based on a low-consumption machine learning model, which can evaluate the motion state with low consumption and high accuracy.

[0005] The embodiments of the present invention are implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a motion state evaluation method based on a low-consumption machine learning model, which includes the following steps:

[0007] Obtain the motion video to be detected;

[0008] Use the target detection technology to detect the motion video to be detected, so as to obtain the face image to be judged and the motion video of the target user;

[0009] Obtain a first preset number of non-fatigued face images, and select a standard non-fatigued face image from all non-fatigued face images;

[0010] Obtain a first preset number of fatigued face images, and select a standard fatigued face image from all fatigued face images;

[0011] Input the face image to be judged into the low-consumption face fatigue detection model for detection to obtain a first detection score;

[0012] If the first detection score is higher than the first preset score, it is determined that the face image to be judged is a non-fatigued face image; if the first detection score is lower than the second preset score, it is determined that the face image to be judged is a fatigued face image; if the first detection score is higher than the second preset score and lower than the first preset score, proceed to the next step;

[0013] Calculate the first similarity between the face image to be judged and the standard non-fatigued face image, and the second similarity between the face image to be judged and the standard fatigued face image;

[0014] If the first similarity is higher than the second similarity, it is determined that the face image to be judged is a non-fatigued face image; if the first similarity is lower than the second similarity, it is determined that the face image to be judged is a fatigued face image;

[0015] If the face image to be judged is a fatigued face image, it is determined that the target user is in a fatigued state; if the face image to be judged is a non-fatigued face image, proceed to the next step;

[0016] Obtain the second preset number of non-fatigued limb movement videos as positive samples, obtain the second preset number of fatigued limb movement videos as negative samples, and use the SVM model to train and learn the positive and negative samples to obtain a low-power limb fatigue detection model;

[0017] Select the standard non-fatigued limb movement videos from all the positive samples, and select the standard fatigued limb movement videos from all the negative samples;

[0018] Use the low-power limb fatigue detection model to detect the movement video to be judged and obtain a second detection score;

[0019] If the second detection score is higher than the third preset score, it is determined that the movement video to be judged is a non-fatigued limb movement video; if the second detection score is lower than the fourth preset score, it is determined that the movement video to be judged is a fatigued limb movement video; if the second detection score is lower than the third preset score and higher than the fourth preset score, proceed to the next step;

[0020] Calculate the third similarity between the movement video to be judged and the standard non-fatigued limb movement video, and the fourth similarity between the movement video to be judged and the standard fatigued limb movement video;

[0021] If the third similarity is higher than the fourth similarity, it is determined that the movement video to be judged is a non-fatigued limb movement video; if the third similarity is lower than the fourth similarity, it is determined that the movement video to be judged is a fatigued limb movement video;

[0022] If the movement video to be judged is a fatigued limb movement video, it is determined that the target user is in a fatigued state; if the movement video to be judged is a non-fatigued limb movement video, it is determined that the target user is in a non-fatigued state.

[0023] In some embodiments of the present invention, before the step of inputting the face image to be judged into the low-power face fatigue detection model for detection, the method further includes:

[0024] Use the first preset number of non-fatigued face images as positive training samples, and use the first preset number of fatigued face images as negative training samples;

[0025] Use the SVM model to train and learn the positive training samples and negative training samples to obtain a low-power face fatigue detection model.

[0026] In some embodiments of the present invention, the step of selecting standard non-fatigued face images from all non-fatigued face images includes:

[0027] Compare the confidence levels of all non-fatigued face images;

[0028] Use the non-fatigued face image with the highest confidence level as the standard non-fatigued face image.

[0029] In some embodiments of the present invention, the step of selecting standard fatigued face images from all fatigued face images includes:

[0030] Compare the confidence levels of all fatigued face images;

[0031] Use the fatigued face image with the highest confidence level as the standard fatigued face image.

[0032] In some embodiments of the present invention, the step of selecting standard non-fatigued limb movement videos from all positive samples includes:

[0033] Compare the confidence levels of all positive samples, and use the positive sample with the highest confidence level as the standard non-fatigued limb movement video.

[0034] In some embodiments of the present invention, the step of selecting standard fatigued limb movement videos from all negative samples includes:

[0035] Compare the confidence levels of all negative samples, and use the negative sample with the highest confidence level as the standard fatigued limb movement video.

[0036] In some embodiments of the present invention, after the step of determining that the target user is in a fatigued state if the motion video to be discriminated is a fatigued limb movement video, the method further includes:

[0037] Send a warning message to remind the target user to rest.

[0038] In a second aspect, an embodiment of the present application provides a motion state evaluation system based on a low-power machine learning model, which includes:

[0039] A motion video to be detected acquisition module, configured to acquire a motion video to be detected;

[0040] A target detection module, which is used to detect the motion video to be detected by using target detection technology, so as to obtain the face image to be judged and the motion video to be judged of the target user;

[0041] A standard non-fatigued face image determination module, which is used to obtain the first preset number of non-fatigued face images and select the standard non-fatigued face image from all non-fatigued face images;

[0042] A standard fatigued face image determination module, which is used to obtain the first preset number of fatigued face images and select the standard fatigued face image from all fatigued face images;

[0043] A first detection module, which is used to input the face image to be judged into a low-power face fatigue detection model for detection to obtain a first detection score;

[0044] An image score determination module, which is used to determine that the face image to be judged is a non-fatigued face image if the first detection score is higher than the first preset score, determine that the face image to be judged is a fatigued face image if the first detection score is lower than the second preset score, and proceed to the next step if the first detection score is higher than the second preset score and lower than the first preset score;

[0045] An image similarity calculation module, which is used to calculate the first similarity between the face image to be judged and the standard non-fatigued face image, and the second similarity between the face image to be judged and the standard fatigued face image;

[0046] An image similarity comparison module, which is used to determine that the face image to be judged is a non-fatigued face image if the first similarity is higher than the second similarity, and determine that the face image to be judged is a fatigued face image if the first similarity is lower than the second similarity;

[0047] An image fatigue state determination module, which is used to determine that the target user is in a fatigued state if the face image to be judged is a fatigued face image, and proceed to the next step if the face image to be judged is a non-fatigued face image;

[0048] A low-power limb fatigue detection model training module, which is used to obtain the second preset number of non-fatigued limb action videos as positive samples, obtain the second preset number of fatigued limb action videos as negative samples, and use the SVM model to train and learn the positive samples and negative samples to obtain a low-power limb fatigue detection model;

[0049] A standard fatigued limb action video determination module, which is used to select the standard non-fatigued limb action video from all positive samples and select the standard fatigued limb action video from all negative samples;

[0050] A second detection module, configured to detect the motion video to be discriminated by using a low-power limb fatigue detection model, and obtain a second detection score;

[0051] A video score determination module, configured to, if the second detection score is higher than a third preset score, determine that the motion video to be discriminated is a non-fatigue limb action video; if the second detection score is lower than a fourth preset score, determine that the motion video to be discriminated is a fatigue limb action video; if the second detection score is lower than the third preset score and higher than the fourth preset score, proceed to the next step;

[0052] A video similarity calculation module, configured to calculate a third similarity between the motion video to be discriminated and a standard non-fatigue limb action video, and a fourth similarity between the motion video to be discriminated and a standard fatigue limb action video;

[0053] A video similarity comparison module, configured to, if the third similarity is higher than the fourth similarity, determine that the motion video to be discriminated is a non-fatigue limb action video; if the third similarity is lower than the fourth similarity, determine that the motion video to be discriminated is a fatigue limb action video;

[0054] A video fatigue state determination module, configured to, if the motion video to be discriminated is a fatigue limb action video, determine that the target user is in a fatigue state; if the motion video to be discriminated is a non-fatigue limb action video, determine that the target user is in a non-fatigue state.

[0055] In a third aspect, an embodiment of the present application provides an electronic device, which includes a memory for storing one or more programs; and a processor. When the one or more programs are executed by the processor, the method according to any one of the above first aspects is implemented.

[0056] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method according to any one of the above first aspects is implemented.

[0057] Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects:

[0058] The present invention provides a method and system for evaluating a motion state based on a low-power machine learning model, which includes the following steps: Obtain a motion video to be detected. Use object detection technology to detect the motion video to be detected, so as to accurately locate the motion target, thereby obtaining the face image to be judged and the motion video to be judged of the target user. Obtain the first preset number of non-fatigued face images, and select the standard non-fatigued face image from all non-fatigued face images. Obtain the first preset number of fatigued face images, and select the standard fatigued face image from all fatigued face images. A sufficient number of non-fatigued face images and fatigued face images are more conducive to selecting the standard non-fatigued face image and the standard fatigued face image. Input the face image to be judged into a low-power face fatigue detection model for detection to obtain a first detection score. If the first detection score is higher than the first preset score, it is determined that the face image to be judged is a non-fatigued face image. If the first detection score is lower than the second preset score, it is determined that the face image to be judged is a fatigued face image. If the first detection score is higher than the second preset score and lower than the first preset score, proceed to the next step. Calculate the first similarity between the face image to be judged and the standard non-fatigued face image, and the second similarity between the face image to be judged and the standard fatigued face image. If the first similarity is higher than the second similarity, it is determined that the face image to be judged is a non-fatigued face image. If the first similarity is lower than the second similarity, it is determined that the face image to be judged is a fatigued face image. Thus, the effect of secondary detection is achieved for the face image to be judged whose first detection score is higher than the second preset score and lower than the first preset score, further improving the accuracy of evaluating the motion state using the face image to be judged. If the face image to be judged is a fatigued face image, it is determined that the target user is in a fatigued state. If the face image to be judged is a non-fatigued face image, the motion video to be judged is subjected to limb movement discrimination to more accurately evaluate the motion state of the user in combination with the limb movement. When detecting the motion video to be judged, obtain the second preset number of non-fatigued limb movement videos as positive samples, obtain the second preset number of fatigued limb movement videos as negative samples, and use the SVM model to train and learn the positive samples and negative samples to obtain a low-power limb fatigue detection model. Since the SVM model is a low-power training and learning model, using the SVM model to train and establish a low-power limb fatigue detection model can greatly reduce the computational consumption when using the low-power limb fatigue detection model. Select the standard non-fatigued limb movement video from all positive samples, and select the standard fatigued limb movement video from all negative samples. Use the low-power limb fatigue detection model to detect the motion video to be judged to obtain a second detection score.If the second detection score is higher than the third preset score, it is determined that the motion video to be discriminated is a non-fatigue limb movement video. If the second detection score is lower than the fourth preset score, it is determined that the motion video to be discriminated is a fatigue limb movement video. If the second detection score is lower than the third preset score and higher than the fourth preset score, the next step is carried out. Calculate the third similarity between the motion video to be discriminated and the standard non-fatigue limb movement video, and the fourth similarity between the motion video to be discriminated and the standard fatigue limb movement video. If the third similarity is higher than the fourth similarity, it is determined that the motion video to be discriminated is a non-fatigue limb movement video. If the third similarity is lower than the fourth similarity, it is determined that the motion video to be discriminated is a fatigue limb movement video. Thus, the purpose of secondary detection of the motion video to be discriminated with the second detection score lower than the third preset score and higher than the fourth preset score is realized, and the accuracy of motion state evaluation using the motion video to be discriminated is further improved. If the motion video to be discriminated is a fatigue limb movement video, it is determined that the target user is in a fatigue state. If the motion video to be discriminated is a non-fatigue limb movement video, it is determined that the target user is in a non-fatigue state. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0060] Figure 1 It is a flowchart of a motion state evaluation method based on a low-power machine learning model provided by an embodiment of the present invention;

[0061] Figure 2 It is a flowchart of establishing a low-power face fatigue detection model provided by an embodiment of the present invention;

[0062] Figure 3 It is a structural block diagram of a motion state evaluation system based on a low-power machine learning model provided by an embodiment of the present invention;

[0063] Figure 4 It is a structural block diagram of a standard non-fatigue face image determination module provided by an embodiment of the present invention;

[0064] Figure 5 It is a schematic structural block diagram of an electronic device provided by an embodiment of the present invention.

[0065] Icons: 110 - Motion video acquisition module to be detected; 120 - Target detection module; 130 - Standard non-fatigued face image determination module; 131 - Non-fatigued face image confidence comparison unit; 132 - Standard non-fatigued face image selection unit; 140 - Standard fatigued face image determination module; 150 - First detection module; 160 - Image score determination module; 170 - Image similarity calculation module; 180 - Image similarity comparison module; 190 - Image fatigue state determination module; 200 - Low-power limb fatigue detection model training module; 210 - Standard fatigued limb action video determination module; 220 - Second detection module; 230 - Video score determination module; 240 - Video similarity calculation module; 250 - Video similarity comparison module; 260 - Video fatigue state determination module; 101 - Memory; 102 - Processor; 103 - Communication interface. Detailed implementation manners

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are some, but not all, of the embodiments of the present application. The components of the embodiments of the present application described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations.

[0067] Therefore, the detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but merely represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts fall within the scope of protection of the present application.

[0068] It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present application, if terms such as "first", "second", etc. appear, they are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0069] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, when the terms "include", "comprise" or any other variant thereof are used, they are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or elements inherent to such process, method, article or device. Without further limitation, if an element is defined by the statement "including one...", it does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

[0070] In the description of the present application, it should be noted that if relational terms such as "upper", "lower", "inner", "outer", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of this application is customarily placed during use, it is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present application.

[0071] In the description of the present application, it should also be noted that unless otherwise clearly specified and limited, if relational terms such as "set" and "connect" are used, they should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific circumstances.

[0072] The following will describe in detail some embodiments of the present application with reference to the drawings. Without conflict, the following embodiments and the features in each embodiment can be combined with each other.

[0073] Embodiment

[0074] Please refer to Figure 1 , Figure 1 The flowchart of a motion state evaluation method based on a low-power machine learning model provided by an embodiment of the present invention is shown. An embodiment of the present application provides a motion state evaluation method based on a low-power machine learning model, which includes the following steps:

[0075] S110: Obtain the motion video to be detected;

[0076] Specifically, the movement video of the target user can be recorded in real time through a mobile phone or a video recorder to obtain the movement video to be detected.

[0077] S120: Use object detection technology to detect the movement video to be detected, so as to obtain the face image to be judged and the movement video of the target user to be judged;

[0078] Specifically, use object detection technology to detect the moving objects in the movement video to be detected, so as to accurately locate the moving objects, and thus obtain the face image to be judged and the movement video of the target user to be judged. Among them, the face image to be judged is obtained by extracting the face image from the movement video to be detected for the located moving object.

[0079] Exemplarily, the above object detection technology can be to use a VJ detector to detect each frame of the movement video to be detected to detect moving objects.

[0080] S130: Obtain the first preset number of non-fatigued face images, and select the standard non-fatigued face image from all non-fatigued face images;

[0081] S140: Obtain the first preset number of fatigued face images, and select the standard fatigued face image from all fatigued face images;

[0082] Among them, the first preset number can be 1000. A sufficient number of non-fatigued face images and fatigued face images are more conducive to selecting the standard non-fatigued face image and the standard fatigued face image.

[0083] Specifically, the confidence levels of all non-fatigued face images can be compared, and the non-fatigued face image with a higher confidence level can be selected as the standard non-fatigued face image. And the confidence levels of all fatigued face images can be compared, and the fatigued face image with a higher confidence level can be selected as the standard fatigued face image. The higher the confidence level of the fatigued face image and the non-fatigued face image, the higher its accuracy, that is, the fatigued face image and the non-fatigued face image with a high confidence level are equivalent to selecting very certain fatigued or non-fatigued images.

[0084] S150: Input the face image to be judged into a low-power face fatigue detection model for detection to obtain the first detection score;

[0085] Specifically, non-fatigued face images can be used as positive training samples, and fatigued face images can be used as negative training samples. Then, an SVM model is used to train and learn the positive and negative training samples to obtain a low-power face fatigue detection model. Since the SVM model is a low-power training and learning model, using the low-power face fatigue detection model established by the SVM model to detect and judge the face image to be judged can greatly reduce the consumption of computing resources.

[0086] S160: If the first detection score is higher than the first preset score, it is determined that the face image to be judged is a non-fatigued face image. If the first detection score is lower than the second preset score, it is determined that the face image to be judged is a fatigued face image. If the first detection score is higher than the second preset score and lower than the first preset score, proceed to the next step;

[0087] Specifically, the basic principle of the SVM model is to use a separating "plane" to divide the sample space into two to complete binary classification. If the first detection score is higher than the first preset score, that is, the face image to be judged is above the separating "plane", it is determined that the face image to be judged is a non-fatigued face image. If the first detection score is lower than the second preset score, that is, the face image to be judged is below the separating "plane", it is determined that the face image to be judged is a fatigued face image. If the first detection score is higher than the second preset score and lower than the first preset score, that is, the face image to be judged is on the edge of the separating "plane", secondary detection is performed to further improve the accuracy of evaluating the motion state using the face image to be judged.

[0088] Exemplarily, the above first preset score can be 81, and the second preset score can be 79. If the first detection score is 82, it is determined that the face image to be judged is a non-fatigued face image. If the first detection score is 78, it is determined that the face image to be judged is a fatigued face image. If the first detection score is 80, secondary detection is performed.

[0089] S170: Calculate the first similarity between the face image to be judged and the standard non-fatigued face image, and the second similarity between the face image to be judged and the standard fatigued face image;

[0090] S180: If the first similarity is higher than the second similarity, it is determined that the face image to be judged is a non-fatigued face image. If the first similarity is lower than the second similarity, it is determined that the face image to be judged is a fatigued face image;

[0091] Specifically, if the similarity between the face image to be judged and the standard non-fatigued face image is high, it is determined to be a non-fatigued face image. If the similarity between the face image to be judged and the standard fatigued face image is high, it is determined to be a fatigued face image.

[0092] S190: If the face image to be judged is a fatigued face image, it is determined that the target user is in a fatigued state. If the face image to be judged is a non-fatigued face image, proceed to the next step;

[0093] Specifically, if it is determined through the face image to be judged that the target user is in a fatigued state, a warning is directly issued to let the target user, i.e., the athlete, take a rest, and there is no need to proceed to the next step. If it is determined through the face image to be judged that the target user is not in a fatigued state, proceed to the next step to perform limb movement discrimination on the movement video to be judged. Thus, the movement state of the user can be more accurately evaluated in combination with limb movements.

[0094] S200: Obtain the second preset number of non-fatigued limb movement videos as positive samples, obtain the second preset number of fatigued limb movement videos as negative samples, and use the SVM model to train and learn the positive samples and negative samples to obtain a low-power limb fatigue detection model;

[0095] Specifically, since the SVM model is a low-consumption training and learning model, using the SVM model to train and establish a low-power limb fatigue detection model can greatly reduce the computational consumption when using the low-power limb fatigue detection model.

[0096] Among them, the second preset number can be 1000. Using the SVM model to train and learn a sufficient amount of positive samples and negative samples can make the recognition result of the low-power limb fatigue detection model more accurate.

[0097] S210: Select the standard non-fatigued limb movement videos from all the positive samples, and select the standard fatigued limb movement videos from all the negative samples;

[0098] Specifically, a sufficient amount of positive samples and negative samples are more conducive to selecting the standard non-fatigued limb movement videos and the standard fatigued limb movement videos.

[0099] S220: Use the low-power limb fatigue detection model to detect the movement video to be judged to obtain a second detection score;

[0100] S230: If the second detection score is higher than the third preset score, it is determined that the movement video to be judged is a non-fatigued limb movement video. If the second detection score is lower than the fourth preset score, it is determined that the movement video to be judged is a fatigued limb movement video. If the second detection score is lower than the third preset score and higher than the fourth preset score, proceed to the next step;

[0101] Specifically, for the motion video to be discriminated, a low-power limb fatigue detection model is used for detection. If the second detection score is higher than the third preset score, that is, the motion video to be discriminated is above the separation "plane", it is determined that the motion video to be discriminated is a non-fatigue limb action video. If the second detection score is lower than the fourth preset score, that is, the motion video to be discriminated is below the separation "plane", it is determined that the motion video to be discriminated is a fatigue limb action video. If the second detection score is lower than the third preset score and higher than the fourth preset score, that is, the motion video to be discriminated is at the edge of the separation "plane", secondary detection is performed to further improve the accuracy of using the motion video to be discriminated for motion state evaluation.

[0102] S240: Calculate the third similarity between the motion video to be discriminated and the standard non-fatigue limb action video, and the fourth similarity between the motion video to be discriminated and the standard fatigue limb action video;

[0103] S250: If the third similarity is higher than the fourth similarity, it is determined that the motion video to be discriminated is a non-fatigue limb action video; if the third similarity is lower than the fourth similarity, it is determined that the motion video to be discriminated is a fatigue limb action video;

[0104] Specifically, if the similarity between the motion video to be discriminated and the standard non-fatigue limb action video is high, it is determined to be a non-fatigue limb action video. If the similarity between the motion video to be discriminated and the standard fatigue limb action video is high, it is determined to be a fatigue limb action video.

[0105] S260: If the motion video to be discriminated is a fatigue limb action video, it is determined that the target user is in a fatigue state; if the motion video to be discriminated is a non-fatigue limb action video, it is determined that the target user is in a non-fatigue state.

[0106] Exemplarily, if it is determined that the target user is in a fatigue state, a warning is issued to remind the target user to take a rest. If it is determined that the target user is in a non-fatigue state, the target user can continue to exercise.

[0107] In the above implementation process, the method first uses object detection technology to detect moving objects in the moving video to be detected, so as to accurately locate the moving objects, and thus obtain the face image to be judged and the moving video to be judged of the target user. Then, standard non-fatigued face images are selected from a sufficient number of non-fatigued face images, and standard fatigued face images are selected from a sufficient number of fatigued face images. Then, a low-consumption face fatigue detection model is used to detect and judge the face image to be judged, so as to greatly reduce the consumption of computing resources. Then, for the face image to be judged whose first detection score is higher than the second preset score and lower than the first preset score, a secondary detection is performed, that is, the first similarity and the second similarity are compared, and the secondary detection of the face image to be judged is completed, so as to further improve the accuracy of using the face image to be judged to evaluate the motion state. And if it is determined through the face image to be judged that the target user is not in a fatigued state, then the limb movements of the moving video to be judged are discriminated, so as to more accurately evaluate the motion state of the user in combination with the limb movements. When detecting the moving video to be judged, first, an SVM model is used to train a sufficient number of positive samples and negative samples to obtain a low-consumption limb fatigue detection model, so as to greatly reduce the computing consumption when using the low-consumption limb fatigue detection model. Then, the low-consumption limb fatigue detection model is used to detect the moving video to be judged. For the moving video to be judged whose second detection score is lower than the third preset score and higher than the fourth preset score, a secondary detection is performed, that is, the third similarity and the fourth similarity are compared, and the secondary detection of the moving video to be judged is completed, so as to further improve the accuracy of using the moving video to be judged to evaluate the motion state.

[0108] Please refer to Figure 2 , Figure 2 The flowchart of establishing a low-consumption face fatigue detection model provided by an embodiment of the present invention is shown. In some embodiments of this embodiment, before the step of inputting the face image to be judged into the low-consumption face fatigue detection model for detection, the method further includes:

[0109] Taking the first preset number of non-fatigued face images as positive training samples, and taking the first preset number of fatigued face images as negative training samples;

[0110] Using the SVM model to train and learn the positive training samples and negative training samples to obtain a low-consumption face fatigue detection model.

[0111] Specifically, since the SVM model is a low-consumption training and learning model, using the SVM model to establish a low-consumption face fatigue detection model can greatly reduce the computing consumption when using the low-consumption face fatigue detection model.

[0112] In some embodiments of this embodiment, the step of selecting standard non-fatigued face images from all non-fatigued face images includes:

[0113] Compare the confidence levels of all non-fatigued facial images;

[0114] Use the non-fatigued facial image with the highest confidence level as the standard non-fatigued facial image.

[0115] Specifically, the higher the confidence level of a non-fatigued facial image, the higher its accuracy, thereby ensuring that the standard non-fatigued facial image is a very accurate non-fatigued image.

[0116] In some embodiments of this embodiment, the step of selecting the standard fatigued facial image from all fatigued facial images includes:

[0117] Compare the confidence levels of all fatigued facial images;

[0118] Use the fatigued facial image with the highest confidence level as the standard fatigued facial image.

[0119] Specifically, the higher the confidence level of a fatigued facial image, the higher its accuracy, thereby ensuring that the standard fatigued facial image is a very accurate fatigued image.

[0120] In some embodiments of this embodiment, the step of selecting the standard non-fatigued limb movement video from all positive samples includes:

[0121] Compare the confidence levels of all positive samples, and use the positive sample with the highest confidence level as the standard non-fatigued limb movement video. Specifically, the higher the confidence level of a positive sample, the higher its accuracy, thereby ensuring that the standard non-fatigued limb movement video is a very accurate non-fatigued video.

[0122] In some embodiments of this embodiment, the step of selecting the standard fatigued limb movement video from all negative samples includes:

[0123] Compare the confidence levels of all negative samples, and use the negative sample with the highest confidence level as the standard fatigued limb movement video. Specifically, the higher the confidence level of a negative sample, the higher its accuracy, thereby ensuring that the standard fatigued limb movement video is a very accurate fatigued video.

[0124] In some embodiments of this embodiment, after the step of determining that the target user is in a fatigued state if the movement video to be discriminated is a fatigued limb movement video, the method further includes:

[0125] Send a warning message to remind the target user to take a rest.

[0126] Please refer to Figure 3 , Figure 3The following is a block diagram of a motion state evaluation system based on a low-power machine learning model provided by an embodiment of the present invention. An embodiment of the present application provides a motion state evaluation system based on a low-power machine learning model, which includes:

[0127] A to-be-detected motion video acquisition module 110, configured to acquire a to-be-detected motion video;

[0128] A target detection module 120, configured to detect the to-be-detected motion video by using target detection technology to obtain a to-be-determined face image and a to-be-determined motion video of a target user;

[0129] A standard non-fatigued face image determination module 130, configured to acquire a first preset number of non-fatigued face images and select a standard non-fatigued face image from all non-fatigued face images;

[0130] A standard fatigued face image determination module 140, configured to acquire a first preset number of fatigued face images and select a standard fatigued face image from all fatigued face images;

[0131] A first detection module 150, configured to input the to-be-determined face image into a low-power face fatigue detection model for detection to obtain a first detection score;

[0132] An image score determination module 160, configured to, if the first detection score is higher than a first preset score, determine that the to-be-determined face image is a non-fatigued face image, if the first detection score is lower than a second preset score, determine that the to-be-determined face image is a fatigued face image, and if the first detection score is higher than the second preset score and lower than the first preset score, proceed to the next step;

[0133] An image similarity calculation module 170, configured to calculate a first similarity between the to-be-determined face image and the standard non-fatigued face image, and a second similarity between the to-be-determined face image and the standard fatigued face image;

[0134] An image similarity comparison module 180, configured to, if the first similarity is higher than the second similarity, determine that the to-be-determined face image is a non-fatigued face image, and if the first similarity is lower than the second similarity, determine that the to-be-determined face image is a fatigued face image;

[0135] An image fatigue state determination module 190, configured to, if the to-be-determined face image is a fatigued face image, determine that the target user is in a fatigued state, and if the to-be-determined face image is a non-fatigued face image, proceed to the next step;

[0136] The low-power limb fatigue detection model training module 200 is used to obtain the second preset number of non-fatigued limb movement videos as positive samples, obtain the second preset number of fatigued limb movement videos as negative samples, and use the SVM model to train and learn the positive and negative samples to obtain a low-power limb fatigue detection model;

[0137] The standard fatigued limb movement video determination module 210 is used to select the standard non-fatigued limb movement videos from all the positive samples and select the standard fatigued limb movement videos from all the negative samples;

[0138] The second detection module 220 is used to detect the to-be-determined motion video by using the low-power limb fatigue detection model to obtain a second detection score;

[0139] The video score determination module 230 is used to, if the second detection score is higher than the third preset score, determine that the to-be-determined motion video is a non-fatigued limb movement video, if the second detection score is lower than the fourth preset score, determine that the to-be-determined motion video is a fatigued limb movement video, and if the second detection score is lower than the third preset score and higher than the fourth preset score, proceed to the next step;

[0140] The video similarity calculation module 240 is used to calculate the third similarity between the to-be-determined motion video and the standard non-fatigued limb movement video, and the fourth similarity between the to-be-determined motion video and the standard fatigued limb movement video;

[0141] The video similarity comparison module 250 is used to, if the third similarity is higher than the fourth similarity, determine that the to-be-determined motion video is a non-fatigued limb movement video, and if the third similarity is lower than the fourth similarity, determine that the to-be-determined motion video is a fatigued limb movement video;

[0142] The video fatigue state determination module 260 is used to, if the to-be-determined motion video is a fatigued limb movement video, determine that the target user is in a fatigued state, and if the to-be-determined motion video is a non-fatigued limb movement video, determine that the target user is in a non-fatigued state.

[0143] In the above implementation process, the system first uses object detection technology to detect moving objects in the moving video to be detected, so as to accurately locate the moving objects, thereby obtaining the face image to be judged and the moving video to be judged of the target user. Then, standard non-fatigued face images are selected from a sufficient number of non-fatigued face images, and standard fatigued face images are selected from a sufficient number of fatigued face images. Then, a low-consumption face fatigue detection model is used to detect and judge the face image to be judged, so as to greatly reduce the consumption of computing resources. Then, for the face image to be judged whose first detection score is higher than the second preset score and lower than the first preset score, a secondary detection is performed, that is, the first similarity and the second similarity are compared, and the secondary detection of the face image to be judged is completed, so as to further improve the accuracy of using the face image to be judged to evaluate the motion state. And if it is determined through the face image to be judged that the target user is not in a fatigued state, then the limb movements of the moving video to be judged are discriminated, so as to more accurately evaluate the motion state of the user in combination with the limb movements. When detecting the moving video to be judged, first, an SVM model is used to train a sufficient number of positive and negative samples to obtain a low-consumption limb fatigue detection model, so as to greatly reduce the computing consumption when using the low-consumption limb fatigue detection model. Then, the low-consumption limb fatigue detection model is used to detect the moving video to be judged. For the moving video to be judged whose second detection score is lower than the third preset score and higher than the fourth preset score, a secondary detection is performed, that is, the third similarity and the fourth similarity are compared, and the secondary detection of the moving video to be judged is completed, so as to further improve the accuracy of using the moving video to be judged to evaluate the motion state.

[0144] In some embodiments of the present embodiment, the above-mentioned motion state evaluation system based on a low-consumption machine learning model further includes:

[0145] A training sample obtaining module, configured to use the first preset number of non-fatigued face images as positive training samples and the first preset number of fatigued face images as negative training samples;

[0146] A low-consumption face fatigue detection model obtaining module, configured to use an SVM model to perform training and learning on the positive training samples and the negative training samples to obtain a low-consumption face fatigue detection model.

[0147] Specifically, since the SVM model is a low-consumption training and learning model, using the SVM model to establish a low-consumption face fatigue detection model can greatly reduce the computing consumption when using the low-consumption face fatigue detection model.

[0148] Please refer to Figure 4 , Figure 4 shown in the structural block diagram of a standard non-fatigued face image determination module 130 provided by an embodiment of the present invention. In some embodiments of the present embodiment, the above-mentioned standard non-fatigued face image determination module 130 includes:

[0149] A non-fatigue face image confidence comparison unit 131 for comparing the confidence levels of all non-fatigue face images;

[0150] A standard non-fatigue face image selection unit 132 for taking the non-fatigue face image with the highest confidence as the standard non-fatigue face image.

[0151] Specifically, the higher the confidence level of a non-fatigue face image, the higher its accuracy, thus ensuring that the standard non-fatigue face image is a very accurate non-fatigue image.

[0152] In some embodiments of this embodiment, the above-mentioned standard fatigue face image determination module 140 includes:

[0153] A fatigue face image confidence comparison unit for comparing the confidence levels of all fatigue face images;

[0154] A standard fatigue face image selection unit for taking the fatigue face image with the highest confidence as the standard fatigue face image.

[0155] Specifically, the higher the confidence level of a fatigue face image, the higher its accuracy, thus ensuring that the standard fatigue face image is a very accurate fatigue image.

[0156] In some embodiments of this embodiment, the above-mentioned standard fatigue limb movement video determination module 210 includes:

[0157] A standard non-fatigue video obtaining unit for comparing the confidence levels of all positive samples and taking the positive sample with the highest confidence as the standard non-fatigue limb movement video. Specifically, the higher the confidence level of a positive sample, the higher its accuracy, thus ensuring that the standard non-fatigue limb movement video is a very accurate non-fatigue video.

[0158] In some embodiments of this embodiment, the above-mentioned standard fatigue limb movement video determination module 210 includes:

[0159] A standard fatigue video obtaining unit for comparing the confidence levels of all negative samples and taking the negative sample with the highest confidence as the standard fatigue limb movement video. Specifically, the higher the confidence level of a negative sample, the higher its accuracy, thus ensuring that the standard fatigue limb movement video is a very accurate fatigue video.

[0160] In some embodiments of this embodiment, the above-mentioned motion state evaluation system based on a low-power machine learning model further includes:

[0161] A warning information sending module for sending a warning message to the user to remind the target user to take a rest.

[0162] Please refer to Figure 5 , Figure 5 which is a schematic structural block diagram of an electronic device provided by an embodiment of the present application. The electronic device includes a memory 101, a processor 102, and a communication interface 103. The memory 101, the processor 102, and the communication interface 103 are directly or indirectly electrically connected to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The memory 101 can be used to store software programs and modules, such as program instructions / modules corresponding to a motion state evaluation system based on a low-power machine learning model provided by an embodiment of the present application. The processor 102 executes various functional applications and data processing by executing the software programs and modules stored in the memory 101. The communication interface 103 can be used for signaling or data communication with other node devices.

[0163] Among them, the memory 101 can be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc.

[0164] The processor 102 can be an integrated circuit chip with signal processing capabilities. The processor 102 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0165] It can be understood that Figure 5 the structure shown is only schematic, and the electronic device may further include more or fewer components than those shown in Figure 5 , or have a structure different from that shown in Figure 5The different configurations shown. Figure 5 Each component shown in can be implemented by hardware, software, or a combination thereof.

[0166] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0167] In addition, each functional module in the various embodiments of this application can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part.

[0168] If the described function is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs, etc., which can store program codes.

[0169] The above are only the preferred embodiments of this application and are not used to limit this application. For those skilled in the art, this application can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of this application shall be included in the protection scope of this application.

[0170] It will be apparent to those skilled in the art that the present application is not limited to the details of the above-described exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, in all respects, the embodiments should be regarded as exemplary and non-limiting. The scope of the present application is defined by the appended claims rather than the above description. Accordingly, all changes that fall within the meaning and scope of the equivalent elements of the claims are intended to be embraced within the present application. Any reference signs in the claims should not be construed as limiting the claims concerned.

Claims

1. A method for evaluating a motion state based on a low-power machine learning model, characterized in that, It includes the following steps: Obtain the motion video to be detected; Use object detection technology to detect the motion video to be detected, so as to obtain the face image to be judged and the motion video to be judged of the target user; Obtain the first preset number of non-fatigued face images, and select the standard non-fatigued face image from all non-fatigued face images; Obtain the first preset number of fatigued face images, and select the standard fatigued face image from all fatigued face images; Input the face image to be judged into the low-power face fatigue detection model for detection to obtain the first detection score; If the first detection score is higher than the first preset score, it is determined that the face image to be judged is a non-fatigued face image. If the first detection score is lower than the second preset score, it is determined that the face image to be judged is a fatigued face image. If the first detection score is higher than the second preset score and lower than the first preset score, proceed to the next step; Calculate the first similarity between the face image to be judged and the standard non-fatigued face image, and the second similarity between the face image to be judged and the standard fatigued face image; If the first similarity is higher than the second similarity, it is determined that the face image to be judged is a non-fatigued face image. If the first similarity is lower than the second similarity, it is determined that the face image to be judged is a fatigued face image; If the face image to be judged is a fatigued face image, it is determined that the target user is in a fatigued state. If the face image to be judged is a non-fatigued face image, proceed to the next step; Obtain the second preset number of non-fatigued limb movement videos as positive samples, obtain the second preset number of fatigued limb movement videos as negative samples, and use the SVM model to train and learn the positive samples and the negative samples to obtain a low-power limb fatigue detection model; Select the standard non-fatigued limb movement video from all positive samples, and select the standard fatigued limb movement video from all negative samples; Use the low-power limb fatigue detection model to detect the motion video to be judged to obtain the second detection score; If the second detection score is higher than the third preset score, it is determined that the motion video to be judged is a non-fatigued limb movement video. If the second detection score is lower than the fourth preset score, it is determined that the motion video to be judged is a fatigued limb movement video. If the second detection score is lower than the third preset score and higher than the fourth preset score, proceed to the next step; Calculate the third similarity between the motion video to be judged and the standard non-fatigued limb movement video, and the fourth similarity between the motion video to be judged and the standard fatigued limb movement video; If the third similarity is higher than the fourth similarity, it is determined that the motion video to be judged is a non-fatigued limb movement video. If the third similarity is lower than the fourth similarity, it is determined that the motion video to be judged is a fatigued limb movement video; If the motion video to be discriminated is a fatigue-type limb movement video, it is determined that the target user is in a fatigue state. If the motion video to be discriminated is a non-fatigue-type limb movement video, it is determined that the target user is in a non-fatigue state.

2. The motion state evaluation method based on a low-power machine learning model according to claim 1, wherein Before the step of inputting the face image to be discriminated into the low-power face fatigue detection model for detection, it further includes: Taking the first preset number of non-fatigue face images as positive training samples and taking the first preset number of fatigue face images as negative training samples; Using the SVM model to train and learn the positive training samples and the negative training samples to obtain a low-power face fatigue detection model.

3. The motion state evaluation method based on a low-power machine learning model according to claim 1, wherein The step of selecting the standard non-fatigue face image from all non-fatigue face images includes: Comparing the confidence levels of all non-fatigue face images; Taking the non-fatigue face image with the highest confidence level as the standard non-fatigue face image.

4. The motion state evaluation method based on a low-power machine learning model according to claim 1, characterized in that The step of selecting the standard fatigue face image from all fatigue face images includes: Comparing the confidence levels of all fatigue face images; Taking the fatigue face image with the highest confidence level as the standard fatigue face image.

5. The motion state evaluation method based on a low-power machine learning model according to claim 1, wherein The step of selecting the standard non-fatigue limb movement video from all positive samples includes: Comparing the confidence levels of all positive samples and taking the positive sample with the highest confidence level as the standard non-fatigue limb movement video.

6. The motion state evaluation method based on a low-power machine learning model according to claim 1, characterized in that The step of selecting the standard fatigue limb movement video from all negative samples includes: Comparing the confidence levels of all negative samples and taking the negative sample with the highest confidence level as the standard fatigue limb movement video.

7. The motion state evaluation method based on a low-power machine learning model according to claim 1, characterized in that After the step of determining that the target user is in a fatigue state if the motion video to be discriminated is a fatigue-type limb movement video, it further includes: Sending a warning message to remind the target user to rest.

8. A motion state evaluation system based on a low-power machine learning model, characterized in that It includes: A motion video acquisition module to be detected, which is used to acquire a motion video to be detected; A target detection module, which is used to detect the motion video to be detected by using target detection technology to obtain the face image to be discriminated and the motion video to be discriminated of the target user; A standard non-fatigue face image determination module, which is used to acquire the first preset number of non-fatigue face images and select the standard non-fatigue face image from all non-fatigue face images; A standard fatigue face image determination module, which is used to acquire the first preset number of fatigue face images and select the standard fatigue face image from all fatigue face images; A first detection module, which is used to input the face image to be discriminated into the low-power face fatigue detection model for detection to obtain a first detection score; An image score determination module, which is used to determine that the face image to be discriminated is a non-fatigue face image if the first detection score is higher than the first preset score, determine that the face image to be discriminated is a fatigue face image if the first detection score is lower than the second preset score, and proceed to the next step if the first detection score is higher than the second preset score and lower than the first preset score; An image similarity calculation module, which is used to calculate the first similarity between the face image to be discriminated and the standard non-fatigue face image and the second similarity between the face image to be discriminated and the standard fatigue face image; An image similarity comparison module, configured to determine that the face image to be determined is a non-fatigued face image if the first similarity is higher than the second similarity, and determine that the face image to be determined is a fatigued face image if the first similarity is lower than the second similarity; An image fatigue state determination module, configured to determine that the target user is in a fatigued state if the face image to be determined is a fatigued face image, and proceed to the next step if the face image to be determined is a non-fatigued face image; A low-power limb fatigue detection model training module, configured to obtain a second preset number of non-fatigued limb movement videos as positive samples, obtain a second preset number of fatigued limb movement videos as negative samples, and use an SVM model to perform training and learning on the positive samples and the negative samples to obtain a low-power limb fatigue detection model; A standard fatigued limb movement video determination module, configured to select a standard non-fatigued limb movement video from all the positive samples and select a standard fatigued limb movement video from all the negative samples; A second detection module, configured to use the low-power limb fatigue detection model to detect the movement video to be determined and obtain a second detection score; A video score determination module, configured to determine that the movement video to be determined is a non-fatigued limb movement video if the second detection score is higher than a third preset score, determine that the movement video to be determined is a fatigued limb movement video if the second detection score is lower than a fourth preset score, and proceed to the next step if the second detection score is lower than the third preset score and higher than the fourth preset score; A video similarity calculation module, configured to calculate a third similarity between the movement video to be determined and the standard non-fatigued limb movement video, and a fourth similarity between the movement video to be determined and the standard fatigued limb movement video; A video similarity comparison module, configured to determine that the movement video to be determined is a non-fatigued limb movement video if the third similarity is higher than the fourth similarity, and determine that the movement video to be determined is a fatigued limb movement video if the third similarity is lower than the fourth similarity; A video fatigue state determination module, configured to determine that the target user is in a fatigued state if the movement video to be determined is a fatigued limb movement video, and determine that the target user is in a non-fatigued state if the movement video to be determined is a non-fatigued limb movement video.

9. An electronic device, characterized in that, Comprising: A memory, configured to store one or more programs; A processor; When the one or more programs are executed by the processor, the method according to any one of claims 1-7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the method according to any one of claims 1-7 is implemented.

Images