A deviation correction guidance method and system based on user motion feature extraction

Abstract

The present invention belongs to the technical field of computer vision, and particularly relates to a deviation correction guidance method and system based on user motion feature extraction. The method includes: obtaining an original frame image of a motion area, and real-time acquiring continuous frame motion images of a target user in the motion area; performing binary background subtraction and segmentation on the continuously acquired continuous frame motion images, obtaining a binary image of the target user in the current frame motion image and marking the contour pixels of the target user, and iteratively calculating the intermediate motion pixels of the target user based on the median value to obtain the feature parameters of the target user in the current frame motion image; comparing the similarity between the motion type loaded in the APP and the standard action guidance parameters with the feature parameters of the current frame motion image, obtaining the best standard motion image and coupling and projecting it onto the current frame motion image to obtain a feature deviation correction recognition result outside the common coupling feature space, and the part that requires deviation correction guidance can be visually observed.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and particularly relates to a deviation correction guidance method and system based on user motion feature extraction. Background Art

[0002] With the continuous development of artificial intelligence technology and the gradual maturity of computer vision technology, human motion recognition has become one of the major hotspots in the field of computer vision in recent years. It is widely used in many fields such as motion capture, human-computer interaction, and video surveillance, and has become an important technology for people to focus on intelligent video analysis research. Therefore, the detection of physical exercise methods based on human motion recognition has been quickly promoted.

[0003] Although the current physical exercise methods are constantly improving, the guidance for the physical exercise process still depends on on-site guidance by coaches or teaching assistants with professional knowledge. For groups that do not have the conditions to guide the physical exercise process, due to improper intensity or incorrect methods during the physical exercise process, there are numerous cases of ineffective exercise or sports injuries.

[0004] Therefore, there is an urgent need for a method and system that can perform deviation correction guidance based on user motion features, obtain the user's motion mode and motion features, monitor the user's motion process, and give timely guidance to ensure that the user can achieve the desired exercise effect. Summary of the Invention

[0005] In view of the above problems, the present invention is proposed. The present invention provides a deviation correction guidance method and system based on user motion feature extraction. Based on the image data containing the user's motion pictures taken by the image capture terminal in different environments, after excluding the influence of external environmental interference or the defects of the shooting target itself, the motion features such as the limb movements of the target user are identified and extracted. Based on the matching of the motion type and the standard action guidance in the APP, the motion process of the user is assisted and deviation-corrected to achieve the expected exercise effect.

[0006] The present invention is implemented by the following technical solutions:

[0007] A deviation correction guidance method based on user motion feature extraction, the method comprising:

[0008] Obtaining the original frame image of the motion area and continuously acquiring the frame motion images of the target user in the motion area in real time;

[0009] Performing binary background subtraction and segmentation on the continuously acquired frame motion images based on the original frame image to obtain a binary image of the target user in the current frame motion image;

[0010] Mark the pixel points of the target user's contour according to the obtained binary image, and iteratively calculate the intermediate motion pixel points of the target user based on the intermediate value to obtain the characteristic parameters of the target user in the current frame of motion image;

[0011] Based on the motion types loaded in the APP and the similarity comparison between the standard action guidance parameters and the characteristic parameters of the current frame of motion image, obtain the best standard motion image and couple and project it onto the current frame of motion image to obtain the feature deviation recognition result outside the common coupled feature space.

[0012] As a further solution of the present invention, the binary background subtraction and segmentation of the continuously acquired frame of motion image based on the original frame image includes:

[0013] Perform binary processing on the continuously acquired frame of motion image and the original frame image, set the original frame image as the background image and use the OTSU algorithm for binary processing of the image;

[0014] Differentiate the current frame of motion image in the continuously acquired frame of motion image from the background image, and equalize the histogram of the differential image;

[0015] Segment the foreground image in the current frame of motion image, remove holes and isolated points, and mark it as the binary image of the target user in the current frame of motion image.

[0016] As a further solution of the present invention, removing holes and isolated points includes:

[0017] Perform binary processing on the histogram of the differential image, and segment the foreground image in the current frame of motion image;

[0018] Obtain the binary value of each pixel point in the foreground image and perform neighborhood analysis and comparison;

[0019] Mark the pixel points with binary values higher than the neighboring points and the average difference from the neighboring points greater than the preset threshold as isolated points, and use morphological opening operation to remove the isolated points and smooth the contour of the binary image of the target user;

[0020] Mark the pixel points with binary values lower than the neighboring points and the absolute value of the average difference from the neighboring points greater than the preset threshold as holes, and use morphological closing operation to remove the hole points and smooth the contour of the binary image of the target user.

[0021] As a further solution of the present invention, marking the pixel points of the target user's contour according to the obtained binary image includes the following steps:

[0022] Based on the findContours() function provided by OpenCV, obtain the contour topology information of the binary image, save the hierarchical information of the contour, eliminate the internal points of the binary image, and obtain the contour point set;

[0023] Draw a contour of the target user based on the set of contour points. Among them, the API is used to draw the set of contour points of the binary image.

[0024] As a further solution of the present invention, when eliminating the internal points of the binary image to obtain the set of contour points, it includes:

[0025] Read the binary image data to obtain the image width, image height, and image row size;

[0026] Set the neighborhood window size and judge the neighborhood of the current pixel point of the binary image;

[0027] If all the neighborhood pixel points of the current pixel point are bright points, then the current pixel point is an internal point;

[0028] If not all the neighborhood pixel points of the current pixel point are bright points, then the current pixel point is a contour point;

[0029] Set all internal points as background points, and the remaining contour points form a set of contour points, completing the contour extraction of the binary image.

[0030] As a further solution of the present invention, set the neighborhood window size to a 3*3 window. If the eight neighborhood pixel points of the current pixel point satisfy:

[0031] P(x,y) is the target pixel. Assuming the target pixel is black 0 and the background pixel is white 255, then P(x,y)=0;

[0032] All eight neighborhood pixel points of P(x,y) are target pixels 0;

[0033] Then the current pixel point is an internal point. Delete the internal points that meet the above conditions and switch to the background point 366 to obtain the image contour.

[0034] As a further solution of the present invention, mark the contour pixel points of the target user according to the obtained binary image, and iteratively calculate the intermediate motion pixel points of the target user based on the intermediate value to obtain the characteristic parameters of the target user in the current frame motion image, including:

[0035] Fit the image contour of the target user based on the obtained target user contour pixel points;

[0036] Use the multi-point recognition technology to identify and extract the human body skeleton key points of the target user in the current frame motion image. Among them, the human body skeleton key points at least correspond to the joint parts of the neck, shoulders, elbows, wrists, waist, knees, and ankles of the target user;

[0037] Map the identified human body skeleton key points to the image contour of the target user, and obtain the human body skeleton key point framework by connecting lines.

[0038] Perform intermediate value iterative calculation on the intersection points on the image contour along the vertical direction of the human body bone key point framework, take the intermediate value and mark it, and fit to form a human body posture map;

[0039] Calculate the angles of the human body posture map between the joints of each part relative to the image border to form the characteristic parameters of the target user.

[0040] As a further solution of the present invention, obtain the best standard motion image and couple and project it onto the current frame motion image to obtain the feature deviation correction recognition result outside the common coupled feature space, including:

[0041] Based on the motion types loaded in the APP and the standard action guidance parameters, compare with the angles of the human body postures between the joints in the characteristic parameters of the current frame motion image relative to the image border, match the motion types and standard actions with similar angles between the joints of each part, and sort them according to the similarity to obtain the best standard motion image;

[0042] Scale the best standard motion image and then couple and project it onto the current frame motion image. The best standard motion image is coupled with the current frame motion image, and the feature deviation correction recognition result is displayed after coupling.

[0043] The present invention also includes a deviation correction guidance system based on user motion feature extraction. The deviation correction guidance system based on user motion feature extraction uses the above-mentioned deviation correction guidance method for user motion feature recognition and deviation correction guidance; the deviation correction guidance system based on user motion feature extraction includes an image acquisition module, a binary image acquisition module, a feature parameter acquisition module, and a coupling deviation correction module.

[0044] The image acquisition module is used to obtain the original frame image of the motion area and continuously acquire the motion images of the target user in the motion area in real time.

[0045] The binary image acquisition module is used to perform binary background subtraction and segmentation on the continuously acquired motion images in real time based on the original frame image to obtain the binary image of the target user in the current frame motion image.

[0046] The feature parameter acquisition module is used to mark the contour pixel points of the target user according to the obtained binary image, and based on intermediate value iterative calculation, obtain the intermediate motion pixel points of the target user, and obtain the feature parameters of the target user in the current frame motion image.

[0047] The coupling and deviation correction module is used to compare the similarity between the motion type loaded in the APP and the standard action guidance parameters and the feature parameters of the current frame motion image, obtain the best standard motion image, couple and project it onto the current frame motion image, and obtain the feature deviation correction recognition result outside the common coupling feature space.

[0048] The present invention further includes a computer device, including a memory and a processor. The memory stores computer-readable instructions, and when the processor loads and executes the computer-readable instructions, the steps of the deviation correction guidance method based on user motion feature extraction are implemented.

[0049] The present invention further includes a storage medium storing computer-readable instructions, and when the computer-readable instructions are loaded and executed by a processor, the steps of the deviation correction guidance method based on user motion feature extraction are implemented.

[0050] The technical solution provided by the present invention has the following beneficial effects:

[0051] The deviation correction guidance method and system based on user motion feature extraction provided by the present invention obtain the feature parameters of the target user by performing binarization processing and motion pixel point calculation on the consecutive frame motion images of the target user, match the motion type and the standard action guidance, and obtain the feature deviation correction recognition result outside the common coupling feature space after coupling and projecting the best standard motion image. By processing the motion image through the APP, the standard motion image is recognized and coupled to the motion image, and the part that needs deviation correction guidance and the corresponding motion precautions can be intuitively observed by viewing the feature deviation correction recognition result in the motion image. Description of the Drawings

[0052] The drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the drawings:

[0053] Figure 1 It is a flowchart of the deviation correction guidance method based on user motion feature extraction according to an embodiment of the present invention.

[0054] Figure 2 It is a flowchart of the binarization processing in the deviation correction guidance method based on user motion feature extraction according to an embodiment of the present invention.

[0055] Figure 3 It is a flowchart of removing holes and isolated points in the deviation correction guidance method based on user motion feature extraction according to an embodiment of the present invention.

[0056] Figure 4 It is an application schematic diagram of the binarization processing in the deviation correction guidance method based on user motion feature extraction according to an embodiment of the present invention.

[0057] Figure 5 This is a schematic diagram of the application of far-infrared scanner processing in the deviation correction guidance method based on user movement characteristics in the embodiments of the present invention.

[0058] Figure 6 This is a flowchart of obtaining the characteristic parameters of the target user in the deviation correction guidance method based on user movement characteristics in the embodiments of the present invention.

[0059] Figure 7 This is a schematic diagram of the multi-point recognition technology for extracting the key points of the human body bones in the deviation correction guidance method based on user movement characteristics in the embodiments of the present invention.

[0060] Figure 8 This is a system block diagram of the deviation correction guidance system based on user movement characteristics in an embodiment of the present invention.

[0061] Figure 9 This is a schematic diagram of the structure of a computer device suitable for implementing the embodiments of the present invention. Detailed implementation manners

[0062] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0063] It should be noted that all the technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the technical field to which this application belongs; the terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" and any variations thereof in the specification and claims of this application and the above drawings are intended to cover non-exclusive inclusion.

[0064] The terms "first", "second", etc. in the specification and claims of this application or the above drawings are used to distinguish different objects and are not used to describe a specific order. It should be understood that although the terms first, second, etc. may be used to describe various information in the embodiments of the present invention, the information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other.

[0065] The following description of at least one exemplary embodiment is actually only illustrative and in no way restricts the disclosure of the present invention and its application or use.

[0066] In the process of implementing the present disclosure, embodiments of the present invention provide a deviation correction guidance method based on user motion feature extraction. Based on image data containing user motion pictures captured by an image acquisition terminal in different environments, after excluding the interference of the external environment or the influence of defects of the shooting target itself, motion features such as the limb movements of the target user are identified and extracted. Based on the matching motion types and standard action guidance in the APP, the motion process of the user is assisted and deviation-corrected to achieve the expected exercise effect.

[0067] Embodiments of the present application provide a deviation correction guidance method based on user motion feature extraction that can be applied to an APP or a server. Specifically, it can be applied to an APP installed on a terminal device or within a system framework composed of a server and a terminal device. The APP and the terminal device, as well as the server and the terminal device, are connected through a wired or wireless network. Various APP client applications can also be installed on the terminal device, such as image processing applications, beauty picture applications, instant messaging software, etc.

[0068] The deviation correction guidance method based on user motion feature extraction provided by embodiments of the present application is applicable to the technical field of motion feature detection and action deviation correction guidance for real-time acquisition of any one or more images of a target user. For example: In practical applications, an application APP applet for processing motion images or graphics can be developed based on the inventive concept of the deviation correction guidance method based on user motion feature extraction provided by embodiments of the present application. The actions of the user's motion can be conveniently recognized by shooting through the camera of the terminal device or importing the user's motion image data through a data port, matching the standard actions, and performing precise guidance based on the coupling of feature parameters and standard motion images. The correct standard motion image is coupled to the motion image or graphic to form a standard motion video synchronized with the actions of the target user in the image. The part outside the coupling feature space is the feature deviation correction recognition result, and the target user can adjust the actions according to the feature deviation correction recognition result.

[0069] Specifically, the embodiments of the present application will be further described below with reference to the accompanying drawings.

[0070] Please refer to Figure 1 as shown Figure 1 The flowchart shows the process of an embodiment of the deviation correction guidance method based on user motion feature extraction according to the present invention. For the convenience of description, only the parts related to the embodiments of the present invention are shown. In the embodiments of the present invention, the present embodiment provides a deviation correction guidance method based on user motion feature extraction, including the following steps:

[0071] S10. Obtain the original frame image of the motion area and real-time acquire the continuous frame motion images of the target user in the motion area;

[0072] In this embodiment, the continuous-frame motion images may be captured by a camera carried by a terminal device, or images of a motion area may be obtained by shooting using acquisition devices such as a digital camera, a video camera, or a far-infrared scanner. When processing the continuous-frame motion images of the motion area, they can be copied into two copies, which are respectively marked as the first group of motion images to be processed and the second group of motion images to be processed.

[0073] In some embodiments of the present application, the continuous-frame images of the motion area may also be the image data of the motion area captured by a camera carried by a terminal device or by acquisition devices such as a digital camera, a video camera, or a far-infrared scanner installed. The continuous-frame motion images of the target time period are obtained through frame-by-frame processing.

[0074] Furthermore, the continuous-frame motion images may also be the video frame data of any time period of the acquired video data. In an embodiment of the present invention, when the set of motion images to be detected is a video (for example: video data captured by a digital camera of a target object, such as a motion video captured in an indoor motion area, or an image segment of a motion in a certain motion venue captured outdoors), the frame rate of the acquired video is transformed to obtain video frame data processed by frame-by-frame processing. Video frame pictures are obtained at a time period to obtain an image data set, and the video frame images are processed one by one in the order of frame-by-frame time. For example, video frame images are obtained at a frame rate of 1 frame per second, and the video frame pictures are sorted in time order as the image data set.

[0075] S20. Perform binary background subtraction and segmentation on the continuously acquired motion images in real time based on the original frame images to obtain a binary image of the target user in the current-frame motion image.

[0076] In this embodiment, referring to Figure 4 the binary image shown in the binary image after the binary processing of the input original image, a binary-processed picture is obtained by performing binary processing on a copy of the set of motion images to be detected. Among them, the image is processed by binary processing to obtain a black-and-white picture.

[0077] S30. Mark the contour pixels of the target user according to the obtained binary image, and iteratively calculate the intermediate motion pixels of the target user based on the intermediate value to obtain the characteristic parameters of the target user in the current-frame motion image.

[0078] S40. Compare the similarity between the motion type loaded in the APP and the standard action guidance parameters and the characteristic parameters of the current-frame motion image to obtain the best standard motion image and couple and project it onto the current-frame motion image to obtain a characteristic deviation recognition result outside the common coupled feature space.

[0079] Specifically, please refer to Figure 2Flowchart of the binarization process shown. The binary background subtraction and segmentation of the continuously acquired consecutive-frame motion images based on the original frame image includes the following steps:

[0080] S201. Binarize the continuously acquired consecutive-frame motion images and the original frame image, set the original frame image as the background image, and perform binarization on the image using the OTSU algorithm;

[0081] S202. Differentiate the current-frame motion image in the consecutive-frame motion images from the background image, and perform histogram equalization on the differential image;

[0082] S203. Segment the foreground image in the current-frame motion image, remove holes and isolated points, and mark it as the binary image of the target user in the current-frame motion image.

[0083] In this embodiment, refer to Figure 3 shown Figure 3 is the flowchart for removing holes and isolated points. When removing holes and isolated points, it includes:

[0084] S2031. Binarize the histogram of the differential image, and segment the foreground image in the current-frame motion image;

[0085] S2032. Obtain the binarization value of each pixel point in the foreground image, and perform neighborhood analysis and comparison;

[0086] S2033. Mark the pixel points with binarization values higher than the neighboring points and the average difference from the neighboring points greater than the preset threshold as isolated points, and use morphological opening operation to remove the isolated points and smooth the contour of the binary image of the target user;

[0087] S2034. Mark the pixel points with binarization values lower than the neighboring points and the absolute value of the average difference from the neighboring points greater than the preset threshold as holes, and use morphological closing operation to remove the hole points and smooth the contour of the binary image of the target user.

[0088] In some embodiments of the present application, for the consecutive-frame motion images of the motion area, they can also be grayscale processed after being copied to obtain the grayscale processed pictures. Among them, the image is grayscale processed to obtain a black-and-white picture.

[0089] In an embodiment of the present application, the grayscale image is based on the RGB model, making the values of the three color components R, G, and B of the color of the motion image to be detected the same, that is, R = G = B = wr * R + wg * G + wb * B, where wr, wg, and wb are the weights of R, G, and B respectively. Among them, the value of R = G = B is called the grayscale value. Each pixel of the grayscale image only needs one byte to store the grayscale value, also known as the intensity value or brightness value, and the grayscale range is 0 - 255. Among them, after determining the average grayscale value of the grayscale image by the average value method and determining the maximum grayscale value of each pixel point in the grayscale image by the maximum value method, the pixel position point area with a grayscale value greater than the average grayscale value and the floating points with grayscale values abnormal to the grayscale values of surrounding pixels can be determined by comparison.

[0090] See Figure 5 As shown, when continuously acquiring the motion images of the target user in the motion area in real time, a far-infrared scanner or an infrared camera can also be used to assist in human body recognition, and cooperate with the above-mentioned binary background subtraction and segmentation to accurately acquire the binary image of the target user in the current frame motion image.

[0091] In an embodiment of the present invention, marking the contour pixels of the target user according to the obtained binary image includes the following steps:

[0092] Based on the findContours() function provided by OpenCV, obtain the contour topological information of the binary image, save the hierarchical information of the contour, eliminate the internal points of the binary image, and obtain a set of contour points;

[0093] Draw a contour of the target user according to the set of contour points, where the API is used to draw the set of contour points of the binary image.

[0094] In an embodiment of the present invention, when eliminating the internal points of the binary image to obtain a set of contour points, it includes:

[0095] Read the binary image data to obtain the image width, image height, and image row size;

[0096] Set the neighborhood window size and judge the neighborhood of the current pixel point of the binary image;

[0097] If all the neighborhood pixel points of the current pixel point are bright points, then the current pixel point is an internal point;

[0098] If not all the neighborhood pixel points of the current pixel point are bright points, then the current pixel point is a contour point;

[0099] Set all internal points as background points, and the remaining contour points form a set of contour points, completing the contour extraction of the binary image.

[0100] In this embodiment, when eliminating the internal points of the binary image, since the binarization process sets the binarization values of the pixel points on the image to 0 or 255 according to a preset rule, the entire image presents an obvious visual effect of only black and white, so as to convert the binarized image into a binary image. The binary image maps the texture features in the motion image to be detected. When eliminating the internal points of the binary image, after binarization processing, the binarization threshold at the pixel position is determined according to the pixel value distribution of each pixel point, so as to distinguish the texture features of white pixels as the foreground area from other background areas of black pixels. All white pixels of the internal points in the foreground area are set to black pixels, and then the contour points of the foreground area are segmented. The remaining contour points form a contour point set, and the contour extraction of the binary image is completed.

[0101] In an embodiment of the present invention, the size of the neighborhood window is set to a 3*3 window. If the eight neighborhood pixel points of the current pixel point satisfy:

[0102] P(x, y) is the target pixel. Assuming that the target pixel is black 0 and the background pixel is white 255, then P(x, y) = 0;

[0103] All eight neighborhood pixel points of P(x, y) are target pixels 0;

[0104] Then the current pixel point is an internal point, and the internal points that meet the above conditions are deleted and switched to background point 366 to obtain the image contour.

[0105] In an embodiment of the present invention, the contour pixels of the target user are marked according to the obtained binary image, and the intermediate motion pixels of the target user are iteratively calculated based on the intermediate value. Refer to Figure 6 and Figure 7 As shown, the characteristic parameters of the target user in the current frame of the motion image are obtained, including:

[0106] S301. Fitting the image contour of the target user based on the obtained contour pixels of the target user;

[0107] S302. Using the multi-point recognition technology to identify and extract the human skeleton key points of the target user in the current frame of the motion image, where the human skeleton key points at least correspond to the joint parts of the neck, shoulders, elbows, wrists, waist, knees, and ankles of the target user;

[0108] S303. Mapping the identified human skeleton key points to the image contour of the target user, and obtaining the human skeleton key point framework by connecting lines;

[0109] S304. Performing intermediate value iterative calculation on the intersection points on the image contour along the vertical direction of the human skeleton key point framework, taking the intermediate value and marking it, and fitting to form a human body posture map;

[0110] S305. Calculate the angles of the human body posture diagram between the joints of each part relative to the image border to form the characteristic parameters of the target user.

[0111] In an embodiment of the present invention, obtaining the best standard motion image and coupling and projecting it onto the current frame motion image to obtain a feature deviation correction recognition result outside the common coupling feature space includes:

[0112] Based on the motion types loaded in the APP and the standard action guidance parameters, compare the angles of the human body posture between the joints in the characteristic parameters of the current frame motion image with the angles relative to the image border, match the motion types and standard actions with similar angles between the joints of each part, and sort them according to the similarity to obtain the best standard motion image;

[0113] Scale the best standard motion image and then perform coupling projection onto the current frame motion image. The best standard motion image is coupled with the current frame motion image, and the feature deviation correction recognition result is displayed after coupling.

[0114] As Figure 8 shown, Figure 8 is a structural block diagram of a deviation correction guidance system based on user motion feature extraction provided by an embodiment of the present application. The deviation correction guidance system based on user motion feature extraction can be applied to a motion deviation correction guidance APP or an image processing device, and can execute the method of deviation correction guidance based on user motion feature extraction in any of the above method embodiments. Specifically, in an embodiment of the present invention, a deviation correction guidance system based on user motion feature extraction is provided, including an image acquisition module 301, a binary image acquisition module 302, a feature parameter acquisition module 303, a coupling deviation correction module 304, and a contour drawing module 305.

[0115] Among them, the image acquisition module 301 is used to obtain the original frame image of the motion area and continuously acquire the motion images of the target user in the motion area in real time.

[0116] The binary image acquisition module 302 is used to perform binary background subtraction and segmentation on the continuously acquired motion images in real time based on the original frame image to obtain the binary image of the target user in the current frame motion image.

[0117] The feature parameter acquisition module 303 is used to mark the contour pixel points of the target user according to the obtained binary image, and iteratively calculate the intermediate motion pixel points of the target user based on the median value to obtain the feature parameters of the target user in the current frame motion image.

[0118] The coupling and deviation correction module 304 is configured to compare the similarity between the motion type loaded in the APP, the standard action guidance parameters, and the feature parameters of the current frame motion image, obtain the optimal standard motion image, couple and project it onto the current frame motion image, and obtain the feature deviation correction recognition result outside the common coupling feature space.

[0119] The contour drawing module 305 is configured to draw a contour of the target user according to the set of contour points. Among them, the API is used to draw the set of contour points of the binary image.

[0120] In this embodiment, when the deviation correction guidance system based on user motion feature extraction is executed, it can also adopt the steps of a deviation correction guidance method based on user motion feature extraction as described above, and can be applied to a graphics processing software for identifying and repairing miscellaneous points in an image during the image processing process.

[0121] The present invention can perform binarization processing and motion pixel point calculation on the consecutive frame motion images of the target user to obtain the feature parameters of the target user, match the motion type and standard action guidance, and obtain the feature deviation correction recognition result outside the common coupling feature space after coupling and projecting the optimal standard motion image. By processing the motion image through the APP, the standard motion image is identified and coupled to the motion image, and the part that needs deviation correction guidance and the corresponding motion precautions can be intuitively observed by viewing the feature deviation correction recognition result in the motion image. Among them, when the deviation correction guidance system based on user motion feature extraction is executed, it adopts the steps of a deviation correction guidance method based on user motion feature extraction as described above. Therefore, the operation process of the deviation correction guidance system based on user motion feature extraction in this embodiment will not be introduced in detail.

[0122] See Figure 9 As shown, in an embodiment of the present invention, a computer device 1000 is further provided, including a memory 1001 and a processor 1002. Computer-readable instructions are stored in the memory 1001, and when the processor 1002 loads and executes the computer-readable instructions, the steps in the above method embodiments are implemented.

[0123] Among them, those skilled in the art of the present technology can understand that the computer device 1000 here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits, programmable gate arrays, digital signal processors, embedded devices, etc. The computer device 1000 can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The computer device 1000 can perform human-computer interaction with the user through a keyboard, a mouse, a remote control, a touchpad, a voice control device, etc.

[0124] In addition, some embodiments of the present invention further include a storage medium having a program for executing the methods described in this specification on a computer, on which computer-readable instructions are stored. When the computer-readable instructions are loaded and executed by the processor 1002, the steps in the above method embodiments are implemented. Examples of the computer-readable recording medium include hardware devices specifically configured to store and execute program commands, such as magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floppy disks, and ROM, RAM, flash memory, etc. Examples of program commands may include machine language codes written by compilers and high-level language codes executed by a computer using interpreters, etc.

[0125] Among them, the processor 1002 may be a central processing unit, a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. In this embodiment, the processor 1002 is used to run the computer-readable instructions stored in the memory 1001 or process data, such as running the computer-readable instructions of the image-based health status recognition method.

[0126] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through computer-readable instructions. The computer-readable instructions can be stored in a non-volatile computer-readable storage medium. When the computer-readable instructions are executed, they may include the processes of the above method embodiments. Among them, any reference to the memory 1001, storage, database, or other media used in the embodiments provided in the present application may include at least one of non-volatile and volatile memories.

[0127] In summary, the deviation correction guidance method and system provided by the present invention based on user motion feature extraction obtain the characteristic parameters of the target user by performing binarization processing and motion pixel point calculation on consecutive frame motion images of the target user, match the motion type and standard action guidance, and obtain the characteristic deviation correction recognition result outside the common coupling feature space after coupling and projecting the best standard motion image. By processing the motion image through the APP, the standard motion image is recognized and coupled to the motion image. By viewing the characteristic deviation correction recognition result in the motion image, the part that needs deviation correction guidance and the corresponding motion precautions can be intuitively observed.

[0128] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims

1. A deviation correction guidance method based on user motion feature extraction, characterized in that, it includes the following steps: Obtain the original frame image of the motion area, and in real time obtain the continuous frame motion images of the target user in the motion area; Perform binary background subtraction and segmentation on the continuously acquired real-time continuous frame motion images based on the original frame image to obtain a binary image of the target user in the current frame motion image; Mark the contour pixel points of the target user according to the obtained binary image, and iteratively calculate the intermediate motion pixel points of the target user based on the median value to obtain the characteristic parameters of the target user in the current frame motion image; Perform a similarity comparison between the motion type loaded in the APP and the standard action guidance parameters and the characteristic parameters of the current frame motion image, obtain the best standard motion image and couple and project it onto the current frame motion image to obtain a characteristic deviation correction recognition result outside the common coupling feature space; Among them, marking the contour pixel points of the target user according to the obtained binary image includes the following steps: Obtain the contour topological information of the binary image based on the findContours() function provided by OpenCV, save the hierarchical information of the contour, eliminate the internal points of the binary image, and obtain a contour point set; Draw a contour of the target user according to the contour point set, where the API is used to draw the contour point set of the binary image; When eliminating the internal points of the binary image to obtain a contour point set, it includes: Read the binary image data, and obtain the image width, image height and image row size; Set the neighborhood window size and judge the neighborhood of the current pixel point of the binary image; If all the neighborhood pixel points of the current pixel point are bright points, then the current pixel point is an internal point; If not all the neighborhood pixel points of the current pixel point are bright points, then the current pixel point is a contour point; Set all internal points as background points, and the remaining contour points form a contour point set to complete the contour extraction of the binary image; Among them, obtaining the best standard motion image and coupling and projecting it onto the current frame motion image to obtain a characteristic deviation correction recognition result outside the common coupling feature space includes: Based on the motion type loaded in the APP and the standard action guidance parameters, compare the angles of the human body postures between the joints of each part in the characteristic parameters of the current frame motion image with respect to the image border, match the motion types and standard actions with similar angles between the joints of each part, and sort them according to the similarity to obtain the best standard motion image; Scale the best standard motion image and then couple and project it onto the current frame motion image. The best standard motion image is coupled with the current frame motion image, and the characteristic deviation correction recognition result is displayed after coupling.

2. The deviation correction guidance method based on user motion feature extraction according to claim 1, characterized in that, the performing binary background subtraction and segmentation on the continuously acquired real-time continuous frame motion images based on the original frame image includes: Perform binary processing on the continuously acquired real-time continuous frame motion images and the original frame image, and set the original frame image as the background image and perform binary processing on the image using the OTSU algorithm; Differentiate the current frame moving image in the continuous frame moving image from the background image, and perform histogram equalization on the differential image; Segment the foreground image in the current frame moving image, remove holes and isolated points, and mark it as the binary image of the target user in the current frame moving image.

3. The deviation correction guidance method based on user motion feature extraction according to claim 2, characterized in that, Removing holes and isolated points includes: Binarize the histogram of the differential image, and segment the foreground image in the current frame moving image; Obtain the binarization value of each pixel point in the foreground image, and perform neighborhood analysis and comparison; Mark the pixel points whose binarization value is higher than the adjacent points and the average difference from the adjacent points is greater than the preset threshold as isolated points, and use morphological opening operation to remove the isolated points and smooth the contour of the binary image of the target user; Mark the pixel points whose binarization value is lower than the adjacent points and the absolute value of the average difference from the adjacent points is greater than the preset threshold as holes, and use morphological closing operation to remove the hole points and smooth the contour of the binary image of the target user.

4. The deviation correction guidance method based on user motion feature extraction according to claim 1, characterized in that, Set the neighborhood window size to a 3*3 window. If the eight neighborhood pixel points of the current pixel point satisfy: P(x,y) is the target pixel. Assuming the target pixel is black 0 and the background pixel is white 255, then P(x,y)=0; All eight neighborhood pixel points of P(x,y) are target pixels 0; Then the current pixel point is an internal point, and the internal points that meet the conditions are deleted and switched to background points 366 to obtain the image contour.

5. The deviation correction guidance method based on user motion feature extraction according to claim 1, characterized in that, Mark the contour pixel points of the target user according to the obtained binary image, and iteratively calculate the intermediate motion pixel points of the target user based on the intermediate value to obtain the characteristic parameters of the target user in the current frame moving image, including: Fit the image contour of the target user based on the obtained contour pixel points of the target user; Adopt multi-point recognition technology to identify and extract the human skeleton key points of the target user in the current frame moving image, where the human skeleton key points at least correspond to the joint parts of the neck, shoulders, elbows, wrists, waist, knees, and ankles of the target user; Map the identified human skeleton key points to the image contour of the target user, and obtain the human skeleton key point framework by connecting lines; Perform intermediate value iterative calculation on the intersection points on the image contour along the vertical direction of the human skeleton key point framework, take the intermediate value and mark it, and fit to form a human posture map; Calculate the angles of the human posture map between each joint part relative to the image border to form the characteristic parameters of the target user.

6. A deviation correction guidance system based on user motion feature extraction, characterized in that, The deviation correction guidance system based on user motion feature extraction uses the deviation correction guidance method based on user motion feature extraction described in any one of claims 1-5 to identify and correct the guidance of user motion features; the deviation correction guidance system based on user motion feature extraction includes: An image acquisition module for obtaining the original frame image of the motion area and real-time obtaining the continuous frame moving image of the target user in the motion area; A binary image acquisition module, which is used to perform binary background subtraction and segmentation on continuously acquired consecutive frame motion images based on the original frame image, and obtain a binary image of the target user in the current frame motion image; A feature parameter acquisition module, which is used to mark the contour pixel points of the target user according to the obtained binary image, and iteratively calculate the intermediate motion pixel points of the target user based on the intermediate value, so as to obtain the feature parameters of the target user in the current frame motion image; A coupling deviation correction module, which is used to compare the similarity between the motion type loaded in the APP and the standard action guidance parameters and the feature parameters of the current frame motion image, obtain the best standard motion image and couple and project it onto the current frame motion image to obtain a feature deviation correction recognition result outside the common coupling feature space; A contour drawing module, which is used to draw a contour of the target user according to the set of contour points. Among them, the API is used to draw the set of contour points of the binary image.

Images