Motion monitoring, evaluating and correcting system based on Openpose and Blazopose algorithms

Abstract

The invention discloses a motion monitoring, evaluating and correcting system based on Openpose and Blazopose algorithms, and the system comprises a camera which is used for collecting a motion video; the display screen is used for displaying the collected motion video in real time and displaying a picture for monitoring, evaluating and correcting the motion posture; the communication module is used for uploading the collected motion video to the cloud platform and transmitting the posture monitoring and evaluation correction result of the cloud platform to the display screen; the power supply module is used for supplying power to the camera, the display screen and the communication module; and the cloud platform is used for carrying out attitude monitoring, evaluation and correction on the motion subject in the motion video based on Openpose and Blazopose algorithms. According to the method, the human body posture under the motion condition can be effectively recognized, and the guidance requirements of ordinary people when the ordinary people carry out different kinds of motion entering are met.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and particularly relates to a motion monitoring, evaluation and correction system based on Openpose and Blazepose algorithms. Background Art

[0002] With the development of living standards, the quality of people's lives has been gradually improved. People increasingly attach importance to the improvement of physical fitness and sports performance. People generally actively participate in various exercises. However, due to the low popularity rate of the specialization of various sports, it is difficult for ordinary people to achieve corresponding effects when practicing. Therefore, there is an urgent need for a product that can automatically evaluate sports postures and provide correction suggestions and guidance. In recent years, deep neural networks have achieved great success in human pose recognition and are widely used in fields such as security, human-computer interaction, and motion sensing games.

[0003] The existing technology has an excessive computational amount for pose recognition. The same algorithm is used for motions of different complexities, resulting in a slow processing speed and low recognition accuracy. Summary of the Invention

[0004] In view of the deficiencies in the existing technology, the present invention provides a motion monitoring, evaluation and correction system based on Openpose and Blazepose algorithms. By using the Openpose algorithm for simple motions and the Blazepose algorithm for complex motions, it can achieve real-time monitoring, evaluation and scoring of the motion postures in the video, and give suggestions, and can effectively identify the human postures in the motion situation.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A motion monitoring, evaluation and correction system based on Openpose and Blazepose algorithms, comprising:

[0007] A camera, used for collecting motion videos;

[0008] A display screen, used for real-time displaying the collected motion videos, as well as displaying the pictures of motion posture monitoring, evaluation and correction;

[0009] A communication module, used for uploading the collected motion videos to the cloud platform and transmitting the results of the posture monitoring, evaluation and correction of the cloud platform to the display screen;

[0010] A power supply module, used for supplying power to the camera, the display screen and the communication module;

[0011] A cloud platform, used for performing posture monitoring, evaluation and correction on the motion subjects in the motion videos based on Openpose and Blazepose algorithms.

[0012] To optimize the above technical solution, the specific measures also include:

[0013] Further, the posture monitoring and evaluation correction of the moving object in the motion video based on the Openpose and Blazepose algorithms are specifically as follows:

[0014] Initially call the Blazepose algorithm to monitor 33 key nodes of the human body;

[0015] Count the number of nodes with relative coordinate offsets. If the number exceeds 16, switch the algorithm to Openpose for motion posture monitoring and evaluation correction. Otherwise, continue to use the Blazepose algorithm for motion posture monitoring and evaluation correction.

[0016] Further, the specific method for counting the number of nodes with relative coordinate offsets is as follows:

[0017] Nmoving = count(K where Kmoving = TRUE)

[0018] In the formula, Nmoving is the number of nodes with relative coordinate offsets, count(K where Kmoving = TRUE) is a counting function, which means adding 1 when the relative coordinate offset of a certain node is true, and K where Kmoving = TRUE means that the relative coordinate offset of a certain node is true.

[0019] Further, the specific method for switching the algorithm to Openpose for motion posture monitoring and evaluation correction is as follows:

[0020] Train a normalization model through a convolutional neural network;

[0021] Compare the posture nodes recognized by the Openpose algorithm with the trained normalization model. Specifically:

[0022] Form a vector from the recognized posture nodes, calculate the vector angle as the joint angle, and compare the cosine value of the joint angle with a soft threshold;

[0023] Give corresponding real-time scores and modification opinions according to the gap between the cosine value of the joint angle and the soft threshold.

[0024] Further, the specific method for using the Blazepose algorithm for motion posture monitoring and evaluation correction is as follows:

[0025] Compare the posture nodes monitored by the Blazepose algorithm with the travel endpoints of repetitive motions to calculate the soft threshold. Set the interval values less than the soft threshold to 0 to indicate that the action is not in place, and set the interval values greater than the soft threshold to 1 to indicate that the action is in place, so as to realize the posture detection of whether the motion posture is standard.

[0026] Furthermore, the convolutional neural network includes an input layer, a convolutional layer, an activation layer, a pooling layer, a fully connected layer, and an output layer;

[0027] The convolutional layer is represented by the following formula:

[0028] Conv(X i ,W k )=ΣjX ij ·W jk +B k

[0029] In the formula, X i is the input feature map, W k is the k-th convolutional kernel, and B k is the bias of the k-th convolutional kernel; j is the serial number of the sliding window, and X ij is the j-th area of the feature map scanned by the sliding window when the convolutional kernel scans the feature map, and W jk is the k-th convolutional kernel of the j-th area of the scanned feature map;

[0030] The pooling layer is max pooling or average pooling. Max pooling is expressed as:

[0031] MaxPool(Z i )=max(Z i )

[0032] In the formula, MaxPool(·) represents max pooling, and Z i is the eigenvalue in the pooling window; max(Zi) represents taking the maximum value among the eigenvalues,

[0033] Average pooling is expressed as:

[0034]

[0035] In the formula, n is the total number of pooling windows, AvgPool(·) represents average pooling, and Z im represents the eigenvalue in the m-th pooling window.

[0036] Furthermore, the specific process of forming a vector from the recognized pose nodes and calculating the vector angle as the joint angle is as follows:

[0037] The shoulder coordinates are S(Sx, Sy, Sz), the hand coordinates are H(Hx, Hy, Hz), and the elbow coordinates are E(Ex, Ey, Ez); the angle of the elbow joint is obtained using the angle between the spatial vectors ES and EH. The calculation process is as follows:

[0038]

[0039] is the vector between the shoulder and the elbow, is the vector between the hand and the elbow, and cos α is the cosine value of the elbow joint angle α.

[0040] Furthermore, the calculation method of the soft threshold is as follows:

[0041]

[0042] In the formula, α is the elbow joint angle, T is the set threshold, and soft(α,T) is the soft threshold.

[0043] Furthermore, the display screen is an HDMI display screen.

[0044] Furthermore, the power supply mode of the power supply module is power supply via Ethernet.

[0045] The beneficial effects of the present invention are as follows: The designed motion posture monitoring and evaluation and correction system of the present invention can realize real-time capture of a single-person motion posture at the video end, and give scores and corresponding posture correction suggestions according to the motion changes. The motion mode is split according to the complexity of the motion actions, into simple motions and complex motions. For complex motions such as badminton and tennis, openpose is used to split the human body key nodes into eighteen skeleton points, and the RGB image is two-dimensional and three-dimensionalized, solving the influence of the depth problem on the inaccurate positioning accuracy of joint nodes. By using the convolutional neural network (CNN) and multiple groups of professional athlete motion videos to train the standardized model, comparing the motion videos of the motion personnel with the standardized model, and giving corresponding modification suggestions and real-time angle prompts according to the soft threshold gap. For simple motions but motions that require precise identification of action details such as fitness leg press and dumbbell curl, blazepose is used to further split the human body into 33 skeleton nodes and improve the frame rate. When performing the leg press motion, nodes 23, 25, and 27 are called, and when performing the dumbbell curl motion, nodes 11, 13, and 15 are called. Comparing the posture nodes with the stroke endpoints of the repetitive motion to calculate the soft threshold, and giving the stroke progress bar of the motion to realize the evaluation of the in-place degree of the motion posture. Experiments show that the system can realize the basic functions and has a certain degree of intelligence, can effectively identify the human body posture in the motion situation, and can perform real-time evaluation and analysis at the video end, helping the observer to focus on observing the posture nodes of the key parts during the motion, and performing real-time evaluation and correction according to the in-place degree of the posture nodes, helping the mover to realize the posture standardization correction under different project motions, improving the motion ability and level, verifying the feasibility of the present invention, having a certain practical value, and being beneficial to the guidance needs of ordinary people when starting different kinds of sports. At the same time, the present invention can also be used as a machine recognition scoring tool for school physical education assessments of all ages, avoiding the occurrence of unfair physical education assessments and situations where the assessment teacher misses seeing and evaluating, ensuring the accuracy and fairness of the assessment. Description of the Drawings

[0046] Figure 1 This is the overall block diagram of the motion monitoring and evaluation correction system proposed by the present invention;

[0047] Figure 2 This is a schematic diagram of the coordinate distribution of Blazepose human detection;

[0048] Figure 3 This is a schematic diagram of the leg press motion in a qualified state;

[0049] Figure 4 This is a schematic diagram of the leg press motion in an unqualified state;

[0050] Figure 5 is a schematic diagram of the barbell curl at different strokes; among them, Figure 5a This is a schematic diagram at the initial stage of the stroke, Figure 5b This is a schematic diagram when the stroke is close to 100%, Figure 5c This is a schematic diagram when the stroke reaches 100%;

[0051] Figure 6 This is a schematic diagram of starting to monitor the swing ball posture;

[0052] Figure 7 This is a schematic diagram of providing suggestions for too low swing ball angle;

[0053] Figure 8 This is a schematic diagram of the normal situation after the swing ball angle is corrected;

[0054] Figure 9 This is the schematic diagram of the BCM2837 chip;

[0055] Figure 10 This is the extended interface module;

[0056] Figure 11 This is the schematic diagram of HDMI;

[0057] Figure 12 This is the schematic diagram of the power supply module;

[0058] Figure 13 This is the schematic diagram of the DC-DC switching voltage regulator chip;

[0059] Figure 14 This is the schematic diagram of the POE power supply module. Detailed Implementation Manner

[0060] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present application.

[0061] Embodiment 1

[0062] The present invention proposes a motion monitoring, evaluation and correction system based on Openpose and Blazepose algorithms. The overall block diagram of the system is as Figure 1 shown and includes:

[0063] A camera for collecting motion videos;

[0064] A display screen for real-time displaying of the collected motion videos and for displaying the pictures of motion posture monitoring, evaluation and correction;

[0065] A communication module for uploading the collected motion videos to the cloud platform and for transmitting the results of posture monitoring, evaluation and correction of the cloud platform to the display screen;

[0066] A power supply module for supplying power to the camera, the display screen and the communication module;

[0067] A cloud platform for performing posture monitoring, evaluation and correction on the moving object in the motion video based on Openpose and Blazepose algorithms.

[0068] In this embodiment, a circuit is built using Raspberry Pi 3B+, a 3.5-inch touch screen, a USB camera and a power supply module. The communication module has been integrated into Raspberry Pi 3B+. The software part mainly includes the development of the cloud platform, posture acquisition and comparison, etc. The front-end camera acquisition module is used to collect single-person motion pictures in real time and can display the collected pictures on the HDMI display screen in real time. At the back end, the video file of the user is retrieved and traversed by the algorithm, and the pictures of motion posture monitoring, evaluation and correction are displayed on the display screen of the Raspberry Pi, realizing the human-computer interaction of the whole system, and achieving the self-correction of the motion posture by using the motion posture monitoring and evaluation correction system based on Raspberry Pi.

[0069] The posture monitoring, evaluation and correction of the moving object in the motion video based on Openpose and Blazepose algorithms specifically are:

[0070] Initially, the Blazepose algorithm is called to monitor 33 key nodes of the human body; Blazepose belongs to the Bottom-up type of algorithm, and its main feature is that the overall speed has been significantly improved.

[0071] Count the number of nodes with relative coordinate offsets. If the number exceeds 16, switch the algorithm to Openpose for motion posture monitoring, evaluation and correction. Otherwise, continue to use the Blazepose algorithm for motion posture monitoring, evaluation and correction. The specific method for counting the number of nodes with relative coordinate offsets is:

[0072] Nmoving = count(K where Kmoving = TRUE)

[0073] In the formula, Nmoving is the number of nodes with relative coordinate offsets, count(K where Kmoving = TRUE) is a counting function, which means adding one when the relative coordinate offset of a certain node is true, and K where Kmoving = TRUE means that the relative coordinate offset of a certain node is true.

[0074] Compared with the traditional Openpose algorithm, the frame rate (fps) of Blazepose is several times that of it, and the difference in accuracy is not very large. Therefore, Blazepose is very suitable for the development of mobile device terminals, and its speed improvement can be clearly reflected by comparing the fps.

[0075] Table 1 Comparison of the two algorithms in various situations

[0076]

[0077] PCK@0.2 in the table is a classic indicator in human pose detection, which represents the Euclidean distance between the key points predicted by the model and the true key points. If this distance is less than 20% of the distance of the entire human torso, the prediction is considered accurate. The traditional coco dataset competition requires 18 key points, while Blazepose can reach 33 key points. The coordinate distribution of Blazepose human detection is as Figure 2 shown.

[0078] The switching algorithm mentioned above is for Openpose to monitor and evaluate the motion pose and correct it specifically as follows:

[0079] Train a standardized model through a convolutional neural network; the convolutional neural network includes an input layer, a convolutional layer, an activation layer, a pooling layer, a fully connected layer, and an output layer;

[0080] The convolutional layer is expressed by the following formula:

[0081] Conv(X i , W k ) = ΣjX ij ·W jk + B k

[0082] In the formula, X i is the input feature map, W k is the kth convolutional kernel, B k is the bias of the kth convolutional kernel; j is the serial number of the sliding window, X ij is the jth area of the sliding window scanning the feature map when the convolutional kernel scans the feature map, W jkis the k-th convolutional kernel in the j-th region of the scanned feature map;

[0083] The pooling layer is max pooling or average pooling. Max pooling is expressed as:

[0084] MaxPool(Z i ) = max(Z i )

[0085] In the formula, MaxPool(·) represents max pooling, and Z i is the feature value in the pooling window; max(Zi) represents taking the maximum value among the feature values.

[0086] Average pooling is expressed as:

[0087]

[0088] In the formula, n is the total number of pooling windows, AvgPool(·) represents average pooling, and Z im represents the feature value in the m-th pooling window.

[0089] Compare the pose nodes recognized by the Openpose algorithm with the trained standardized model. Specifically:

[0090] Form the recognized pose nodes into a vector, and calculate the vector angle as the joint angle. Specifically:

[0091] The shoulder coordinates are S(Sx, Sy, Sz), the hand coordinates are H(Hx, Hy, Hz), and the elbow coordinates are E(Ex, Ey, Ez); use the angle between the spatial vectors ES and EH to obtain the elbow joint angle. The calculation process is as follows:

[0092]

[0093] is the vector between the shoulder and the elbow, is the vector between the hand and the elbow, and cos α is the cosine value of the elbow joint angle α.

[0094] Compare the cosine value of the joint angle with the soft threshold; the calculation method of the soft threshold is:

[0095]

[0096] In the formula, α is the elbow joint angle, T is the set threshold, and soft(α, T) is the soft threshold.

[0097] Give corresponding real-time scores and modification opinions according to the difference between the cosine value of the joint angle and the soft threshold.

[0098] The specific process of using the Blazepose algorithm for motion posture monitoring and evaluation and correction is as follows:

[0099] Compare the pose nodes detected by the Blazepose algorithm with the stroke endpoints of repetitive motions to calculate the soft threshold. Set the interval values less than the soft threshold to 0, indicating that the movement is not in place, and set the interval values greater than the soft threshold to 1, indicating that the movement is in place, so as to realize the pose detection of whether the motion posture is standard.

[0100] For the detection of the leg press motion mode, open and start the program, and import the video into the host computer interface as Figure 3 and Figure 4 shown. The progress bar on the right shows the degree of leg press in place. 100% is a perfect posture, 80% and above is qualified. The key nodes of the monitored leg are displayed in the video source to better observe the motion posture of the moving leg.

[0101] For the detection of the barbell curl motion mode, open and start the program, and import the video into the host computer interface as shown in Figure 5. The progress bar on the right shows the barbell curl stroke. 100% is the end. The progress bar needs to be trained basically from 0 to 100. The key nodes of the monitored arm are displayed in the video source to better observe the motion posture of the moving arm.

[0102] For complex motions, it is not possible to use Blazepose to give suggestions for pose recognition. In this case, this embodiment selects to use Openpose for pose recognition and gives correction suggestions according to the deviation between the recognized angle and the standard model. This embodiment uses the tennis motion as the complex motion for recognition. The specific operations are as follows: Open and run the program, and the key body nodes of the mover in the video source can be observed in real time. Mark the swing angle above the mover's head, and provide real-time evaluation of the swing height and angle according to the change of the arm angle and give improvement suggestions. As Figures 6 to 8 shown.

[0103] The circuit principle of this system is as follows:

[0104] Figure 9 This is a partial application circuit diagram of the Broadcom BCM2837, the main control chip of the Raspberry Pi 3B+. The BCM2837 ARM A53 chip adopts a 64-bit architecture and has four cores, mastering a variety of processing capabilities. The core frequency of 1.2GHz can perform multiple tasks simultaneously. The Raspberry Pi 3B+ has now built-in 802.11n WiFi and Bluetooth 4.0.

[0105] The N2 pin and the B11 pin are respectively connected to pins 27 and 28 of the expansion interface. These two pins are prepared for the HAT IDEEPROM. When enabled, this I2C interface will be queried to find an EEPROM that identifies the add-on board and allows for the automatic setting of gpio (and optionally the Linux driver). Do not use these pins except for attaching an I2C IDEEPROM. GPIO2 is the SDA1 data transfer port; GPIO3 is the SCL1 clock line; GPIO4 is the GPIO_GCLK; GPIO9, GPIO10, and GPIO11 are the MISO, MOSI, and SCLK ends of the SPI bus respectively; GPIO14 and GPIO15 are the TXD0 and RXD0 input / output ends respectively; GPIO7 and GPIO8 are the enable pins CE0 and CE1 of the SPI respectively.

[0106] The expansion interface of the Raspberry Pi 3 is as Figure 10 shown. This interface can implement numerous peripheral interfaces. An external SPI display, a serial screen, various sensors, etc. can be added. In this system, the expansion interface module is used to connect a 3.5-inch display, and IO ports 1 - 26 are connected to the display interface. Pin 1 is the 3.3V power input; pins 2 and 4 are the 5V power inputs, and a 100n capacitor is used to protect the circuit at the same time; pins 3 and 5 NC are not used and are reserved; pin 6 is the power ground; pins 7 and 8 are also not used and are reserved; pins 9, 14, 20, and 25 are the power lines grounded; pins 10 - 13, 15 - 16, 18, and 24 are not used and are reserved. Pin 17 is the 3.3V power input; pin 19 is the touch screen SPI bus data input; pin 21 is the touch screen SPI bus output; pin 22 is the touch screen interrupt (low level when a touch is detected); pin 23 is the touch screen SPI bus clock signal; pin 26 is the touch screen chip select signal (enabled by low level).

[0107] The display used in this embodiment is an HDMI display, Figure 11 which is the HDMI schematic diagram; RT9741CGV is a high-voltage driver chip. The HDMI and VGA VGA adapters both obtain power from this place. That is to say, the BCM857BS 5V voltage can output 2.5A, with a maximum of 12.5W.

[0108] On the circuit of the Raspberry Pi 3B +, there are different types of 5V, 3V3, and 1V8; 5V_CORE, VDD_CORE, PLL_VDD, RUN, H5V (HDMI), AUD_3V3 (audio). Figure 12 This is the schematic diagram of the power supply module.

[0109] First, looking at the power supply input end of MicoUSB, this interface only serves for power supply and does not have a USB serial port.

[0110] There is a tube F1 at the back. This is a self - resetting fuse, model - MSMF250 / 16. The maximum voltage withstand of this fuse is 16V, the continuous passing current is up to 2.5A, and the peak current is 5A. The specific parameters are shown in Table 2:

[0111] Table 2 Parameters of MF - MSMF250 / 16

[0112]

[0113] Q3 is a free - wheeling diode, which has the function of preventing reverse connection. BCM857BS is a triode of NXP, and it realizes 5V voltage stabilization together with the D5 transient voltage suppressor diode (TVS tube) SMJ5.0a.

[0114] Table 3 Pins of BCM857BS

[0115]

[0116] The PAM2306AYPKE chip is a DC - DC switching voltage regulator chip, which provides 1.8V and 3.3V voltages for the Raspberry Pi CPU and peripherals here. The typical application circuit is as Figure 13 shown.

[0117] Channel 1 of VIN1 is connected to the 5V power supply input; the EN1 pin is valid for enabling the chip of channel 1 (high level), V EN1 ≤V IN1 ; the pin VIN2 is the power supply input of channel 2; the pin EN2 is valid for enabling the chip of channel 2 (high level), V EN2 ≤V IN2 ; pins 3 and 9 are both grounded for shielding to obtain the maximum power dissipation; the pin LX1 is used to switch the pin of channel 1; the pin FB1 is used as the feedback of channel 1; the pin LX2 is used to switch the pin of channel 2; the pin FB2 is used as the feedback of channel 2; the pins NC1 and NC2 are not connected; the pin PAD is grounded.

[0118] The Raspberry Pi 3b + also provides Power over Ethernet (POE). The schematic diagram of the POE power module is as Figure 14As shown in the figure. Its advantages are that it can transmit data for some IP terminals such as IP phones, wireless LAN access points AP, network cameras, etc., and can provide DC power for these devices. Three aspects should be noted in the application of POE: not all Ethernet switches have the PoE power supply feature, the power supply module can be built-in or external, and it is usually relatively expensive; the terminal also needs to have the ability to be powered by PoE; for the power supply over the network cable, the power supply itself has limitations, so pay attention to observing the operation guides and power requirements of various devices. This function provides an additional power supply method for the Raspberry Pi, and only one network cable is needed without the need to additionally configure a power supply.

[0119] A standard Category 5 network cable has four pairs of twisted pairs, and the pins are the four pairs of 1 and 2, 3 and 6, 7 and 8, 9 and 10, and current is transmitted on them. The IEEE802.3af standard allows two power supply methods with different wire sequences: one method is to transmit current through wires 7, 8, 9, and 10, and it is stipulated that wires 7 and 8 are the positive poles. Another power supply method is to transmit power through wires 1, 2, 3, and 6, and the polarity is relatively flexible. Pins 4 and 5 are the negative poles grounded. 15, 16, 17, and 18 are connected to the positive pole and are connected with two LED display modules to show the working status. DC current is transmitted simultaneously on the data transmission core wires, and the transmission of DC current uses a different frequency from the Ethernet data signal to ensure that it will not interfere with data transmission. The four pins of J14 are all the center taps of the network transformer on the network interface, and at the same time, it can be seen that the output and input ends of the network data are not directly connected most directly, and the data output after conversion by the network transformer.

[0120] Embodiment 2

[0121] The present invention proposes a motion monitoring, evaluation and correction method based on the system of Embodiment 1, including the following steps:

[0122] S1 Turn on the power of the Raspberry Pi, start the Raspberry Pi, configure the network connection settings of the Raspberry Pi so that it can automatically connect to the available network when starting up, and configure the external camera settings of the Raspberry Pi so that it can automatically connect to the camera when starting up.

[0123] S2 Set the program to start automatically when the system boots up, import the motion video on the serial touch screen connected to the Raspberry Pi. This patent innovatively automatically calls different algorithms according to the complexity of the motion postures in the imported video. Set the initial called algorithm to the Blazepose algorithm. When more than half of the key nodes at the 33rd key node are in motion during system monitoring, that is, when the coordinates of more than 16 nodes are offset relative to each other during the motion process, switch the algorithm to Openpose for complex motion posture detection, otherwise call the Blazepose algorithm for simple motion posture recognition.

[0124] S3 When the video motion posture is complex motion (such as tennis, badminton, etc.), the Openpose algorithm is called for posture recognition, and the two-dimensional coordinates of 18 human skeleton points are reconstructed in three dimensions. The coordinates obtained after reconstruction are (258, 178, 2.70), (258, 205, 2.74), (236, 208, 2.83), (194, 212, 2.86), (157, 208, 2.80), (281, 205, 2.78), (318, 207, 2.87), (352, 205, 2.90), (243, 287, 2.79), (247, 340, 2.78), (251, 385, 2.90), (273, 287, 2.73), (277, 336, 2.87), (277, 381, 3.01), (251, 171, 2.69), (262, 171, 2.69), (247, 178, 2.72), (270, 175, 3.18). This process can avoid the problem of blurred human body depth positioning in RGB images, solve the problems of inaccurate posture recognition and missing accuracy. A standardized model is trained through a convolutional neural network (CNN). The posture nodes are compared with the trained standard model, and corresponding real-time scores and modification opinions are given according to the gap between the calculated joint angles and the soft threshold. Assuming the shoulder coordinates are S(Sx, Sy, Sz), the hand coordinates are H(Hx, Hy, Hz), and the elbow coordinates are E(Ex, Ey, Ez), for the calculation of the elbow joint angle, the included angle of the spatial vectors ES and EH can be directly used.

[0125] S4 When the video motion posture is simple motion (such as barbell curl, press, etc.), the Blazepose algorithm is called for posture recognition. The Blazepose model extends the human key nodes to 33, making the posture recognition accuracy more precise. When performing a leg press movement, the Figure 2 nodes 23, 25, 27 in Figure 2 are called. When performing a dumbbell curl movement, the

[0126] nodes 11, 13, 15 in Figure 2 are called. The posture nodes are compared with the stroke endpoints of the repetitive motion to calculate the soft threshold. The present invention gives a threshold weighted calculation, so that the calculated threshold covers the influence of the joint angle on the threshold accuracy judgment, making the given motion stroke progress bar more accurate and realizing the evaluation of the in-place degree of the motion posture.

[0126] S5 When a remote computer needs to connect to the Raspberry Pi, the software VNCviewer can be installed on the computer, and the network-connected IP of the Raspberry Pi is input in VNCviewer to realize the visualization operation of the Raspberry Pi interface and can remotely operate it.

[0127] In the embodiments disclosed in the present application, the computer storage medium may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The computer storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of the computer storage medium would include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0128] Those of ordinary skill in the art will appreciate that the units and algorithm steps of the examples described in connection with the embodiments disclosed in the present application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present application.

[0129] The above are only the preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions falling within the concept of the present invention belong to the protection scope of the present invention. It should be noted that for those of ordinary skill in the art in the technical field, several improvements and refinements made without departing from the principle of the present invention should be regarded as within the protection scope of the present invention.

Claims

1. A motion monitoring, evaluation and correction system based on Openpose and Blazepose algorithms, characterized in that: include: Camera, used to capture motion video; A display screen, used to display the collected motion video in real time, as well as the images of motion posture monitoring and evaluation correction; The communication module is used to upload the collected motion video to the cloud platform and transmit the results of the posture monitoring and evaluation correction of the cloud platform to the display screen; A power module, used to supply power to the camera, display screen and communication module; A cloud platform for posture monitoring, evaluation and correction of moving subjects in motion videos based on Openpose and Blazepose algorithms.

2. The motion monitoring, evaluation and correction system based on Openpose and Blazepose algorithms as claimed in claim 1, characterized in that: The posture monitoring and evaluation correction of the moving subject in the motion video based on the Openpose and Blazepose algorithms are specifically as follows: Initially, the Blazepose algorithm is called to monitor 33 key nodes of the human body; Count the number of relative coordinate offset nodes. If the number exceeds 16, switch the algorithm to Openpose for motion posture monitoring and evaluation correction. Otherwise, continue to use the Blazepose algorithm for motion posture monitoring and evaluation correction.

3. The motion monitoring, evaluation and correction system based on Openpose and Blazepose algorithms as claimed in claim 2, characterized in that: The number of the statistical relative coordinate offset nodes is specifically: Nmoving=count(KwhereKmoving=TRUE) Wherein, Nmoving is the number of nodes for relative coordinate shifting, count(KwhereKmoving=TRUE) is a counting function, which indicates that when the relative coordinate shift of a node is true, it is incremented by one, and KwhereKmoving=TRUE indicates that the relative coordinate shift of a node is true.

4. The motion monitoring, evaluation and correction system based on Openpose and Blazepose algorithms as claimed in claim 2, characterized in that: The switching algorithm is Openpose to perform motion posture monitoring and evaluation correction specifically as follows: The standardized model is trained by convolutional neural network; Compare the posture nodes identified by the Openpose algorithm with the trained standardized model, specifically: The identified posture nodes are formed into vectors, the vector angle is calculated as the joint angle, and the cosine value of the joint angle is compared with the soft threshold; Corresponding real-time scores and modification suggestions are given based on the difference between the cosine value of the joint angle and the soft threshold.

5. The motion monitoring, evaluation and correction system based on Openpose and Blazepose algorithms as claimed in claim 2, characterized in that: The use of the Blazepose algorithm for motion posture monitoring and evaluation correction is specifically as follows: The posture nodes monitored by the Blazepose algorithm are compared with the travel endpoints of repetitive motion to calculate the soft threshold. The interval values ​​less than the soft threshold are set to 0 to indicate that the action is not in place, and the interval values ​​greater than the soft threshold are set to 1 to indicate that the action is in place, thereby realizing posture detection of whether the motion posture is standard.

6. The motion monitoring, evaluation and correction system based on Openpose and Blazepose algorithms as claimed in claim 4, characterized in that: The convolutional neural network includes an input layer, a convolution layer, an excitation layer, a pooling layer, a fully connected layer and an output layer; The convolutional layer is expressed by the formula as follows: Conv(X i ,W k )=ΣjX ij ·W jk +B k Where, X i is the input feature map, W k is the kth convolution kernel, B k is the bias of the kth convolution kernel; j is the number of the sliding window, X ij is the jth region of the feature map scanned by the sliding window when the convolution kernel scans the feature map, W jk It is the kth convolution kernel that scans the jth region of the feature map; The pooling layer is maximum pooling or average pooling, and the maximum pooling is expressed as: MaxPool(Z i )=max(Z i ) In the formula, MaxPool(·) represents the maximum pooling, Z i is the eigenvalue in the pooling window; max(Zi) means taking the maximum value among the eigenvalues. Average pooling is expressed as: Where n is the total number of pooling windows, AvgPool(·) represents average pooling, and Z im Represents the feature value in the mth pooling window.

7. The motion monitoring, evaluation and correction system based on Openpose and Blazepose algorithms as claimed in claim 4, characterized in that: The steps of forming the identified posture nodes into vectors and calculating the vector angle as the joint angle are as follows: The shoulder coordinates are S (Sx, Sy, Sz), the hand coordinates are H (Hx, Hy, Hz), and the elbow coordinates are E (Ex, Ey, Ez). The angle between the space vectors ES and EH is used to obtain the elbow joint angle. The calculation process is as follows: is the vector between the shoulder and the elbow, is the vector between the hand and the elbow, and cosα is the cosine of the elbow joint angle α.

8. The motion monitoring, evaluation and correction system based on Openpose and Blazepose algorithms as claimed in claim 4, characterized in that: The calculation method of the soft threshold is: Where α is the elbow joint angle, T is the set threshold, and soft(α,T) is the soft threshold.

9. The motion monitoring, evaluation and correction system based on Openpose and Blazepose algorithms as claimed in claim 1, characterized in that: The display screen is an HDMI display screen.

10. The motion monitoring, evaluation and correction system based on Openpose and Blazepose algorithms as claimed in claim 1, characterized in that: The power supply mode of the power module is Power over Ethernet.

Images