A method, device and equipment for determining action standard degree

Abstract

The application relates to the information technology field and provides a determination method, device and equipment of action standard degree, wherein a user terminal image and a coach terminal image are acquired; first key point information corresponding to a plurality of key points of a user is acquired according to the user terminal image, and second key point information corresponding to a plurality of key points of a coach is acquired according to the coach terminal image; wherein each key point information comprises stereoscopic coordinate information, angle information and convex hull volume information; a high-dimensional vector of an exercise action corresponding to the user is obtained based on the first key point information of all the key points, and a high-dimensional vector of a teaching action corresponding to the coach is obtained based on the second key point information of all the key points; a similarity characteristic value is obtained based on the high-dimensional vector of the exercise action and the high-dimensional vector of the teaching action; and the similarity characteristic value represents the standard degree of the action of the user in the fitness process. The method can improve the accuracy of the action standard degree evaluation.

Description

Technical Field

[0001] This application relates to the field of information technology, and in particular to a method, apparatus and equipment for determining the standard of motion. Background Technology

[0002] As society continues to develop, people are pursuing a higher quality of life, and AI fitness products have become a popular and well-received fitness trend in recent years.

[0003] AI-powered fitness products are integrated into smartphones, smart TVs, or fitness mirrors. When using these products, users can perform exercises based on fitness tutorial videos played on the device. The AI-powered fitness product captures real-time video of the user through its camera and uses a human skeleton detection algorithm to identify key skeletal points from both the fitness video and the user's video. The product then compares the matching degree between the instructor's skeletal points in the tutorial video and the user's skeletal points to determine the accuracy of the user's movements, significantly enriching people's fitness options.

[0004] However, in related technologies, the method used by fitness products to compare the matching degree between the skeletal points of the instructor and the user in fitness tutorial videos to generate the user's movement standard is relatively simple, resulting in low accuracy of movement standard assessment. Therefore, movement standard assessment suffers from low accuracy. Summary of the Invention

[0005] This application provides a method, apparatus, and device for determining the standard of motion, which can more comprehensively represent motion information and improve the accuracy of motion standard assessment.

[0006] In a first aspect, embodiments of this application provide a method for determining the standard of motion, the method comprising: Acquire user-side images and coach-side images; the user-side images are exercise images of the user during the playback of the fitness video; the coach-side images are teaching images of the coach in the fitness video that correspond to the user-side images. The user's first key point information corresponding to multiple key points is obtained from the user's terminal image, and the coach's second key point information corresponding to multiple key points is obtained from the coach's terminal image; wherein, each key point information includes 3D coordinate information, angle information and convex hull volume information; Based on the first key point information of all the key points, a high-dimensional vector of the practice action corresponding to the user is obtained, and based on the second key point information of all the key points, a high-dimensional vector of the teaching action corresponding to the coach is obtained. Based on the high-dimensional vectors of the practice movements and the teaching movements, similarity feature values ​​are obtained; the similarity feature values ​​characterize the standard of the user's movements during the fitness process.

[0007] A second aspect of this application provides a device for determining the accuracy of a movement, the device comprising: The image acquisition unit is used to acquire user-end images and coach-end images; the user-end images are exercise images of the user during the playback of the fitness video; the coach-end images are teaching images of the coach in the fitness video that correspond to the user-end images. The key point information determination unit is used to obtain first key point information corresponding to multiple preset key points of the user based on the user terminal image, and to obtain second key point information corresponding to multiple key points of the coach based on the coach terminal image; wherein, each key point information includes 3D coordinate information, angle information and convex hull volume information; A high-dimensional vector construction unit is used to obtain a high-dimensional vector of the user's practice movements based on the first key point information of all the key points, and to obtain a high-dimensional vector of the coach's teaching movements based on the second key point information of all the key points. The standardization assessment unit is used to obtain similarity feature values ​​based on the high-dimensional vectors of the practice movements and the high-dimensional vectors of the instruction movements; the similarity feature values ​​characterize the degree of standardization of the user's movements during the fitness process.

[0008] According to a third aspect provided in the embodiments of this application, a computer storage medium is provided, the computer storage medium storing a computer program for performing the method as described in the first aspect.

[0009] According to a fourth aspect provided in the embodiments of this application, an electronic device is provided, including a processor and a memory, wherein the memory stores a computer program that can run on the processor, and when the computer program is executed by the processor, implements the method as described in the first aspect.

[0010] In the above embodiments of this application, user-end images and coach-end images are acquired. The user-end image is a practice image of the user during the playback of a fitness video. The coach-end image is a teaching image of the coach in the fitness video corresponding to the user-end image. First keypoint information corresponding to multiple key points of the user is obtained from the user-end image, and second keypoint information corresponding to multiple key points of the coach is obtained from the coach-end image. Each keypoint information includes 3D coordinate information, angle information, and convex hull volume information. Based on the first keypoint information of all the key points, a high-dimensional vector of the user's corresponding practice movement is obtained, and based on the second keypoint information of all the key points, a high-dimensional vector of the coach's corresponding teaching movement is obtained. Based on the high-dimensional vector of the practice movement and the high-dimensional vector of the teaching movement, a similarity feature value is obtained. The similarity feature value characterizes the standardization of the user's movements during fitness. Therefore, this embodiment can construct a high-dimensional vector of movements and then judge the standardization of movements based on the similarity of the high-dimensional vectors of movements, which can more comprehensively characterize movement information, improve the accuracy of movement standardization, and enhance the accuracy of movement standardization assessment. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 An exemplary flowchart illustrates a method for determining the standard of motion provided in an embodiment of this application; Figure 2 An exemplary schematic diagram illustrates the process of generating first key point information in the method for determining the standard of motion provided in this application embodiment; Figure 3 An exemplary schematic diagram illustrates the process of generating second key point information in the method for determining the standard of motion provided in this application embodiment; Figure 4 An exemplary schematic diagram illustrates the process of constructing a high-dimensional vector of an action using the method for determining the standard of action provided in this application embodiment; Figure 5 An exemplary illustration shows one of the flowcharts illustrating the method for determining the action standard provided in this application, which obtains similarity feature values; Figure 6 The second illustrative flowchart of the method for determining the action standard provided in this application, which obtains similarity feature values, is shown in the example. Figure 7 An exemplary schematic diagram of one of the structural diagrams of the motion standard determination device provided in an embodiment of this application is shown; Figure 8 A second schematic diagram of the structure of the motion standard determination device provided in this application embodiment is shown as an example; Figure 9 A third schematic diagram of the structure of the motion standard determination device provided in this application embodiment is shown as an example; Figure 10 An exemplary hardware structure diagram of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0013] To make the objectives, implementation methods and advantages of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the described exemplary embodiments are only some embodiments of this application, and not all embodiments.

[0014] Based on the exemplary embodiments described in this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the appended claims. Furthermore, although the disclosures in this application are presented by way of one or more exemplary examples, it should be understood that each aspect of these disclosures can also constitute a complete implementation on its own.

[0015] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0016] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to be omnipresent but not exclusive; for example, a product or device comprising a series of components is not necessarily limited to those explicitly listed, but may include other components not explicitly listed or inherent to such product or device.

[0017] As used in this application, the term "module" means any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code capable of performing the functions associated with that element.

[0018] In this application's embodiments, the term "terminal device" refers to a device capable of installing various applications, including built-in applications and third-party applications, and capable of displaying objects provided by the installed applications. This terminal device can be mobile or fixed. Examples include mobile phones, tablets, various wearable devices, in-vehicle devices, personal digital assistants (PDAs), point-of-sale (POS) terminals, or other terminal devices capable of performing the aforementioned functions.

[0019] As used in this application, the term "convex hull" refers to a minimal convex polyhedron corresponding to a point set Q, such that a point in Q is either on or inside a vertex of the polyhedron. In the embodiments of this application, the point set Q is composed of multiple skeletal points.

[0020] The following is an overview of the ideas behind the embodiments of this application.

[0021] As society continues to develop, people are pursuing a higher quality of life, and AI fitness products have become a popular and well-received fitness trend in recent years.

[0022] AI-powered fitness products are integrated into smartphones, smart TVs, or fitness mirrors. When using these products, users can perform exercises based on fitness tutorial videos played on the device. The AI-powered fitness product captures real-time video of the user through its camera and uses a human skeleton detection algorithm to identify key skeletal points from both the fitness video and the user's video. The product then compares the matching degree between the instructor's skeletal points in the tutorial video and the user's skeletal points to determine the accuracy of the user's movements, significantly enriching people's fitness options.

[0023] However, in related technologies, the method used by fitness products to compare the matching degree between the skeletal points of the instructor and the user in fitness tutorial videos to generate the user's movement standard is relatively simple, resulting in low accuracy of movement standard assessment. Therefore, movement standard assessment suffers from low accuracy.

[0024] To address the problems existing in the prior art, this application provides a method for determining the standard of movement. This method involves acquiring user-side images and coach-side images. The user-side image is a practice image of the user during a fitness video playback; the coach-side image is a teaching image of the coach in the fitness video corresponding to the user-side image. First keypoint information corresponding to multiple key points of the user is obtained from the user-side image, and second keypoint information corresponding to multiple key points of the coach is obtained from the coach-side image. Each keypoint information includes 3D coordinate information, angle information, and convex hull volume information. Based on the first keypoint information of all key points, a high-dimensional vector of the user's corresponding practice movement is obtained, and based on the second keypoint information of all key points, a high-dimensional vector of the coach's corresponding teaching movement is obtained. A similarity feature value is obtained based on the high-dimensional vectors of the practice movement and the teaching movement. The similarity feature value characterizes the standard of the user's movements during fitness. Therefore, this embodiment can construct a high-dimensional vector of movements and then judge the standard of movement based on the similarity of the high-dimensional vectors, which can more comprehensively represent movement information, improve the accuracy of movement standardization, and enhance the accuracy of movement standardization assessment. The embodiments of this application are described in detail below with reference to the accompanying drawings.

[0025] Figure 1 An exemplary embodiment of this application illustrates a method for determining the standard of motion, such as... Figure 1 As shown, the following steps may be included: S101, acquire user-side image and coach-side image; user-side image is the exercise image of the user during the playback of the fitness video; coach-side image is the teaching image of the coach in the fitness video corresponding to the user-side image.

[0026] In practice, when a user uses an AI fitness product to play a fitness video and practice, the system acquires the user's real-time video through the camera, thereby obtaining the user's image. At the same time, it also acquires the coach's image from the fitness video. The coach's image is the teaching image of the coach in the fitness video that corresponds to the user's image acquired by the camera.

[0027] For example, the user-side image PIC_u and the coach-side image PIC_t are obtained; the user-side image PIC_u is the exercise image of the user during the playback of the fitness video; the coach-side image PIC_t is the teaching image of the coach in the fitness video corresponding to the user-side image PIC_u.

[0028] S102, obtain the first key point information corresponding to multiple key points of the user based on the user terminal image, and obtain the second key point information corresponding to multiple key points of the coach based on the coach terminal image; wherein, each key point information includes 3D coordinate information, angle information and convex hull volume information.

[0029] For example, the first key point information Key_Info_1 corresponding to multiple key points of the user is obtained from the user-side image PIC_u, and the second key point information Key_Info_2 corresponding to multiple key points of the coach is obtained from the coach-side image PIC_t; wherein, the first key point information Key_Info_1 and the second key point information Key_Info_2 both include 3D coordinate information Place_Inf, angle information Angle_Inf, and convex hull volume information convex_Inf.

[0030] In some embodiments of this application, in S102, the process of obtaining the first key point information based on the user-end image can be achieved through... Figure 2 The steps shown demonstrate how to obtain the second keypoint information from the coach's image. Figure 3 The steps shown are implemented. Among them, The process of obtaining the first key point information corresponding to multiple key points of a user based on the user's image, such as... Figure 2 As shown, it can be specifically described as follows: S201, extract human body key points from user terminal images to obtain user skeletal points corresponding to multiple key points of the user.

[0031] In some embodiments of this application, multiple key points can be 24 sequential skeletal points. These 24 sequential skeletal points can be obtained by extracting the three-dimensional spatial positions of 14 parts, including the top of the head, tip of the nose, eyes, ears, jaw, neck, shoulders, elbows, wrists, hips, knees, ankles, first toes, and fifth toes, from a coach image or user image using existing algorithms.

[0032] In the following embodiments of this application, the multiple key points are specifically 24 sequential skeletal points, including the top of the head, tip of the nose, left eye, right eye, left ear, right ear, jaw, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, right ankle, left first toe, right first toe, left fifth toe, and right fifth toe.

[0033] S202 generates the user's first keypoint information based on all user skeletal points.

[0034] The process of obtaining second keypoint information corresponding to multiple keypoints of the coach based on the coach's image, such as... Figure 3 As shown, it can be specifically described as follows: S301, extract human body key points from the coach's image to obtain the coach skeletal points corresponding to the coach's multiple key points.

[0035] S302 generates the coach's second keypoint information based on all coach skeletal points.

[0036] In some embodiments of this application, the first key point information includes first stereo coordinate information, first angle information, and first convex hull volume information; the second key point information includes second stereo coordinate information, second angle information, and second convex hull volume information; each stereo coordinate information includes a horizontal coordinate component vector, a vertical coordinate component vector, and a depth coordinate component vector; each angle information includes a head component vector, a limb component vector, and a trunk component vector; each convex hull volume information includes a head convex hull component vector, a limb convex hull component vector, a limb-trunk convex hull component vector, and a whole-body convex hull component vector.

[0037] For example, both the first key point information Key_Info_1 and the second key point information Key_Info_2 include 3D coordinate information Place_Inf, angle information Angle_Inf, and convex hull volume information convex_Inf. Specifically, the first key point information Key_Info_1 may include first 3D coordinate information Place_Inf_1, first angle information Angle_Inf_1, and first convex hull volume information convex_Inf_1. The second key information Key_Info_2 includes the second stereo coordinate information Place_Inf_2, the second angle information Angle_Inf_2, and the second convex hull volume information convex_Inf_2; each stereo coordinate information Place_Inf includes the horizontal coordinate vector P1, the vertical coordinate vector P2, and the depth coordinate vector P3; each angle information Angle_Inf includes the head vector A1, the limb vector A2, and the trunk vector A3; each convex hull volume information convex_Inf includes the head convex hull vector C1, the limb convex hull vector C2, the limb and trunk convex hull vector C3, and the whole body convex hull vector C4.

[0038] S103: Based on the first key point information of all key points, obtain the high-dimensional vector of the user's corresponding practice action, and based on the second key point information of all key points, obtain the high-dimensional vector of the coach's corresponding teaching action.

[0039] For example, based on the first key point information Key_Info_1 corresponding to all key points of user User, the high-dimensional vector V_1 of the practice action corresponding to user User is obtained, and based on the second key point information Key_Info_2 corresponding to all key points of coach coach, the high-dimensional vector V_2 of the teaching action corresponding to coach coach is obtained.

[0040] In some embodiments of this application, a high-dimensional vector of the user's practice movements is obtained based on the first key point information of all key points, and a high-dimensional vector of the coach's teaching movements is obtained based on the second key point information of all key points, such as... Figure 4 As shown, this can be achieved through the following steps: S401: Based on the x-coordinate, y-coordinate, and depth coordinates of all the user's key points, construct a row vector to obtain the first 3D coordinate information; and based on the x-coordinate, y-coordinate, and depth coordinates of all the coach's key points, construct a row vector to obtain the second 3D coordinate information.

[0041] For example, based on the x-coordinate, y-coordinate, and depth coordinates of the 24 sequential skeletal points of the user, a row vector is constructed to obtain the first 3D coordinate information Place_Inf_1, and based on the x-coordinate, y-coordinate, and depth coordinates of the 24 sequential skeletal points of the coach, a row vector is constructed to obtain the second 3D coordinate information Place_Inf_2.

[0042] In some embodiments of this application, the process of constructing row vectors based on the x-coordinates, y-coordinates, and depth coordinates of all key points of the user to obtain the first stereo coordinate information in step S401 can be specifically described as follows: constructing row vectors from the x-coordinates of all key points of the user in a preset first order to obtain a first x-coordinate component vector; constructing row vectors from the y-coordinates of all key points of the user in a preset first order to obtain a first y-coordinate component vector; constructing row vectors from the depth coordinates of all key points of the user in a preset first order to obtain a first depth coordinate component vector; sequentially concatenating the first x-coordinate component vector, the first y-coordinate component vector, and the first depth coordinate component vector in a preset second order to construct a row vector, and performing normalization processing to obtain the first stereo coordinate information. Similarly, the process in step S401 of constructing row vectors based on the x-coordinates, y-coordinates, and depth coordinates of all the coach's key points to obtain the second 3D coordinate information can be specifically described as follows: constructing row vectors from the x-coordinates of all the coach's key points in a preset first order to obtain the second x-coordinate component vector; constructing row vectors from the y-coordinates of all the coach's key points in a preset first order to obtain the second y-coordinate component vector; constructing row vectors from the depth coordinates of all the coach's key points in a preset first order to obtain the second depth coordinate component vector; and sequentially concatenating the second x-coordinate component vector, the second y-coordinate component vector, and the second depth coordinate component vector in a preset second order to construct a row vector, and performing normalization processing to obtain the second 3D coordinate information.

[0043] In the embodiments of this application, the first stereo coordinate information is the stereo coordinate information corresponding to the user, and the second stereo coordinate information is the stereo coordinate information corresponding to the coach. The process of constructing a row vector based on the x-coordinate, y-coordinate, and depth coordinates of all the user's keypoints to obtain the first stereo coordinate information is similar to the process of constructing a row vector based on the x-coordinate, y-coordinate, and depth coordinates of all the coach's keypoints to obtain the second stereo coordinate information. For simplicity, the specific implementation process described below will not specifically specify the first or second stereo coordinate information, but will use stereo coordinate information as an example. In the following description of the specific implementation process, the stereo coordinate information can be either the first or the second stereo coordinate information.

[0044] In practice, the three-dimensional coordinate information can be the three-dimensional spatial coordinate component vector P of the skeleton point. Specifically, the three-dimensional spatial coordinate component vector P of the skeleton point consists of three parts: the horizontal coordinate component P1, the vertical coordinate component P2, and the vertical coordinate component P3, that is, P = (P1, P2, P3).

[0045] Here, P1 is a row vector composed of the abscissas of the three-dimensional spatial positions of 24 skeletal points (referred to as 24 sequential skeletal points), including the top of the head, tip of the nose, left eye, right eye, left ear, right ear, jaw, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, right ankle, left first toe, right first toe, left fifth toe, and right fifth toe; P2 is a row vector composed of the ordinates of the three-dimensional spatial positions of the 24 sequential skeletal points; and P3 is a row vector composed of the depth coordinates of the three-dimensional spatial positions of the 24 sequential skeletal points. The origin of the coordinate system containing the abscissa, ordinate, and depth coordinates is the centroid of the 24 sequential skeletal points. P1, P2, and P3 are all 24-dimensional row vectors, and P is a 72-dimensional row vector.

[0046] In some embodiments, the 3D coordinate information can also be normalized. Specifically, using the largest absolute value among the horizontal, vertical, and depth coordinates as a measure, all coordinate values ​​in the 3D spatial coordinate vector P of the skeleton point are normalized, that is, the largest absolute value is scaled to 1 by a scaling factor t, and all other coordinate values ​​are scaled by a scaling factor t.

[0047] S402, construct row vectors based on the head feature angles, limb feature angles, and trunk feature angles corresponding to all key points of the user to obtain the first angle information, and construct row vectors based on the head feature angles, limb feature angles, and trunk feature angles corresponding to all key points of the coach to obtain the second angle information.

[0048] In some embodiments of this application, the head feature angles include four angles: left ear-nose tip-neck, right ear-nose tip-neck, jaw-neck-left shoulder, and jaw-neck-right shoulder; the limb feature angles include four angles: left shoulder-left elbow-left wrist, right shoulder-right elbow-right wrist, left hip-left knee-left ankle, and right hip-right knee-right ankle; the trunk feature angles include four angles: neck-left shoulder-left elbow, neck-right shoulder-right elbow, left elbow-left shoulder-left hip, right elbow-right shoulder-right hip, left shoulder-left hip-left knee, right shoulder-right hip-right knee, right hip-left hip-left knee, and left hip-right hip. -12 angles: right knee, left knee-left ankle-left first toe, right knee-right ankle-right first toe, left knee-left ankle-left fifth toe, right knee-right ankle-right fifth toe; The first angle information is a 20-dimensional row vector constructed from the angles included in the user's head features, the angles included in the limb features, and the angles included in the trunk features; The second angle information is a 20-dimensional row vector constructed from the angles included in the coach's head features, the angles included in the limb features, and the angles included in the trunk features.

[0049] In practice, the angle information can be the skeletal point angle vector A. Specifically, the skeletal point angle vector A consists of three parts: head vector A1, limb vector A2, and trunk vector A3, i.e., A = (A1, A2, A3).

[0050] Where A1 is a row vector consisting of four head feature angles: left ear-nose tip-neck (i.e., the angle formed by the spatial points of the left ear, nose tip, and neck, with the vertex at the nose tip, the same below), right ear-nose tip-neck, jaw-neck-left shoulder, and jaw-neck-right shoulder; A2 is a row vector consisting of four limb feature angles: left shoulder-left elbow-left wrist, right shoulder-right elbow-right wrist, left hip-left knee-left ankle, and right hip-right knee-right ankle. A3 is a row vector composed of 12 torso feature angles: neck-left shoulder-left elbow, neck-right shoulder-right elbow, left elbow-left shoulder-left hip, right elbow-right shoulder-right hip, left shoulder-left hip-left knee, right shoulder-right hip-right knee, right hip-left hip-left knee, left hip-right hip-right knee, left knee-left ankle-left first toe, right knee-right ankle-right first toe, left knee-left ankle-left fifth toe, and right knee-right ankle-right fifth toe. A1, A2, and A3 are 4-dimensional, 4-dimensional, and 12-dimensional row vectors, respectively, and A is a 20-dimensional row vector.

[0051] In some embodiments, the skeletal point angle vector A can also be normalized. Specifically, the largest angle value among the 20 angles in the skeletal point angle vector A is used as the metric, and all angle values ​​in the skeletal point angle vector A are normalized. That is, the largest angle value (not exceeding π) is scaled to 1 by a scaling factor u, and all other angle values ​​are scaled by a scaling factor u.

[0052] S403, construct row vectors based on the feature convex hull volumes corresponding to all key points of the user to obtain the first convex hull volume information, and construct row vectors based on the feature convex hull volumes corresponding to all key points of the coach to obtain the second convex hull volume information.

[0053] In some embodiments of this application, the key points are 24 sequential skeletal points; the characteristic convex hull volume includes the head convex hull volume, the limb convex hull volume, the mixed convex hull volume, and the whole body convex hull volume; the head convex hull volume is twice the volume of the convex hull formed by the eight points: the top of the head, the tip of the nose, the left eye, the right eye, the left ear, the right ear, the jaw, and the neck; the limb convex hull volume is the volume corresponding to the convex hull space formed by the two arms, the convex hull space formed by the two legs, the convex hull space formed by the four limbs, the convex hull space formed by the left arm and the left leg, and the convex hull space formed by the right arm and the right leg; the mixed convex hull volume is the volume corresponding to the convex hull space formed by the left arm and the torso, the convex hull space formed by the right arm and the torso, and the convex hull formed by the left leg and the torso. The volume corresponding to the convex hull space formed by the right leg and torso, and the convex hull space formed by the limbs and torso; the full-body convex hull volume is the volume of the convex hull space formed by all skeletal points; the first convex hull volume information is constructed from the volume included in the user's head convex hull volume, the volume included in the limb convex hull volume, the volume included in the mixed convex hull volume, and the volume included in the full-body convex hull volume, forming a row vector, which is then normalized to obtain a 12-dimensional row vector; the first convex hull volume information is constructed from the volume included in the coach's head convex hull volume, the volume included in the limb convex hull volume, the volume included in the mixed convex hull volume, and the volume included in the full-body convex hull volume, forming a row vector, which is then normalized to obtain a 12-dimensional row vector.

[0054] In practice, the convex hull volume information can be the three-dimensional convex hull component vector C of the skeletal points. Specifically, the three-dimensional convex hull component vector C of the skeletal points consists of three parts: the head convex hull component vector C1, the limb convex hull component vector C2, the mixed convex hull component vector C3, and the whole body convex hull component vector C4, that is, C = (C1, C2, C3, C4).

[0055] Where C1 is a 1-dimensional row vector formed by the volume of the head convex hull, and the volume of the head convex hull is twice the volume of the convex hull formed by the eight points: top of the head, tip of the nose, left eye, right eye, left ear, right ear, jaw, and neck. C2 is a 5-dimensional row vector formed by the convex hull volumes of the four limbs, which are row vectors formed by the volumes of the convex hull spaces formed by the two arms, the two legs, the four limbs, the left arm and left leg, and the right arm and right leg. C3 is a 5-dimensional row vector composed of the mixed convex hull volumes, which are row vectors composed of the volumes of the convex hull spaces formed by the left arm and torso, the right arm and torso, the left leg and torso, the right leg and torso, and the limbs and torso. C4 is a 1-dimensional row vector formed by the volume of the whole body convex hull, which is the volume of the convex hull space formed by all bone points.

[0056] In the embodiments of this application, the skeletal points of the left arm are the left shoulder, left elbow, and left wrist; the skeletal points of the right arm are the right shoulder, right elbow, and right wrist; the skeletal points of the left leg are the left hip, left knee, left ankle, left first toe, and left fifth toe; the skeletal points of the right leg are the right hip, right knee, right ankle, right first toe, and right fifth toe; the skeletal points of the torso are the neck, left shoulder, right shoulder, left hip, and right hip; the two arms include the skeletal points of the left and right arms; the two legs include the skeletal points of the left and right legs; the limbs include the skeletal points of the two arms and two legs; and the whole body includes the aforementioned left and right skeletal points.

[0057] C1, C2, C3, and C4 are 1-dimensional, 5-dimensional, 5-dimensional, and 1-dimensional row vectors, respectively, and C is a 12-dimensional row vector.

[0058] In some embodiments, the 3D convex hull component vector C of the skeleton points can also be normalized. Specifically, the 3D convex hull component vector C of the skeleton points includes the largest convex hull volume value among the above 12 convex hull volumes. All convex hull volume values ​​included in the 3D convex hull component vector C of the skeleton points are normalized, that is, the largest convex hull volume value is scaled to 1 by a scaling factor v, and all other convex hull volume values ​​are scaled by a scaling factor v.

[0059] In the embodiments of this application, the convex hull generation method adopts a common method already available in the prior art, which will not be described in detail here.

[0060] S404. Based on the first solid coordinate information, the first angle information, and the first convex hull volume information, construct row vectors to obtain the high-dimensional vector of the user's practice actions; and based on the second solid coordinate information, the second angle information, and the second convex hull volume information, construct row vectors to obtain the high-dimensional vector of the coach's corresponding teaching actions.

[0061] In specific implementation, the high-dimensional vector of the action in the embodiments of this application can be a high-dimensional vector V. The high-dimensional vector V consists of three parts: the three-dimensional coordinate component vector P of the skeleton point, the angle component vector A of the skeleton point, and the three-dimensional convex hull component vector C of the skeleton point. Specifically, V = (P, A, C).

[0062] The constructed high-dimensional vector V = (P, A, C) = (P1, P2, P3, A1, A2, A3, C1, C2, C3, C4) is a 104-dimensional vector containing three parts: the three-dimensional spatial coordinate vector P of the skeleton point, the angle vector A of the skeleton point, and the three-dimensional convex hull vector C of the skeleton point.

[0063] S104. Based on the high-dimensional vectors of practice movements and instructional movements, similarity feature values ​​are obtained; the similarity feature values ​​characterize the standard of the user's movements during the fitness process.

[0064] In practice, the numerical value of the similarity feature is in the range of [0,1]. The larger the value, the higher the standard, and vice versa.

[0065] For example, based on the high-dimensional vector V_1 of the practice movements and the high-dimensional vector V_2 of the teaching movements, the similarity feature value D is obtained; the similarity feature value D represents the degree of standard of the user's movements during the fitness process.

[0066] In some embodiments of this application, step S104 involves obtaining similarity feature values ​​based on the high-dimensional vectors of practice actions and teaching actions, such as... Figure 5 As shown, this can be achieved through the following steps: S501, calculate the cosine similarity between the high-dimensional vectors of the practice movements and the high-dimensional vectors of the teaching movements to obtain the first similarity value.

[0067] In practice, the formula for calculating cosine similarity can be as follows:

[0068] Where A represents the high-dimensional vector of the user's corresponding practice action; B represents the high-dimensional vector of the coach's corresponding teaching action.

[0069] For example, the cosine similarity between the high-dimensional vector V_1 of the practice action and the high-dimensional vector V_2 of the teaching action is calculated to obtain the first similarity value CS_1.

[0070] S502, Based on the first similarity value, obtain the similarity feature value.

[0071] In specific implementation, the first similarity value CS_1 ranges from [-1, 1]. The value of the first similarity value CS_1 is then shifted and scaled, i.e., D_1 = (CS_1 + 1) / 2 The obtained value of D_1 is in the range of [0, 1], and is used as the similarity feature value D.

[0072] In some embodiments of this application, a gain coefficient can be added to the vector element in the 104-dimensional vector corresponding to the skeletal points of the preset body parts to increase the weight of the element in the vector, thereby generating information that measures the standard of user actions more accurately.

[0073] In some embodiments of this application, step S104 involves obtaining similarity feature values ​​based on the high-dimensional vectors of practice actions and teaching actions, such as... Figure 6 As shown, it can also be achieved through the following steps: S601, based on the high-dimensional vector of the practice action and the preset vector gain coefficient, obtain the high-dimensional vector of the first target.

[0074] In practical implementation, the three-dimensional coordinate information is a 3D spatial coordinate vector P of the skeletal points. This vector P consists of three parts: an abscissa component P1, an ordinate component P2, and a ordinate component P3, i.e., P = (P1, P2, P3). The 3D spatial coordinate vector P can be divided into multiple groups based on the different skeletal points of the corresponding preset body parts; that is, the abscissa component P1, the ordinate component P2, and the ordinate component P3 are each divided into multiple groups based on the different skeletal points of the corresponding preset body parts. For example, taking the horizontal coordinate component P1 as an example, the horizontal coordinate component P1 can include the head horizontal coordinate component P1_1 and the limb horizontal coordinate component P1_2, that is, P1 = (P1_1, P1_2); the head horizontal coordinate component P1_1 is a row vector composed of the horizontal coordinates of the three-dimensional spatial positions of 8 skeletal points, namely the top of the head, the tip of the nose, the left eye, the right eye, the left ear, the right ear, the jaw, and the neck; the limb horizontal coordinate component P1_2 is a row vector composed of the horizontal coordinates of the three-dimensional spatial positions of 16 skeletal points, namely the left shoulder, the right shoulder, the left elbow, the right elbow, the left wrist, the right wrist, the left hip, the right hip, the left knee, the right knee, the left ankle, the right ankle, the first toe of the left foot, the first toe of the right foot, the fifth toe of the left foot, and the fifth toe of the right foot. The division of the vertical coordinate component P2 and the vertical coordinate component P3 can be similar to the division of the horizontal coordinate component P1, and the similarities will not be repeated.

[0075] For the 3D spatial coordinate component vector P of the skeletal points, the horizontal, vertical, and triangular coordinates of the limb skeletal points are multiplied by gain coefficients respectively; for the angular component vector A of the skeletal points, the angular quantities of the skeletal points associated with the limbs are multiplied by gain coefficients respectively; for the 3D convex hull component vector C of the skeletal points, the 3D convex hull component vectors of the skeletal points directly related to the torso are multiplied by gain coefficients respectively. The gain coefficients for each element in the vector are different and determined according to their importance. The value of the gain coefficient is greater than 1 and generally not greater than 10.

[0076] For example, for each vector element in the 3D convex hull subvector C of the skeleton points, the gain coefficient is in the form of...

[0077] C = (λ1×x1, λ2×x2, λ3×x3,…, λ12×x12)

[0078] Where λ1, λ2, λ3, ..., λ12 are gain coefficients. Depending on the weights of the vector elements in the convex hull, the values ​​are λ1 = 1, λ2 = 1, ..., λ6 = 1. The remaining gain coefficients are all greater than 1, i.e., λ7 > 1, λ8 > 1, ..., λ12 > 1.

[0079] It should be noted that the vector elements x7, x8, ..., x12 mentioned above all contain limb-related factors and are the three-dimensional convex hulls of skeletal points directly related to the torso, so their gain coefficients are greater than 1.

[0080] For example, the first target high-dimensional vector V_u is obtained based on the high-dimensional vector V_1 of the practice action and the preset vector gain coefficient λ.

[0081] S602, based on the high-dimensional vector of the teaching action and the vector gain coefficient, the high-dimensional vector of the second target is obtained.

[0082] The specific process of obtaining the high-dimensional vector of the second target based on the high-dimensional vector of the teaching action and the vector gain coefficient is similar to the process of obtaining the high-dimensional vector of the first target in step S601, and the similarities will not be repeated.

[0083] For example, the second target high-dimensional vector V_t is obtained based on the high-dimensional vector V_2 of the practice action and the preset vector gain coefficient λ.

[0084] S603, calculate the cosine similarity between the high-dimensional vector of the first target and the high-dimensional vector of the second target to obtain the second similarity value.

[0085] For example, the cosine similarity between the first target high-dimensional vector V_u and the second target high-dimensional vector V_t is calculated to obtain the second similarity value CS_1.

[0086] S604. Based on the second similarity value, the similarity feature value is obtained.

[0087] In practice, the second similarity value CS_2 ranges from [-1, 1]. The value of the second similarity value CS_2 is then shifted and scaled, i.e., D_2 = (CS_2 + 1) / 2 The obtained value of D_2 is in the range of [0, 1], and is used as the similarity feature value D.

[0088] In the above process, by constructing high-dimensional action vectors and then judging the standardization of actions based on the similarity of these high-dimensional action vectors, action information can be represented more comprehensively, the accuracy of action standardization can be improved, and the accuracy of action standardization assessment can be enhanced.

[0089] Based on the same inventive concept, the method for determining the accuracy of a movement as disclosed above can also be implemented by a device for determining the accuracy of a movement. The effect of this device is similar to that of the aforementioned method, and will not be described in detail here.

[0090] Figure 7 This is a schematic diagram of the structure of a motion standard determination device according to an embodiment of this application.

[0091] like Figure 7 As shown, the motion standard determination device 700 of this application may include an image acquisition unit 710, a key point information determination unit 720, a high-dimensional vector construction unit 730, and a standard evaluation unit 740.

[0092] The image acquisition unit 710 is used to acquire user-end images and coach-end images; the user-end images are the exercise images of the user during the playback of the fitness video; the coach-end images are the teaching images of the coach in the fitness video that correspond to the user-end images. The key point information determination unit 720 is used to obtain the first key point information of the user corresponding to multiple preset key points based on the user terminal image, and to obtain the second key point information corresponding to multiple key points of the coach based on the coach terminal image; wherein, each key point information includes 3D coordinate information, angle information and convex hull volume information. The high-dimensional vector construction unit 730 is used to obtain the high-dimensional vector of the user's practice action based on the first key point information of all key points, and to obtain the high-dimensional vector of the coach's teaching action based on the second key point information of all key points. The standardization assessment unit 740 is used to obtain similarity feature values ​​based on the high-dimensional vectors of practice movements and instructional movements; the similarity feature values ​​characterize the degree of standardization of the user's movements during fitness.

[0093] In one embodiment, such as Figure 8 As shown, the key point information determination unit 720 includes a first information determination unit 810 and a second information determination unit 820.

[0094] The first information determining unit 810 is specifically used for: Extract human body key points from user-side images to obtain the user's skeletal points corresponding to multiple key points; Based on all user skeleton points, generate the user's first key point information; The second information determination unit 820 is specifically used for: Human body key points are extracted from the coach's image to obtain the coach's skeletal points corresponding to the multiple key points of the coach. Based on all coach skeletal points, generate the coach's second key point information.

[0095] In one embodiment, the first key point information includes first stereo coordinate information, first angle information, and first convex hull volume information; the second key point information includes second stereo coordinate information, second angle information, and second convex hull volume information; each stereo coordinate information includes a horizontal coordinate component vector, a vertical coordinate component vector, and a depth coordinate component vector; each angle information includes a head component vector, a limb component vector, and a trunk component vector; each convex hull volume information includes a head convex hull component vector, a limb convex hull component vector, a limb-trunk convex hull component vector, and a whole-body convex hull component vector.

[0096] In one embodiment, the high-dimensional vector construction unit 730 is specifically used for: Based on the x-coordinate, y-coordinate, and depth coordinates of all the user's key points, a row vector is constructed to obtain the first three-dimensional coordinate information. Based on the x-coordinate, y-coordinate, and depth coordinates of all the coach's key points, a row vector is constructed to obtain the second three-dimensional coordinate information. Based on the head feature angles, limb feature angles, and trunk feature angles corresponding to all key points of the user, a row vector is constructed to obtain the first angle information; and based on the head feature angles, limb feature angles, and trunk feature angles corresponding to all key points of the coach, a row vector is constructed to obtain the second angle information. Based on the feature convex hull volume corresponding to all key points of the user, construct row vectors to obtain the first convex hull volume information; and based on the feature convex hull volume corresponding to all key points of the coach, construct row vectors to obtain the second convex hull volume information. Based on the first 3D coordinate information, the first angle information, and the first convex hull volume information, row vectors are constructed to obtain the high-dimensional vector of the user's practice actions. Based on the second 3D coordinate information, the second angle information, and the second convex hull volume information, row vectors are constructed to obtain the high-dimensional vector of the coach's corresponding teaching actions.

[0097] In one embodiment, such as Figure 9 As shown, the high-dimensional vector construction unit 730 includes a first coordinate information unit 910 and a second coordinate information unit 920.

[0098] The first coordinate information unit 910 is specifically used for: The x-coordinates of all the user's key points are arranged into a row vector according to a preset first order to obtain the first x-coordinate sub-vector; The y-coordinates of all the user's key points are arranged into a row vector according to a preset first order, thus obtaining the first y-coordinate sub-vector; The depth coordinates of all key points of the user are arranged into a row vector according to a preset first order to obtain the first depth coordinate sub-vector; The first horizontal coordinate vector, the first vertical coordinate vector, and the first depth coordinate vector are sequentially concatenated according to a preset second order to construct a row vector, and then normalized to obtain the first three-dimensional coordinate information. The second coordinate information unit 920 is specifically used for: The horizontal coordinates of all the coach's key points are arranged into a row vector according to a preset first order, and the second horizontal coordinate component vector is obtained. The y-coordinates of all the coach's key points are arranged into a row vector according to a preset first order, and the second y-coordinate component vector is obtained. The depth coordinates of all the coach's key points are arranged into a row vector according to a preset first order, and the second depth coordinate component vector is obtained. The second horizontal coordinate vector, the second vertical coordinate vector, and the second depth coordinate vector are sequentially concatenated according to a preset second order to construct a row vector, and then normalized to obtain the second three-dimensional coordinate information.

[0099] In one embodiment, the standardization evaluation unit 740 is specifically used for: Calculate the cosine similarity between the high-dimensional vectors of practice movements and the high-dimensional vectors of teaching movements to obtain the first similarity value; Based on the first similarity value, the similarity feature value is obtained.

[0100] In another embodiment, the standardization evaluation unit 740 is specifically used for: Based on the high-dimensional vector of the practice action and the preset vector gain coefficient, the high-dimensional vector of the first target is obtained; Based on the high-dimensional vector of the teaching action and the vector gain coefficient, the high-dimensional vector of the second target is obtained; Calculate the cosine similarity between the high-dimensional vector of the first target and the high-dimensional vector of the second target to obtain the second similarity value; Based on the second similarity value, the similarity feature value is obtained.

[0101] After introducing the method and apparatus for determining the action standard of exemplary embodiments of the present invention, the electronic device according to another exemplary embodiment of the present invention will be introduced next.

[0102] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: entirely in hardware, entirely in software (including firmware, microcode, etc.), or in a combination of hardware and software, collectively referred to herein as “circuit,” “module,” or “system.”

[0103] In some possible implementations, the electronic device according to the present invention may include at least one processor and at least one memory. The memory stores a computer program executable on the processor, which, when executed by the processor, causes the processor to perform the steps in the method for determining the standard of motion according to various exemplary embodiments of the present invention described above. For example, the processor may perform actions such as... Figure 1 Steps S101-S104 are shown in the figure.

[0104] The following reference Figure 10 To describe an electronic device 1000 according to this embodiment of the present invention. Figure 10 The electronic device 1000 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0105] like Figure 10 As shown, the electronic device 1000 is manifested in the form of a general-purpose electronic device. The components of the electronic device 1000 may include, but are not limited to: at least one processor 1001, at least one memory 1002, a communication interface 1003, and a bus 1004. The processor 1001, the memory 1002, and the communication interface 1003 are interconnected via the bus 1004.

[0106] Processor 1001 is configured to read instructions from memory 1002 and execute them, so that at least one processor is capable of performing the method of any of the above embodiments.

[0107] The memory 1002 is used to store various instructions and programs for determining the standard of action provided in the above embodiments.

[0108] Bus 1004 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 10 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0109] Processor 1001 can be a central processing unit (CPU), a network processor (NP), a graphics processing unit (GPU), or any combination of CPU, NP, and GPU. It can also be a hardware chip. The aforementioned hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0110] This application also provides a computer-readable storage medium for storing instructions that, when executed, can perform the methods described in the foregoing embodiments.

[0111] This application also provides a computer program product for storing a computer program that performs the methods described in the foregoing embodiments.

[0112] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0113] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0114] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0115] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0116] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for determining the standard of motion, characterized in that, The method includes: Acquire user-side images and coach-side images; the user-side images are exercise images of the user during the playback of the fitness video; the coach-side images are teaching images of the coach in the fitness video that correspond to the user-side images. The system obtains first keypoint information corresponding to multiple keypoints of the user based on the user-side image, and obtains second keypoint information corresponding to multiple keypoints of the coach based on the coach-side image; wherein, each keypoint information includes 3D coordinate information, angle information, and convex hull volume information; each 3D coordinate information includes a horizontal coordinate vector, a vertical coordinate vector, and a depth coordinate vector; each angle information includes a head vector, a limb vector, and a trunk vector; each convex hull volume information includes a head convex hull vector, a limb convex hull vector, a limb-trunk convex hull vector, and a full-body convex hull vector; Based on the first key point information of all the key points, a high-dimensional vector of the practice action corresponding to the user is obtained, and based on the second key point information of all the key points, a high-dimensional vector of the teaching action corresponding to the coach is obtained. Based on the high-dimensional vectors of the practice movements and the teaching movements, similarity feature values ​​are obtained; the similarity feature values ​​characterize the standard of the user's movements during the fitness process.

2. The method according to claim 1, characterized in that, The step of obtaining the first key point information corresponding to multiple key points of the user based on the user terminal image includes: Human body key points are extracted from the user terminal image to obtain the user skeletal points corresponding to the multiple key points of the user. Based on all the user's skeletal points, generate the user's first keypoint information; The step of obtaining the second key point information corresponding to multiple key points of the coach based on the coach-end image includes: Human body key points are extracted from the coach's image to obtain the coach skeletal points corresponding to the multiple key points of the coach. Based on all the coach skeleton points, the coach's second key point information is generated.

3. The method according to claim 1, characterized in that, Based on the first key point information of all the key points, a high-dimensional vector of the user's practice movements is obtained, and based on the second key point information of all the key points, a high-dimensional vector of the coach's teaching movements is obtained, including: Based on the x-coordinate, y-coordinate, and depth coordinates of all the key points of the user, a row vector is constructed to obtain the first three-dimensional coordinate information; and based on the x-coordinate, y-coordinate, and depth coordinates of all the key points of the coach, a row vector is constructed to obtain the second three-dimensional coordinate information. Based on the head feature angle, limb feature angle, and trunk feature angle corresponding to all the key points of the user, a row vector is constructed to obtain the first angle information; and based on the head feature angle, limb feature angle, and trunk feature angle corresponding to all the key points of the coach, a row vector is constructed to obtain the second angle information. Based on the feature convex hull volume corresponding to all the key points of the user, a row vector is constructed to obtain the first convex hull volume information; and based on the feature convex hull volume corresponding to all the key points of the coach, a row vector is constructed to obtain the second convex hull volume information. Based on the first 3D coordinate information, the first angle information, and the first convex hull volume information, row vectors are constructed to obtain the high-dimensional vector of the user's practice actions. Based on the second 3D coordinate information, the second angle information, and the second convex hull volume information, row vectors are constructed to obtain the high-dimensional vector of the coach's corresponding teaching actions.

4. The method according to claim 3, characterized in that, The step of constructing a row vector based on the x-coordinate, y-coordinate, and depth coordinates of all the key points of the user to obtain the first 3D coordinate information includes: The horizontal coordinates of all the key points of the user are arranged in a preset first order to form a row vector, thus obtaining the first horizontal coordinate sub-vector; The ordinates of all the key points of the user are arranged in a preset first order to form a row vector, thus obtaining the first ordinate sub-vector; The depth coordinates of all the key points of the user are arranged in a preset first order to form a row vector, thus obtaining a first depth coordinate sub-vector; The first horizontal coordinate vector, the first vertical coordinate vector, and the first depth coordinate vector are sequentially concatenated in a preset second order to construct a row vector, and then normalized to obtain the first three-dimensional coordinate information. The step of constructing a row vector based on the x-coordinate, y-coordinate, and depth coordinates of all the key points mentioned by the coach to obtain the second 3D coordinate information includes: The horizontal coordinates of all the key points of the coach are arranged in a preset first order to form a row vector, and a second horizontal coordinate sub-vector is obtained. The ordinates of all the key points of the coach are arranged in a preset first order to form a row vector, and a second ordinate component vector is obtained. The depth coordinates of all the key points of the coach are arranged in a preset first order to form a row vector, and a second depth coordinate sub-vector is obtained. The second horizontal coordinate vector, the second vertical coordinate vector, and the second depth coordinate vector are sequentially concatenated in a preset second order to construct a row vector, and then normalized to obtain the second 3D coordinate information.

5. The method according to claim 1, characterized in that, The process of obtaining similarity feature values ​​based on the high-dimensional vectors of the practice movements and the teaching movements includes: Calculate the cosine similarity between the high-dimensional vector of the practice action and the high-dimensional vector of the teaching action to obtain the first similarity value; Based on the first similarity value, the similarity feature value is obtained.

6. The method according to claim 1, characterized in that, The process of obtaining similarity feature values ​​based on the high-dimensional vectors of the practice movements and the teaching movements includes: Based on the high-dimensional vector of the exercise action and the preset vector gain coefficient, the first target high-dimensional vector is obtained; Based on the high-dimensional vector of the teaching action and the vector gain coefficient, the second target high-dimensional vector is obtained; Calculate the cosine similarity between the high-dimensional vector of the first target and the high-dimensional vector of the second target to obtain the second similarity value; Based on the second similarity value, the similarity feature value is obtained.

7. A device for determining the accuracy of a movement, characterized in that, include: The image acquisition unit is used to acquire user-end images and coach-end images; the user-end images are exercise images of the user during the playback of fitness videos; The coach-side image is the instructional image in the fitness video that corresponds to the user-side image. The key point information determination unit is used to obtain first key point information corresponding to multiple key points of the user based on the user terminal image, and to obtain second key point information corresponding to multiple key points of the coach based on the coach terminal image; wherein, each key point information includes 3D coordinate information, angle information, and convex hull volume information; each 3D coordinate information includes a horizontal coordinate component vector, a vertical coordinate component vector, and a depth coordinate component vector; each angle information includes a head component vector, a limb component vector, and a trunk component vector; each convex hull volume information includes a head convex hull component vector, a limb convex hull component vector, a limb-trunk convex hull component vector, and a full-body convex hull component vector; A high-dimensional vector construction unit is used to obtain a high-dimensional vector of the user's practice movements based on the first key point information of all the key points, and to obtain a high-dimensional vector of the coach's teaching movements based on the second key point information of all the key points. The standardization assessment unit is used to obtain similarity feature values ​​based on the high-dimensional vectors of the practice movements and the high-dimensional vectors of the instruction movements; the similarity feature values ​​characterize the degree of standardization of the user's movements during the fitness process.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

9. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program that can run on the processor, and when the computer program is executed by the processor, it implements the method of any one of claims 1 to 6.

Images