Visual perception of human action digital twin analysis method and system

Abstract

The application provides a visual perception character action digital twin analysis method and system, belongs to the technical field of machine vision, and aims to solve the problems of lack of objectivity and effectiveness in current evaluation of eight-section-chi actions. When a target user performs a target eight-section-chi action, the spatial position of a marking point on the body and the action orientation of the target user are used to obtain the eight-section-chi body posture of the target user through a machine vision method, and further comparison is made between the generated digital twin simulation image and the action template image. In the case of matching, the target user is prompted about the action execution standard. The action of the user can be automatically evaluated and guided through the machine vision method, the objectivity and effectiveness of the eight-section-chi action evaluation can be improved to a certain extent, and the eight-section-chi is conducive to the promotion in actual scenes.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, and in particular to a method and system for visual perception of human motion digital twin analysis. Background Technology

[0002] The Eight-Section Brocade emphasizes slow, gentle movements with a balance of tension and relaxation, and focuses on coordinated breathing (such as stretching while inhaling and relaxing while exhaling). During practice, it's essential to maintain a unity of mind and body, avoiding stiffness or excessive force. The Eight-Section Brocade integrates physical movement, breathing regulation, and psychological adjustment, combining scientific principles with artistic expression. Its movement design is based on Traditional Chinese Medicine's meridian theory, systematically exercising various parts of the body while emphasizing internal and external coordination. However, currently, practitioners of the Eight-Section Brocade often exhibit problems such as incorrect movement technique, mismatched breathing rhythm with movement, and insufficient force in execution. Furthermore, there is a lack of effective supervision methods; currently, there is no effective way to assess and correct practitioners' movements. All of these factors hinder the widespread adoption of the Eight-Section Brocade in practical settings. Summary of the Invention

[0003] In view of the above problems, embodiments of this application provide a method, system, electronic device, and readable storage medium for visually perceptual digital twin analysis of human movements, in order to overcome the above problems or at least partially solve the above problems.

[0004] In a first aspect, embodiments of this application provide a method for visually perceiving digital twin analysis of human movements, the method comprising: When the target user performs the target Baduanjin movement, obtain the first spatial position of the first marker point corresponding to the first limb of the target user and the first movement orientation of the target user when performing the target Baduanjin movement; Based on the first movement orientation and the first spatial position, determine the first eight-section brocade limb postures corresponding to the first limbs respectively; Based on the first Baduanjin posture and the first relative position between each of the first limbs, a first Baduanjin digital twin simulation image of the target user is generated; When the template image of the target Baduanjin movement matches the first digital twin simulation image of Baduanjin, a first prompt message is generated to indicate to the target user that the execution of the target Baduanjin movement conforms to the standard.

[0005] Optionally, determining the posture of the first limb corresponding to the first eight sections of the brocade based on the first movement orientation and the first spatial position includes: Based on the first identifier of the first marker, the first marker is grouped to obtain the first marker group corresponding to each of the first limbs; Based on the first group of markers, the first spatial location is grouped to obtain the first spatial location group corresponding to each group of the first markers. Based on each first spatial position and the first movement orientation contained in the first spatial position group, the first eight-section brocade limb posture corresponding to each first limb is determined.

[0006] Optionally, determining the first eight-section brocade limb posture corresponding to each of the first limbs based on each of the first spatial positions included in the first spatial position group and the first movement orientation includes: When the orientation of the first movement is the same as the orientation of the template movement of the target Baduanjin movement, the first Baduanjin limb posture corresponding to each of the first limbs is determined based on each of the first spatial positions included in the first spatial position group.

[0007] Optionally, determining the first eight-section brocade limb posture corresponding to each of the first limbs based on each of the first spatial positions included in the first spatial position group and the first movement orientation includes: If the orientation of the first movement is different from the orientation of the template movement of the target Baduanjin movement, determine the orientation angle deviation between the orientation of the first movement and the orientation of the template movement; Using the center line of the target user's torso image as the rotation axis, and based on the orientation angle deviation, adjust each of the first spatial positions contained in the first spatial position group to obtain the second spatial position; Based on the second spatial positions corresponding to the first spatial position groups, the postures of the first eight sections of the brocade are determined for each of the first limbs.

[0008] Optionally, the method further includes: Using the target user's shoulder position as a reference point, and when the shoulder position of the action template image coincides with the shoulder position of the first Baduanjin digital twin simulation image, a first number of overlapping pixels between the action template image and the first Baduanjin digital twin simulation image is determined. If the ratio between the first quantity and the second quantity of image pixels in the action template image is greater than or equal to a first preset threshold, it is determined that the action template image matches the first digital twin simulation image of the Eight-Section Brocade.

[0009] Optionally, generating the first prompt information includes: Determine the first distance between the chest cavity marker and the back marker of the target user; Based on the relationship between the first distance and time, the first breathing state of the target user is determined; the first breathing state includes exhalation and inhalation. When the first breathing state matches the target Baduanjin movement, a first prompt message is generated.

[0010] Optionally, the method further includes: Determine the target breathing state that is compatible with the target Eight-Section Brocade movements; If the first breathing state is the same as the target breathing state, it is determined that the first breathing state matches the target Baduanjin movement.

[0011] Optionally, the method further includes: If the target action template image does not match the first Baduanjin digital twin simulation image, or if the first breathing state does not match the target Baduanjin action, a second prompt message is generated to prompt the target user that the action execution of the target Baduanjin action does not meet the standard.

[0012] Optionally, the target user performs the target Baduanjin movement while wearing a first detachable weight, and the first mass of the first detachable weight matches the target Baduanjin movement.

[0013] Secondly, embodiments of this application provide a visually perceptual digital twin analysis system for human movement, the system comprising: The first acquisition module is used to acquire the first spatial position of the first marker point corresponding to the first limb of the target user and the first movement orientation of the target user when performing the target Baduanjin movement, when the target user performs the target Baduanjin movement. The first determining module determines the posture of the first eight-section brocade limbs corresponding to the first limbs based on the first action orientation and the first spatial position. The first generation module is used to generate a first digital twin simulation image of the target user's Eight-Section Brocade posture based on the first Eight-Section Brocade posture and the first relative position between each of the first limbs; The second generation module is used to generate a first prompt message when the action template image of the target Baduanjin movement matches the first Baduanjin digital twin simulation image, so as to prompt the target user that the action execution of the target Baduanjin movement meets the standard.

[0014] Optionally, the first determining module includes: The first grouping submodule is used to group the first marker points based on the first identifier of the first marker points to obtain the first marker point groups corresponding to each of the first limbs. The second grouping submodule is used to group the first spatial location based on the first marker point group to obtain the first spatial location group corresponding to each of the first marker point groups; The first determining submodule is used to determine the posture of the first eight-section brocade limbs corresponding to each of the first limbs based on each of the first spatial positions included in the first spatial position group and the first action orientation.

[0015] Optionally, the first determining submodule includes: The first determining unit is used to determine the first Baduanjin limb posture corresponding to each of the first limbs based on each of the first spatial positions included in the first spatial position group, when the first movement orientation is the same as the template movement orientation of the target Baduanjin movement.

[0016] Optionally, the first determining submodule includes: The second determining unit is used to determine the angular deviation between the first action orientation and the template action orientation when the first action orientation is different from the template action orientation of the target Baduanjin action. The adjustment unit is used to adjust each of the first spatial positions included in the first spatial position group based on the orientation angle deviation, with the center line of the target user's torso image as the rotation axis, to obtain the second spatial position; The third determining unit is used to determine the posture of the first eight-section brocade limbs corresponding to each of the first limbs based on the second spatial positions corresponding to the first spatial position groups.

[0017] Optionally, the device further includes: The second determining module is used to determine a first number of overlapping pixels between the action template image and the first eight-section brocade digital twin simulation image, with the target user's shoulder position as the reference point position. The third determining module is used to determine that the action template image matches the first Baduanjin digital twin simulation image when the ratio between the first quantity and the second quantity of image pixels of the action template image is greater than or equal to a first preset threshold.

[0018] Optionally, the second generation module includes: The second determining submodule is used to determine the first distance between the chest cavity marker and the back marker of the target user; The third determining submodule is used to determine the first breathing state of the target user based on the relationship between the first distance and time; the first breathing state includes exhalation and inhalation. A generation submodule is used to generate a first prompt message when the first breathing state matches the target Baduanjin movement.

[0019] Optionally, the device further includes: The fourth determining module is used to determine the target breathing state that is compatible with the target Baduanjin movements; The fifth determining module is used to determine that the first breathing state matches the target Baduanjin movement when the first breathing state is the same as the target breathing state.

[0020] Optionally, the device further includes: The third generation module is used to generate a second prompt message when the target action template image does not match the first Baduanjin digital twin simulation image, or when the first breathing state does not match the target Baduanjin action, so as to prompt the target user that the action execution of the target Baduanjin action does not meet the standard.

[0021] Optionally, the target user performs the target Baduanjin movement while wearing a first detachable weight, and the first mass of the first detachable weight matches the target Baduanjin movement.

[0022] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the visual perception-based digital twin analysis method for human motion as described above.

[0023] Fourthly, embodiments of this application provide a readable storage medium storing a program or instructions, which, when executed by a processor, implement the visual perception-based digital twin analysis method for human actions as described above.

[0024] The specific beneficial effects are as follows: This application embodiment obtains the first spatial position of the first marker point corresponding to the first limb of the target user and the first movement orientation of the target user when performing the target Baduanjin movement. Based on the first movement orientation and the first spatial position, the first Baduanjin limb posture corresponding to the first limb is determined. Based on the first Baduanjin posture and the first relative position between each first limb, a first Baduanjin digital twin simulation image of the target user is generated. When the action template image of the target Baduanjin movement matches the first Baduanjin digital twin simulation image, a first prompt message is generated to prompt the target user that the action execution of the target Baduanjin movement meets the standard. When the target user performs the target Baduanjin movement, the spatial position of the marker point on the limb and the target user's movement orientation can be obtained through machine vision methods to obtain the target user's Baduanjin limb posture. Furthermore, the generated digital twin simulation image is compared with the action template image. When they match, the target user is prompted that the action execution meets the standard. It can automatically evaluate and guide the user's movement through machine vision methods, which can improve the objectivity and effectiveness of Baduanjin movement evaluation to a certain extent and is conducive to the promotion of Baduanjin in real-world scenarios. Attached Figure Description

[0025] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart illustrating a visual perception-based digital twin analysis method for human movement provided in an embodiment of this application. Figure 2 This is a flowchart illustrating another visual perception-based digital twin analysis method for human movement provided in an embodiment of this application. Figure 3 This is a flowchart illustrating a method for generating a second prompt message according to an embodiment of this application; Figure 4 This is a schematic diagram illustrating the use of an Eight-Section Brocade movement analysis system provided in an embodiment of this application; Figure 5 A comparison diagram of human movements provided in an embodiment of this application; Figure 6 This is a logic block diagram of a visual perception-based digital twin analysis system for human movement provided in an embodiment of this application; Figure 7 This is a schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0028] Reference Figure 1 , Figure 1 A flowchart illustrating a visual perception-based digital twin analysis method for human movement, provided in this application embodiment, includes: Step 101: When the target user performs the target Baduanjin movement, obtain the first spatial position of the first marker point corresponding to the first limb of the target user and the first movement orientation of the target user when performing the target Baduanjin movement.

[0029] In the embodiments of this application, there are two methods for marking the target user's first limb: the first is to mark joint points as first marker points in limb recognition, and the second is for the target user to wear a motion capture device with the first marker points. Since the Baduanjin movements are mainly limb movements, the second method can be used to mark the target user's first limb to improve the accuracy of motion recognition. Thus, when the target user performs the Baduanjin movements, the first spatial position of the first marker point corresponding to each of the target user's first limbs can be obtained by recognizing the first marker points on the motion capture device; simultaneously, facial recognition can also be used to obtain the first orientation of the target user's Baduanjin movements by observing the direction of the line connecting the target user's eyes.

[0030] Step 102: Based on the first movement orientation and the first spatial position, determine the first eight-section brocade limb postures corresponding to the first limbs.

[0031] In the embodiments of this application, the posture of each first limb corresponding to the first Baduanjin exercise can be determined by connecting the first spatial positions based on the first movement orientation and the first spatial position. The first movement orientation can be used to determine the correction amount for the first spatial position. If the first movement orientation is the same as the standard orientation of the target Baduanjin movement, the correction amount can be 0.

[0032] Step 103: Based on the first Baduanjin posture and the first relative position between each of the first limbs, generate the first Baduanjin digital twin simulation image of the target user.

[0033] In the embodiments of this application, digital twin images are an important component of digital twin technology. Digital twin images, through high-precision modeling, data acquisition, and simulation technologies, transform physical entities (such as buildings, equipment, or systems) into dynamic, visualized digital copies. They not only present a static appearance but also reflect the object's operational status and behavioral logic in real time, becoming a key link between reality and virtual space. Digital twin images can integrate sensor data streams to achieve functions such as parameter optimization and fault prediction. In the embodiments of this application, the human body recognition details of the target user in the required digital twin simulation image can only include limb movements and facial orientation; other details can be blurred. First, the first relative position between each first limb can be determined based on the connection point between the first limb and the torso. Then, based on the first Baduanjin posture and the aforementioned first relative position, a first Baduanjin digital twin simulation image of the target user can be generated. In the first Baduanjin digital twin simulation image, only the movement characteristics of Baduanjin (such as limb movements) and facial orientation can be reflected. Facial orientation can be identified by the eyes.

[0034] Step 104: If the template image of the target Baduanjin movement matches the first digital twin simulation image of Baduanjin, generate a first prompt message to indicate to the target user that the execution of the target Baduanjin movement conforms to the standard.

[0035] In embodiments of this application, a template image of the target Baduanjin movement can be compared with a first digital twin simulation image of Baduanjin. If the template image matches the first digital twin simulation image, a first prompt message can be generated. This first prompt message can be used to remind the target user that the execution of the target Baduanjin movement conforms to the standard. For example, a voice message such as "The movement is standard, please continue" can be generated, and the aforementioned text can also be displayed on the display module.

[0036] In the embodiments of this application, when a target user performs a target Baduanjin movement, the first spatial position of the first marker point corresponding to the first limb of the target user and the first movement orientation of the target user performing the target Baduanjin movement are obtained. Based on the first movement orientation and the first spatial position, the first Baduanjin limb posture corresponding to the first limb is determined. Based on the first Baduanjin posture and the first relative position between each first limb, a first Baduanjin digital twin simulation image of the target user is generated. When the movement template image of the target Baduanjin movement matches the first Baduanjin digital twin simulation image, a first prompt message is generated to prompt the target user that the movement execution of the target Baduanjin movement meets the standard. When the target user performs the target Baduanjin movement, the spatial position of the marker point on the limb and the movement orientation of the target user can be obtained through machine vision methods to obtain the Baduanjin limb posture of the target user. Furthermore, the generated digital twin simulation image is compared with the movement template image. When they match, the target user is prompted that the movement execution meets the standard. The machine vision method can automatically evaluate and guide the user's movement, which can improve the objectivity and effectiveness of Baduanjin movement evaluation to a certain extent and is conducive to the promotion of Baduanjin in real-world scenarios.

[0037] Reference Figure 2 , Figure 2 A flowchart illustrating another visual perception-based digital twin analysis method for human movement provided in this application embodiment, the method may include: Step 201: When the target user performs the target Baduanjin movement, obtain the first spatial position of the first marker point corresponding to the first limb of the target user and the first movement orientation of the target user when performing the target Baduanjin movement.

[0038] In the embodiments of this application, the implementation of this step can be referred to the embodiment of step 101, and will not be repeated here.

[0039] Optionally, the target user performs the target Baduanjin movement while wearing a first detachable weight, and the first mass of the first detachable weight matches the target Baduanjin movement.

[0040] In the embodiments of this application, the Baduanjin movements typically require sustained exertion of the limbs to achieve the desired training effect. Different Baduanjin movements require different levels of force. Therefore, the target user can perform the target Baduanjin movement while wearing a first detachable weight, the mass of which can be matched to the target Baduanjin movement. That is, the mass of the first detachable weight can be determined by Baduanjin experts based on the actual training effect and the weight conversion effect of the wearable device. This can further improve the accuracy of Baduanjin movement assessment.

[0041] Step 202: Based on the first identifier of the first marker, the first marker is grouped to obtain the first marker group corresponding to each of the first limbs.

[0042] In the embodiments of this application, the first marker point may have a first identifier. For the same first limb (such as the left arm), the first identifier of the first marker point may be the same. In this way, the first marker points can be grouped according to the first identifier of the first marker point, thereby obtaining the first marker point group corresponding to each first limb.

[0043] Step 203: Based on the first group of marker points, the first spatial location is grouped to obtain the first spatial location group corresponding to each group of the first marker points.

[0044] In the embodiments of this application, since each first marker point corresponds to a first spatial location, the first spatial locations can also be grouped according to the first marker point group, thereby obtaining the first spatial location groups corresponding to each first marker point group.

[0045] Step 204: Based on each first spatial position and the first movement orientation contained in the first spatial position group, determine the first eight-section brocade limb posture corresponding to each first limb.

[0046] In embodiments of this application, the first eight-section brocade limb postures corresponding to each first limb can be determined by grouping each first spatial position according to the first spatial position and combining it with the first movement orientation. The first eight-section brocade limb posture can be a broken line segment image along the first limb from the connection point between the first limb and the torso to the limb's end. The bending point of this broken line segment corresponds to a joint of the first limb (for example, for the left arm, there can be two bending points, namely the elbow joint and the wrist joint). The first movement orientation can be used to determine the correction amount for the first spatial position.

[0047] Optionally, step 204 may include the following sub-steps: Sub-step 2041: When the first movement orientation is the same as the template movement orientation of the target Baduanjin movement, determine the first Baduanjin limb posture corresponding to each of the first limbs based on each of the first spatial positions included in the first spatial position group.

[0048] In the embodiments of this application, if the orientation of the first movement is the same as the orientation of the template movement of the target Baduanjin movement, the posture of the first Baduanjin limb corresponding to each first limb can be determined according to each first spatial position included in the first spatial position group. In this case, there is no need to correct the first spatial position.

[0049] In the embodiments of this application, by determining the first eight-section limb posture corresponding to each first limb based on each first spatial position contained in the first spatial position group when the first action orientation is the same as the template action orientation of the target eight-section limb posture, the first eight-section limb posture can be generated according to each first spatial position contained in the first spatial position group when the first action orientation is the same as the template action orientation, which can improve the accuracy and reliability of the first eight-section limb posture to a certain extent.

[0050] Sub-step 2042: If the orientation of the first movement is different from the orientation of the template movement of the target Baduanjin movement, determine the orientation angle deviation between the orientation of the first movement and the orientation of the template movement.

[0051] In the embodiments of this application, if the orientation of the first movement is different from the orientation of the template movement of the target Baduanjin movement, the orientation angle deviation between the first movement orientation and the template movement orientation can be determined. There are two methods to calculate the orientation angle deviation. The first method involves directly using the left and right shoulders of the target user as two specific positions, taking the center of the line connecting the left and right shoulders as the rotation center, and drawing a horizontal line passing through the center of the line. Finally, the angle between the line connecting the left and right shoulders and the horizontal line is taken as the orientation angle deviation. This is a relatively rough calculation method, which does not consider the visual deviation caused by the actual observation point and the left and right shoulders during rotation. However, in this application scenario, precise calculation is not required, so the first method can meet the actual needs. The second method involves spatially modeling the left and right shoulders of the target user and the actual observation point (i.e., the machine vision capture point), and using the mathematical expression of motion trajectory projection to back-calculate, thereby accurately calculating the orientation angle deviation. The specific modeling and calculation process is as follows: Suppose a straight line is drawn from the first midpoint of the first line connecting the left and right shoulders of the target user, and the second midpoint of the second line connecting the left and right thigh roots, passing through the first and second midpoints. This straight line can be considered the target user's orientation rotation axis. Assuming the left shoulder marker is a reference point for the orientation angle deviation, the spatial trajectory traversed by this reference point when the target user rotates along the orientation rotation axis is a circle. Furthermore, if the machine vision sensing point is neither on the orientation rotation axis nor on the plane containing the aforementioned spatial trajectory, then the planar trajectory traversed by the reference point observed from the machine vision sensing point is an ellipse. That is, the projection of the spatial trajectory traversed by the reference point onto the observation plane is an ellipse, where the observation plane is perpendicular to the line connecting the center of the aforementioned spatial trajectory and the machine vision sensing point. Based on this, solid geometric modeling can be performed.

[0052] Suppose an object is undergoing uniform circular motion in three-dimensional space, with its plane being the XY plane, its center at the origin O(0, 0, 0), and its radius being r. The position of the object on its trajectory can be represented as: (Equation 1); In equation 1 above, It is the polar angle.

[0053] The coordinates of the machine vision observation point are ,satisfy and Construct an observation point plane, which passes through the machine vision observation point and is perpendicular to the line connecting the machine vision observation point and the center of the circle. The projection of the aforementioned motion trajectory onto the observation point plane is then an ellipse.

[0054] Since this embodiment only requires the deflection angle of the left shoulder marker, the calculation can be performed using points on the projection. Specifically, a reference line is defined as the line connecting the reference point of the left shoulder marker and the center of the circle, satisfying the following condition: this reference line is perpendicular to the vertical line from the machine vision observation point to the axis of rotation (i.e., the reference point is the left shoulder marker pointing downwards in the direction of the template's movement). Thus, knowing the coordinates on the projection, the coordinates of the object on its actual motion trajectory can be calculated through coordinate transformation, thereby allowing the determination of the steering angle deviation.

[0055] The affine transformation matrix from the trajectory to the projection is shown in Equation 2 below: (Equation 2); In equation 2 above, Denotes the affine transformation matrix. Represents the identity matrix. The normal vector of the plane at the observation point In this embodiment of the application, the normal vector is actually... , Indicates the coordinates of the observation point.

[0056] For the two projection points of the observation point plane and The true points of the two projection points on the motion trajectory can be obtained by using the inverse affine transformation matrix. and As shown in equations 3 and 4 below: (Equation 3); (Equation 4); In equations 3 and 4 above, the symbol " " represents the affine transformation matrix The inverse matrix, The coordinates of the left shoulder marker point, The coordinates are the reference point for the left shoulder marker.

[0057] Finally, the steering angle deviation can be calculated using the arctangent function, as shown in Equation 5 below: (Equation 5); In equation 5 above, and They are respectively The y-coordinates and x-coordinates of the point and They are respectively The y-coordinates and x-coordinates of the point This refers to the steering angle deviation.

[0058] Sub-step 2043: Using the center line of the target user's torso image as the rotation axis, and based on the orientation angle deviation, adjust each of the first spatial positions included in the first spatial position group to obtain the second spatial position.

[0059] In the embodiments of this application, the first spatial positions included in the first spatial position group can be adjusted according to the orientation angle deviation, with the center line of the target user's torso image as the rotation axis, thereby obtaining the second spatial position. The second spatial position can be shown in Equation 6 below: (Formula 6); In equation 6 above, Indicates the second spatial position coordinates. The deflection angle in the first spatial coordinate system. This represents the orientation angle deviation.

[0060] Sub-step 2044: Based on the second spatial positions corresponding to the first spatial positions, determine the first eight-section brocade limb postures corresponding to each of the first limbs.

[0061] In the embodiments of this application, the first limb postures corresponding to the first limbs can be determined based on the second spatial positions corresponding to each first spatial position group. For details regarding the implementation of the first limb postures, please refer to the implementation content under step 204, which will not be repeated here.

[0062] In the embodiments of this application, when the first action orientation is different from the template action orientation of the target Baduanjin movement, the orientation angle deviation between the first action orientation and the template action orientation is determined. Taking the center line of the target user's torso image as the rotation axis, the first spatial positions contained in the first spatial position group are adjusted based on the orientation angle deviation to obtain the second spatial position. Based on the second spatial positions corresponding to the first spatial position groups, the first Baduanjin limb postures corresponding to each first limb are determined. When the first action orientation is different from the template action orientation, the first spatial position can be adjusted according to the orientation angle deviation between the first action orientation and the template action orientation to obtain the second spatial position. Finally, the first Baduanjin limb postures are generated based on the second spatial positions, which can further improve the accuracy and reliability of the first Baduanjin limb postures.

[0063] In the embodiments of this application, the first marker points are grouped based on the first identifier of the first marker point to obtain the first marker point group corresponding to each first limb. Based on the first marker point group, the first spatial position is grouped to obtain the first spatial position group corresponding to each first marker point group. Based on each first spatial position and the first action orientation contained in the first spatial position group, the first eight-section brocade limb posture corresponding to each first limb is determined. The first eight-section brocade limb posture corresponding to each first limb can be generated separately by group, which can improve the generation efficiency and accuracy of the first eight-section brocade limb posture to a certain extent.

[0064] Step 205: Based on the first Baduanjin posture and the first relative position between each of the first limbs, generate the first Baduanjin digital twin simulation image of the target user.

[0065] In the embodiments of this application, the implementation of this step can be referred to the embodiment of step 103, and will not be repeated here.

[0066] Step 206: Using the target user's shoulder positions as reference points, and assuming the shoulder positions of the action template image and the first Baduanjin digital twin simulation image overlap, determine the first number of overlapping pixels between the action template image and the first Baduanjin digital twin simulation image.

[0067] In the embodiments of this application, the position of the target user's shoulders can be used as a reference point. If the creation position of the action template image coincides with the position of the shoulders of the first Baduanjin digital twin simulation image, the first number of overlapping pixels between the action template image and the first Baduanjin digital twin simulation image can be counted.

[0068] Step 207: If the ratio between the first quantity and the second quantity of image pixels in the action template image is greater than or equal to a first preset threshold, determine that the action template image matches the first Baduanjin digital twin simulation image.

[0069] In the embodiments of this application, a second number of image pixels of the action template image can be pre-counted. If the ratio between the first number and the second number is greater than or equal to a first preset threshold (e.g., 0.85 or 0.9), the action template image can be considered to match the first Baduanjin digital twin simulation image.

[0070] In the embodiments of this application, by using the shoulder positions of the target user as reference points, when the shoulder positions of the action template image and the first Baduanjin digital twin simulation image coincide, a first number of overlapping pixels between the action template image and the first Baduanjin digital twin simulation image is determined. If the ratio between the first number and the second number of image pixels of the action template image is greater than or equal to a first preset threshold, it is determined that the action template image and the first Baduanjin digital twin simulation image match. Using the shoulder positions as reference points to determine whether the action template image and the first Baduanjin digital twin simulation image match can improve the accuracy and reliability of the matching determination to a certain extent.

[0071] Step 208: If the template image of the target Baduanjin movement matches the first digital twin simulation image of Baduanjin, a first prompt message is generated to prompt the target user that the execution of the target Baduanjin movement conforms to the standard.

[0072] In the embodiments of this application, the implementation of this step can be referred to the embodiment of step 104, and will not be repeated here.

[0073] Optionally, step 208 may include the following sub-steps: Sub-step 2081: Determine the first distance between the chest cavity marker and the back marker of the target user.

[0074] In the embodiments of this application, while the target user is performing the target Baduanjin movements, the target user's breathing status can also be monitored simultaneously. Considering that the chest cavity rises and falls with breathing, a chest cavity marker point can be set on the target user's chest cavity, and a back marker point can be set on the back. The chest cavity marker point and the back marker point can sense each other, and the first distance between the chest cavity marker point and the back marker point can be obtained in real time.

[0075] Sub-step 2082: Based on the relationship between the first distance and time, determine the first breathing state of the target user; the first breathing state includes exhalation state and inhalation state.

[0076] In the embodiments of this application, considering that the chest cavity contracts during exhalation and expands during inhalation, the target user's first breathing state can be determined based on the relationship between the first distance and time. If the first distance gradually increases over time, it indicates that the first breathing state is inhalation; if the first distance gradually decreases over time, it indicates that the first breathing state is exhalation.

[0077] Sub-step 2083: Determine the target breathing state that is compatible with the target Baduanjin movements.

[0078] In the embodiments of this application, a target breathing state adapted to the target Baduanjin movements can be determined. The target breathing state is either an exhalation state or an inhalation state.

[0079] Sub-step 2084: If the first breathing state is the same as the target breathing state, determine that the first breathing state matches the target Baduanjin movement.

[0080] In the embodiments of this application, if the first breathing state is the same as the target breathing state, it can be considered that the first breathing state matches the target Baduanjin movement.

[0081] In the embodiments of this application, by determining the target breathing state that is compatible with the target Baduanjin movement, and if the first breathing state is the same as the target breathing state, it is determined that the first breathing state matches the target Baduanjin movement, which can improve the accuracy and reliability of the matching determination to a certain extent.

[0082] Sub-step 2085: When the first breathing state matches the target Baduanjin movement, generate the first prompt message.

[0083] In the embodiments of this application, if the first breathing state matches the target Baduanjin movement, a first prompt message can be generated. The implementation content of the first prompt message can be referred to the embodiment of step 104, and will not be repeated here.

[0084] In the embodiments of this application, by determining the first distance between the chest marker and the back marker of the target user, and based on the relationship between the change of the first distance and time, the first breathing state of the target user is determined; the first breathing state includes an exhalation state and an inhalation state. When the first breathing state matches the target Baduanjin movement, a first prompt message is generated. The first prompt message can be generated when the first breathing state matches the target Baduanjin movement and the first Baduanjin digital twin image matches the target movement template image, which can improve the accuracy and reliability of the first prompt message to a certain extent.

[0085] Based on the above implementation method, refer to Figure 3 , Figure 3 This is a flowchart illustrating a method for generating a second prompt message according to an embodiment of this application. After step 208, the visually perceived human action digital twin analysis method may further include: Step 209: If the target action template image does not match the first Baduanjin digital twin simulation image, or if the first breathing state does not match the target Baduanjin action, a second prompt message is generated to prompt the target user that the action execution of the target Baduanjin action does not meet the standard.

[0086] In the embodiments of this application, if the target action template image does not match the first Baduanjin digital twin simulation image, or if the first breathing state does not match the target Baduanjin action, a second prompt message can be generated. The second prompt message can be used to prompt the target user that the action execution of the target Baduanjin action does not meet the standard.

[0087] In the embodiments of this application, a second prompt message is generated when the target action template image does not match the first Baduanjin digital twin simulation image, or when the first breathing state does not match the target Baduanjin action, to prompt the target user that the action execution of the target Baduanjin action does not meet the standard. This can prompt the target user when the target user's execution of the target Baduanjin action is not standard, which can improve the practicality and applicability of the human action analysis method to a certain extent.

[0088] Reference Figure 4 , Figure 4 This is a schematic diagram illustrating the use of an Eight-Section Brocade movement analysis system provided in this application embodiment. In the diagram, 1 is a machine vision sensing point, 2 is a display device, 3 is a left shoulder marker, 4 is a right shoulder marker, and 5 is a left leg marker. The remaining black dots are ordinary first markers on the limbs. All markers are connected sequentially along the human skeletal growth path. When performing Eight-Section Brocade movement analysis on a user, the display device can first display a template image of the target Eight-Section Brocade movement to be performed. The user can then perform the Eight-Section Brocade movement matching the template image based on the image. Through markers 3, 4, and 5, as well as the various ordinary markers, the user's limb movements can be identified, thereby generating a digital twin virtual image of the user's movements, and finally obtaining the analysis results.

[0089] according to Figure 4 The described process, combined with Figure 5 Explain the process of correcting the character's orientation. Figure 5This application provides a comparison diagram of human movements, wherein (1) is a model diagram of the Baduanjin movement under the standard orientation, (2) is a model diagram of the Baduanjin movement under the user's actual orientation, and (3) is a model diagram of the Baduanjin movement after correcting the actual orientation of image (2) to the standard orientation. When capturing image (2), the user is 5 meters away from the machine vision perception point, the height of the machine vision perception point is 1 meter, and the height of the marker point on the user's neck is 1.6 meters. Using the marker point on the user's neck as the origin for three-dimensional modeling, the projection coordinates of the marker point on the right shoulder are (-24, 509, -63), and the normal vector is (0, -500, -60). Figure 1 The standard coordinates of the right shoulder are (-30, 0, 0). Calculations show that... Figure 2 The true coordinates of the right shoulder are (27.3, -12, 0) and the orientation angle deviation is -26.25°. Therefore, by rotating all the perceived markers in the figure by -26.25°, we can obtain image (3). In image 3, the coordinates of the right shoulder marker after the turning angle correction are (-29.98, 0, 0), which are close to the standard coordinates.

[0090] Reference Figure 6 , Figure 6 This application provides a logic block diagram of a visually perceptual digital twin analysis system for human motion, wherein the visually perceptual digital twin analysis system 600 may include: The first acquisition module 601 is used to acquire the first spatial position of the first marker point corresponding to the first limb of the target user and the first movement orientation of the target user when performing the target Baduanjin movement, when the target user performs the target Baduanjin movement. The first determining module 602 determines the posture of the first eight-section brocade limbs corresponding to the first limbs based on the first action orientation and the first spatial position. The first generation module 603 is used to generate a first digital twin simulation image of the target user's Eight-Section Brocade based on the first Eight-Section Brocade posture and the first relative position between each of the first limbs; The second generation module 604 is used to generate a first prompt message when the action template image of the target Baduanjin movement matches the first Baduanjin digital twin simulation image, so as to prompt the target user that the action execution of the target Baduanjin movement meets the standard.

[0091] Optionally, the first determining module 602 includes: The first grouping submodule is used to group the first marker points based on the first identifier of the first marker points to obtain the first marker point groups corresponding to each of the first limbs. The second grouping submodule is used to group the first spatial location based on the first marker point group to obtain the first spatial location group corresponding to each of the first marker point groups; The first determining submodule is used to determine the posture of the first eight-section brocade limbs corresponding to each of the first limbs based on each of the first spatial positions included in the first spatial position group and the first action orientation.

[0092] Optionally, the first determining submodule includes: The first determining unit is used to determine the first Baduanjin limb posture corresponding to each of the first limbs based on each of the first spatial positions included in the first spatial position group, when the first movement orientation is the same as the template movement orientation of the target Baduanjin movement.

[0093] Optionally, the first determining submodule includes: The second determining unit is used to determine the angular deviation between the first action orientation and the template action orientation when the first action orientation is different from the template action orientation of the target Baduanjin action. The adjustment unit is used to adjust each of the first spatial positions included in the first spatial position group based on the orientation angle deviation, with the center line of the target user's torso image as the rotation axis, to obtain the second spatial position; The third determining unit is used to determine the posture of the first eight-section brocade limbs corresponding to each of the first limbs based on the second spatial positions corresponding to the first spatial position groups.

[0094] Optionally, the device 600 further includes: The second determining module is used to determine a first number of overlapping pixels between the action template image and the first eight-section brocade digital twin simulation image, with the target user's shoulder position as the reference point position. The third determining module is used to determine that the action template image matches the first Baduanjin digital twin simulation image when the ratio between the first quantity and the second quantity of image pixels of the action template image is greater than or equal to a first preset threshold.

[0095] Optionally, the second generation module 604 includes: The second determining submodule is used to determine the first distance between the chest cavity marker and the back marker of the target user; The third determining submodule is used to determine the first breathing state of the target user based on the relationship between the first distance and time; the first breathing state includes exhalation and inhalation. A generation submodule is used to generate a first prompt message when the first breathing state matches the target Baduanjin movement.

[0096] Optionally, the device 600 further includes: The fourth determining module is used to determine the target breathing state that is compatible with the target Baduanjin movements; The fifth determining module is used to determine that the first breathing state matches the target Baduanjin movement when the first breathing state is the same as the target breathing state.

[0097] Optionally, the device 600 further includes: The third generation module is used to generate a second prompt message when the target action template image does not match the first Baduanjin digital twin simulation image, or when the first breathing state does not match the target Baduanjin action, so as to prompt the target user that the action execution of the target Baduanjin action does not meet the standard.

[0098] Optionally, the target user performs the target Baduanjin movement while wearing a first detachable weight, and the first mass of the first detachable weight matches the target Baduanjin movement.

[0099] The visual perception-based digital twin analysis system for human movement in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc., and can also be network attached storage (NAS), personal computer (PC), etc. This application embodiment does not specifically limit the scope of the system.

[0100] The visual perception-based digital twin analysis system for human movement in this application embodiment can be a system with an operating system. This operating system can be Android, Linux, Windows, or other possible operating systems; this application embodiment does not specifically limit it.

[0101] The visual perception-based digital twin analysis system for human movement provided in this application can achieve… Figures 1 to 2 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0102] This application provides an electronic device, see [link to relevant documentation] Figure 7 The electronic device 70 includes a processor 701, a memory 702, and a computer program 7021 stored in the memory 702 and executable on the processor 701. When the processor 701 executes the program, it implements the visual perception-based digital twin analysis method for human movement described in the foregoing embodiments.

[0103] This application also provides a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements the steps in the visual perception-based digital twin analysis method for human movement as disclosed in this application.

[0104] This application also provides a computer program product that, when run on an electronic device, enables the processor to execute the steps in the visual perception-based digital twin analysis method for human movement as disclosed in this application.

[0105] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0106] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, systems, electronic devices, and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0107] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction set implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0108] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0109] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of this application.

[0110] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0111] The above provides a detailed description of a visual perception-based digital twin analysis method, system, and device for human movement, as provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application areas based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for analyzing human movement using visual perception digital twins, characterized in that, The method includes: When the target user performs the target Baduanjin movement, obtain the first spatial position of the first marker point corresponding to the first limb of the target user and the first movement orientation of the target user when performing the target Baduanjin movement; Based on the first movement orientation and the first spatial position, determine the first eight-section brocade limb postures corresponding to the first limbs respectively; Based on the first Baduanjin posture and the first relative position between each of the first limbs, a first Baduanjin digital twin simulation image of the target user is generated; If the template image of the target Baduanjin movement matches the first digital twin simulation image of Baduanjin, a first prompt message is generated to indicate to the target user that the execution of the target Baduanjin movement conforms to the standard.

2. The method according to claim 1, characterized in that, The step of determining the posture of the first limb corresponding to the first eight sections of the brocade exercise based on the first movement orientation and the first spatial position includes: Based on the first identifier of the first marker, the first marker is grouped to obtain the first marker group corresponding to each of the first limbs; Based on the first group of markers, the first spatial location is grouped to obtain the first spatial location group corresponding to each group of the first markers. Based on each first spatial position and the first movement orientation contained in the first spatial position group, the first eight-section brocade limb posture corresponding to each first limb is determined.

3. The method according to claim 2, characterized in that, The step of determining the first eight-section brocade limb posture corresponding to each first limb based on each first spatial position included in the first spatial position group and the first movement orientation includes: When the orientation of the first movement is the same as the orientation of the template movement of the target Baduanjin movement, the first Baduanjin limb posture corresponding to each of the first limbs is determined based on each of the first spatial positions included in the first spatial position group.

4. The method according to claim 2, characterized in that, The step of determining the first eight-section brocade limb posture corresponding to each first limb based on each first spatial position included in the first spatial position group and the first movement orientation includes: If the orientation of the first movement is different from the orientation of the template movement of the target Baduanjin movement, determine the orientation angle deviation between the orientation of the first movement and the orientation of the template movement; Using the center line of the target user's torso image as the rotation axis, and based on the orientation angle deviation, adjust each of the first spatial positions contained in the first spatial position group to obtain the second spatial position; Based on the second spatial positions corresponding to the first spatial position groups, the postures of the first eight sections of the brocade are determined for each of the first limbs.

5. The method according to claim 1, characterized in that, The method further includes: Using the target user's shoulder position as a reference point, and when the shoulder position of the action template image coincides with the shoulder position of the first Baduanjin digital twin simulation image, a first number of overlapping pixels between the action template image and the first Baduanjin digital twin simulation image is determined. If the ratio between the first quantity and the second quantity of image pixels in the action template image is greater than or equal to a first preset threshold, it is determined that the action template image matches the first digital twin simulation image of the Eight-Section Brocade.

6. The method according to claim 1, characterized in that, The generation of the first prompt information includes: Determine the first distance between the chest cavity marker and the back marker of the target user; Based on the relationship between the first distance and time, the first breathing state of the target user is determined; the first breathing state includes exhalation and inhalation. When the first breathing state matches the target Baduanjin movement, a first prompt message is generated.

7. The method according to claim 6, characterized in that, The method further includes: Determine the target breathing state that is compatible with the target Eight-Section Brocade movements; If the first breathing state is the same as the target breathing state, it is determined that the first breathing state matches the target Baduanjin movement.

8. The method according to claim 6, characterized in that, The method further includes: If the target action template image does not match the first Baduanjin digital twin simulation image, or if the first breathing state does not match the target Baduanjin action, a second prompt message is generated to prompt the target user that the action execution of the target Baduanjin action does not meet the standard.

9. The method according to claim 1, characterized in that, The target user performs the target Baduanjin movement while wearing a first detachable weight, and the first mass of the first detachable weight matches the target Baduanjin movement.

10. A visually perceptual digital twin analysis system for human movement, characterized in that, The system includes: The first acquisition module is used to acquire the first spatial position of the first marker point corresponding to the first limb of the target user and the first movement orientation of the target user when performing the target Baduanjin movement, when the target user performs the target Baduanjin movement. The first determining module determines the posture of the first eight-section brocade limbs corresponding to the first limbs based on the first action orientation and the first spatial position. The first generation module is used to generate a first digital twin simulation image of the target user's Eight-Section Brocade posture based on the first Eight-Section Brocade posture and the first relative position between each of the first limbs; The second generation module is used to generate a first prompt message when the action template image of the target Baduanjin movement matches the first Baduanjin digital twin simulation image, so as to prompt the target user that the action execution of the target Baduanjin movement meets the standard.

Images