Joint coordinate system construction method and device based on human body surface grid model reconstruction, equipment and medium

Abstract

The invention discloses a joint coordinate system construction method and device based on human body surface grid model reconstruction, equipment and a medium. The method comprises the steps that a human body image of a target area is collected; preprocessing the human body image to obtain a standard image block; reconstructing the standard image block based on the parameterized human body grid model to obtain a human body surface grid model; acquiring joint coordinate system key points based on the human body surface grid model, and executing automatic screening of the joint coordinate system key points; and establishing a local coordinate system by using the screened joint coordinate system key points. Therefore, by introducing a human body surface grid model reconstruction technology, a clear joint coordinate system definition is established, and the left-right symmetry and time sequence continuity of the human body are ensured.

Description

Technical Field

[0001] This invention relates to a method, apparatus, device, and medium for constructing a joint coordinate system based on a human body surface mesh model, and relates to the field of computer vision technology. Background Technology

[0002] Motion capture (MoCap) technology acquires human motion data to accurately record and analyze human movement. The purpose of motion capture technology is to use various sensor inputs, such as inertial measurement units (IMUs), color cameras, and infrared cameras that detect reflective points, to acquire human postures at specific moments and combine these postures in chronological order to generate a sequence of human motion movements. By analyzing these sequences, the biomechanical characteristics of human movement can be obtained, or combined with 3D animation rendering technology for digital film and television creation. For motion capture systems, motion capture accuracy, the continuity and symmetry of human body shape, and compatibility with downstream tasks are key factors affecting their application effectiveness.

[0003] Existing motion capture technologies mainly include infrared optical motion capture, inertial motion capture, and visual motion capture. Infrared optical motion capture uses pre-placed infrared cameras to collect the 3D coordinates of reflective markers at specific locations on the subject's body. This technology boasts high precision, but the equipment is expensive and limited by fixed environments. Inertial motion capture uses IMU sensors placed on the limbs to calculate human posture based on IMU data. This technology has a wide range of applications, but it suffers from limitations such as sensor drift and dependence on initial posture. Visual motion capture is based on deep learning-based computer vision technology, using color cameras to acquire images and detect key joint points. This technology is low-cost and flexible in application scenarios, but currently faces technical challenges such as a lack of clear joint coordinate system definitions, insufficient precision in joint angle and rotation calculations, and difficulty in guaranteeing temporal continuity. Summary of the Invention

[0004] This invention aims to solve the technical problems existing in the prior art. Therefore, in response to the above problems, the purpose of this invention is to provide a method, apparatus, device, and medium for constructing a joint coordinate system based on human body surface mesh model reconstruction. By introducing human body surface mesh model reconstruction technology, a joint coordinate system definition compatible with infrared optical motion capture systems is established, ensuring the left-right symmetry of human body shape reconstruction and realizing the temporal continuity of motion sequences.

[0005] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows: In a first aspect, the present invention provides a method for constructing a joint coordinate system based on a reconstructed human body surface mesh model, comprising: Acquire human images of the target area; Preprocessing of human body images yields standard image blocks; Standard image patches are reconstructed based on a parametric human body mesh model to obtain a human body surface mesh model; Key points of the joint coordinate system are obtained based on the human body surface mesh model, and automatic filtering of key points of the joint coordinate system is performed. A local coordinate system is established using the key points of the joint coordinate system obtained through screening.

[0006] In some possible implementations, the human body image is preprocessed to obtain standard image patches, specifically as follows: A deep learning object detection algorithm is used to identify human body regions in human images, and the detected human body regions are cropped into standard image patches that meet the requirements.

[0007] In some possible implementations, a parametric human body mesh model is used to map 2D image information to the shape parameter space of a 3D mesh model, wherein the parametric human body mesh model adopts the parametric human body mesh model of the SMPLX standard or a custom human body mesh model modified based on the SMPLX standard.

[0008] In some possible implementations, joint coordinate system keypoints are used to define feature points of major parts of the human body, wherein joint coordinate system keypoints include head region keypoints, trunk region keypoints, upper limb keypoints, lower limb keypoints and / or distal end keypoints.

[0009] Some possible implementations include performing automatic filtering of key points in the joint coordinate system, including: The number of vertices and faces of the human body surface mesh model is preset, and the body shape features of the human body surface mesh model are parametrically controlled through shape parameters, where the shape parameters are a set of real-valued vectors, denoted as b=[b1, b2, …,b n The shape parameter ] is used to represent the offset of the human body shape in the statistical modeling space. By adjusting this set of shape parameters, the overall shape of the human body mesh model can be continuously adjusted, thereby generating human body surface mesh models of different body shapes. Customize the key point positions of the joint coordinate system according to requirements; Based on the expert-defined joint coordinate system key point template, only the corresponding key points are retained from the vertices of the human body surface mesh model, and their three-dimensional coordinates are obtained.

[0010] Some possible implementations also include a step of smoothing the time-series data of the selected joint coordinate system key points, specifically: using a low-pass filter to smooth the trajectory of the joint coordinate system key points, and by adjusting the cutoff frequency, suppressing the jitter of the joint coordinate system key point positions while maintaining motion details.

[0011] In some possible implementations, a local coordinate system is established using the selected joint coordinate system key points, including: Based on anatomical features, the connection relationships between key points in the joint coordinate system are determined to form a complete skeleton structure. The skeleton structure is represented as an ordered graph structure with a set of directed edges connecting the key point nodes of the joint coordinate system. Based on the key points of the joint coordinate system in the above skeleton structure, a local coordinate system can be defined at each joint. Specifically, at a certain joint, a right-handed Cartesian coordinate system is defined, with its origin located at the current joint position. The coordinate axis direction is obtained by calculating the orientation of adjacent bone segments, thereby realizing the expression of local spatial posture.

[0012] Secondly, the present invention also discloses a joint coordinate system construction device based on the reconstruction of a human body surface mesh model, comprising: The image acquisition unit is configured to acquire human images of the target area; The image processing unit is configured to preprocess human images to obtain standard image blocks; The mesh model reconstruction unit is configured to reconstruct standard image patches based on a parametric human body mesh model to obtain a human body surface mesh model. The key point filtering unit is configured to obtain key points in the joint coordinate system based on the reconstructed human body surface mesh model and perform automatic filtering of key points in the joint coordinate system. The coordinate system reconstruction unit is configured to establish a local coordinate system using the selected joint key points.

[0013] Thirdly, the present invention also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform any of the methods described herein.

[0014] Fourthly, the present invention also provides a computer-readable storage medium for storing one or more programs, said one or more programs including computer instructions for causing a computer to perform the method.

[0015] Secondly, the present invention also provides a joint coordinate system construction device based on human body surface mesh model reconstruction, comprising: The image acquisition unit is configured to acquire human images of the target area; The image processing unit is configured to preprocess human images to obtain standard image blocks; The mesh model reconstruction unit is configured to reconstruct standard image patches based on a parametric human body mesh model to obtain a human body surface mesh model. The key point filtering unit is configured to obtain key points in the joint coordinate system based on the reconstructed human body surface mesh model and perform automatic filtering of key points in the joint coordinate system. The coordinate system reconstruction unit is configured to establish a local coordinate system using the selected joint key points.

[0016] Thirdly, the present invention also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform the method described thereon.

[0017] Fourthly, the present invention also provides a computer-readable storage medium for storing one or more programs, said one or more programs including computer instructions for causing a computer to perform the method.

[0018] Because the present invention adopts the above technical solution, it has the following characteristics: 1. This invention establishes a clear joint coordinate system definition by introducing human body surface mesh model reconstruction technology, ensuring the left-right symmetry and temporal continuity of the human body; the output data format is compatible with infrared optical motion capture systems and can be directly used in existing motion analysis software.

[0019] 2. The joint coordinate system customization of this invention includes, but is not limited to, the following operations: Custom key point participation method: allows users to customize the key point set for a certain joint coordinate system; Custom axis definition rules: allows users to set which bone segment direction the principal axis (such as the z-axis) of the local coordinate system is calculated from, and which orthogonal supplementation method is used for other axes; Custom coordinate system origin offset, supports adding offsets at key point positions for adaptation to device ends, clothing simulation and other applications; This invention provides flexible joint coordinate system customization function, mainly reflected in the design of skeleton structure based on manually selected human body surface key points, expanding the application scenarios.

[0020] In summary, this invention can be widely applied to human motion capture. Attached Figure Description

[0021] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. In the drawings: Figure 1 This is a flowchart illustrating the method for constructing a joint coordinate system based on a human body surface mesh model according to an embodiment of the present invention.

[0022] Figure 2 A structural diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0023] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0024] Although terms such as first, second, third, etc., may be used in this document to describe multiple elements, components, regions, layers, and / or segments, these elements, components, regions, layers, and / or segments should not be limited by these terms. These terms may be used only to distinguish one element, component, region, layer, or segment from another. Unless the context clearly indicates otherwise, terms such as "first," "second," and other numerical terms used herein do not imply order or sequence. Therefore, the first element, component, region, layer, or segment discussed below may be referred to as the second element, component, region, layer, or segment without departing from the teachings of the exemplary embodiments.

[0025] For ease of description, spatial relative terms may be used in the text to describe the relationship of one element or feature relative to another element or feature as shown in the figure. These relative terms include, for example, "inside," "outside," "middle," "outer," "below," "above," etc. Such spatial relative terms are intended to include different orientations of the device in use or operation, other than those depicted in the figure.

[0026] Existing visual motion capture systems suffer from technical problems such as unclear joint coordinate system definitions and difficulty in guaranteeing temporal continuity. This invention provides a method, apparatus, device, and medium for constructing a joint coordinate system based on a human body surface mesh model. The method includes: acquiring human images of a target region; preprocessing the human images to obtain standard image blocks; reconstructing the standard image blocks based on a parametric human body mesh model to obtain a human body surface mesh model; obtaining key points of the joint coordinate system based on the human body surface mesh model and performing automatic filtering of these key points; and establishing a local coordinate system using the filtered key points. Therefore, this invention not only solves the problems of left-right symmetry and temporal continuity in the human body but also achieves data format compatibility with infrared optical motion capture systems, enabling visual motion capture technology to directly interface with existing human motion analysis software, such as OpenSim.

[0027] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the invention and to fully convey the scope of the invention to those skilled in the art.

[0028] Example 1: As Figure 1 As shown, the joint coordinate system construction method based on human body surface mesh model reconstruction provided in this embodiment includes: S1. Acquire human images of the target area.

[0029] In this embodiment, a 640×480 resolution color camera with a frame rate of 30 fps and a field of view of 85 degrees can be used to acquire human images of the target area, ensuring that human images can be fully acquired within a range of 3-8 meters. This is an example, but not limited to this.

[0030] S2. Preprocess the human body image to obtain standard image blocks.

[0031] In this embodiment, a deep learning object detection algorithm is used to identify human body regions in a human body image, and the detected human body regions are cropped into standard image blocks, such as cropping into standard image blocks of size 224×224. This is just one example, and it is not limited to this.

[0032] S3. Input the standard image block into the parameterized human body mesh model for reconstruction to obtain the human body surface mesh model.

[0033] In this embodiment, a parametric human body mesh model based on the SMSLX standard or a custom human body mesh model modified based on the SMSLX standard is used for reconstruction. The SMSLX parametric human body mesh model includes 10 shape parameters used to adjust features such as height and body shape. The topology of the parametric human body mesh model follows the number of vertices and faces defined in the SMSLX standard.

[0034] Furthermore, a parametric human body mesh model can be obtained through a pre-trained deep neural network (such as HMR or SPIN methods). The parametric human body mesh model can map 2D image information to the shape parameter space of a 3D mesh model, thereby achieving accurate reconstruction of the human body surface mesh model. The training process of the parametric human body mesh model will not be elaborated here.

[0035] S4. Based on the reconstructed human body surface mesh model, perform automatic selection of key points in the joint coordinate system.

[0036] In this embodiment, 51 key points in the joint coordinate system are defined, including feature points of major human body parts such as the head, torso, and limbs. Using expert-predefined key point templates in the joint coordinate system, only the 3D coordinates of these key points are retained from the reconstructed human body surface mesh model.

[0037] Furthermore, key points include: key points in the head region, including the vertex of the head and the forehead; key points in the trunk region, including the center point of the neck, the thoracic vertebrae, and the lumbar vertebrae; key points in the upper limbs, including the left and right shoulder joints and the elbow joints; key points in the lower limbs, including the hip joints, the knee joints, and the ankle joints; and key points in the extremities, including the finger joints and the toe joints.

[0038] In this embodiment, the automatic selection of key points in the joint coordinate system includes: The number of vertices and faces of the human body surface mesh model is preset, and the body shape features of the human body surface mesh model are parametrically controlled through shape parameters; specifically, the shape parameters are a set of real-valued vectors, denoted as b=[b1, b2,…, b n The shape parameters [] represent the offset of the human body shape in the statistical modeling space, such as height, shoulder width, limb thickness, and other full-body geometric features. By adjusting these shape parameters, the overall shape of the human body surface mesh model can be continuously adjusted, thereby generating human body surface mesh models of different body shapes.

[0039] The user can customize the key point positions of the joint coordinate system according to their needs through a visual interactive interface. Through the above-mentioned visual interactive page, the expert-predefined joint coordinate system key point template retains only the corresponding key points from the vertices of the reconstructed human body surface mesh model and obtains their three-dimensional coordinates.

[0040] S5. Perform time-series data smoothing on the key points selected.

[0041] In this embodiment, a low-pass filter is used to smooth the trajectory of key points, which can effectively eliminate high-frequency noise and ensure the continuity of the motion trajectory. By adjusting the cutoff frequency, the jitter of the key point position in the joint coordinate system is suppressed while maintaining the motion details.

[0042] S6. Establish a local coordinate system using the key points of the joint coordinate system.

[0043] In this embodiment, a local coordinate system is established using key points of the joint coordinate system, including: Based on anatomical features, the connection relationships between key points are determined to form a complete skeletal structure; for example, the connection between the "left hip joint" and the "left knee joint" can represent the "thigh bone segment". The skeletal structure is usually represented as an ordered graph structure with a set of directed edges connecting key point nodes.

[0044] Based on the key points of the joint coordinate system in the aforementioned skeletal structure, a custom local coordinate system can be further defined at each joint to achieve compatibility with standard motion capture data formats. For example, the direction of the bone segment used to calculate the principal axis (such as the z-axis) of the local coordinate system can be set, and the orthogonal supplementation method used for other axes can be specified. Offsets can be added at key point locations for adaptation to device end effects, clothing simulation, and other applications. Specifically, at a certain joint, a right-handed Cartesian coordinate system is defined, with its origin located at the current joint position. The coordinate axis directions can be calculated from the orientation of adjacent bone segments. For example, the direction from the "hip" to the "knee" can be set as the local z-axis, and the x and y axes can be calculated based on the normals to other bone segments, thereby expressing the local spatial pose.

[0045] Example 2: Following the method for constructing a joint coordinate system based on a human body surface mesh model reconstructed in Example 1, this example provides a device for constructing a joint coordinate system based on a human body surface mesh model reconstructed. The device provided in this example can implement the method for constructing a joint coordinate system based on a human body surface mesh model reconstructed in Example 1. This device can be implemented through software, hardware, or a combination of both. For ease of description, this example is described by dividing the functionality into various units. Of course, in implementation, the functions of each unit can be implemented in one or more software and / or hardware components. For example, the device may include integrated or separate functional modules or units to execute the corresponding steps in the methods of Example 1. Since the device in this example is basically similar to the method example, the description process of this example is relatively simple. For relevant details, please refer to the description in Example 1. The example of the joint coordinate system construction device based on a human body surface mesh model reconstructed by this invention is merely illustrative.

[0046] Specifically, the present invention also provides a joint coordinate system construction device based on human body surface mesh model reconstruction, comprising: The image acquisition unit is configured to acquire human images of the target area; The image processing unit is configured to preprocess human images to obtain standard image blocks; The mesh model reconstruction unit is configured to reconstruct standard image patches based on a parametric human body mesh model to obtain a human body surface mesh model. The key point filtering unit is configured to obtain key points in the joint coordinate system based on the reconstructed human body surface mesh model and perform automatic filtering of key points in the joint coordinate system. The coordinate system reconstruction unit is configured to establish a local coordinate system using the selected joint key points.

[0047] In a preferred embodiment of the present invention, the image acquisition unit can be deployed at a preset location in actual use, including but not limited to: a fixed bracket installed on the ceiling in an indoor environment, an adjustable tripod in a ground environment, or a drone, robotic arm, or sliding rail system in a mobile environment. This ensures that its field of view covers the target movement area, enabling it to acquire images of the human body during movement and transmit data via wired or wireless communication.

[0048] Furthermore, the image acquisition unit can be implemented using a color camera sensor, lens, computing acquisition unit, supplementary light, and battery, and has independent acquisition and data transmission capabilities.

[0049] In a preferred embodiment of the present invention, a time synchronization unit is further included, which is used to establish time synchronization of the image acquisition unit, the mesh model reconstruction unit, the key point screening unit and the coordinate system reconstruction unit.

[0050] In a preferred embodiment of the present invention, a visual interactive interface is also provided for customizing key points of the joint coordinate system. This interface supports the display and interactive control of the human body surface mesh model, provides manual annotation functionality for key points of the joint coordinate system, and has a mechanism for storing and reusing annotation data, including: The view control panel is used to adjust the rotation, scaling, and translation parameters of the human body surface mesh model; The key point annotation panel is used to select specific vertices on the human body surface mesh model through mouse interaction, and to name and store the vertices; The coordinate system display panel is used for real-time preview and verification of the positions of labeled joint coordinate system key points.

[0051] In summary, this invention provides a clear geometric definition for the joint coordinate system by introducing human body surface mesh model reconstruction technology, and by combining the left and right symmetry constraints of the human body, it significantly improves the accuracy and reliability of joint coordinate system reconstruction. This device can flexibly define different joint coordinate systems, effectively expanding the application scope of visual motion capture systems.

[0052] Example 3: This example provides an electronic device corresponding to the joint coordinate system construction method based on human body surface mesh model reconstruction provided in Example 1. The electronic device can be an electronic device for client use, such as a mobile phone, laptop, tablet computer, desktop computer, etc., to execute the method of Example 1.

[0053] like Figure 2 As shown, the electronic device includes a processor, a memory, a communication interface, and a bus. The processor, memory, and communication interface are connected via the bus to enable communication between them. The memory stores a computer program that can run on the processor. When the processor runs the computer program, it executes the method of Embodiment 1. The implementation principle and technical effects are similar to those of Embodiment 1, and will not be repeated here. Those skilled in the art will understand that... Figure 2 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computing device on which the present application is applied. The specific computing device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0054] In a preferred embodiment, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), and optical discs.

[0055] In a preferred embodiment, the processor can be any type of general-purpose processor such as a central processing unit (CPU) or a digital signal processor (DSP), and is not limited thereto.

[0056] Example 4: This example provides a computer-readable storage medium for storing one or more programs, the one or more programs including computer instructions, which, when executed by a computer, cause the computer to perform the method provided in Example 1 above.

[0057] In a preferred embodiment, the computer-readable storage medium may be a tangible device for holding and storing instructions executable, such as, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof. The computer-readable storage medium stores computer program instructions that cause a computer to perform the method provided in Embodiment 1 above.

[0058] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0059] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0060] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0061] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In the description of this specification, the terms "a preferred embodiment," "furthermore," "specifically," "in this embodiment," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments in this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a joint coordinate system based on the reconstruction of a human body surface mesh model, characterized in that, include: Acquire human images of the target area; Preprocessing of human body images yields standard image blocks; Standard image patches are reconstructed based on a parametric human body mesh model to obtain a human body surface mesh model; Key points of the joint coordinate system are obtained based on the human body surface mesh model, and automatic filtering of key points of the joint coordinate system is performed. A local coordinate system is established using the key points of the joint coordinate system obtained through screening.

2. The method for constructing a joint coordinate system based on human body surface mesh model reconstruction according to claim 1, characterized in that, The human body image is preprocessed to obtain standard image patches, specifically: A deep learning object detection algorithm is used to identify human body regions in human images, and the detected human body regions are cropped into standard image patches that meet the requirements.

3. The method for constructing a joint coordinate system based on human body surface mesh model reconstruction according to claim 1, characterized in that, Parametric human body mesh models are used to map 2D image information to the shape parameter space of a 3D mesh model. The parametric human body mesh model adopts the parametric human body mesh model of the SMPLX standard or a custom human body mesh model modified based on the SMPLX standard.

4. The method for constructing a joint coordinate system based on human body surface mesh model reconstruction according to claim 1, characterized in that, Joint coordinate system keypoints are used to define feature points of major parts of the human body. These keypoints include head region keypoints, trunk region keypoints, upper limb keypoints, lower limb keypoints, and / or distal end keypoints.

5. The method for constructing a joint coordinate system based on human body surface mesh model reconstruction according to claim 1, characterized in that, Perform automatic filtering of key points in the joint coordinate system, including: The number of vertices and faces of the human body surface mesh model is preset, and the body shape features of the human body surface mesh model are parametrically controlled through shape parameters, where the shape parameters are a set of real-valued vectors, denoted as b=[b1, b2, …, b n The shape parameter ] is used to represent the offset of the human body shape in the statistical modeling space. By adjusting this set of shape parameters, the overall shape of the human body mesh model can be continuously adjusted, thereby generating human body surface mesh models of different body shapes. Customize the key point positions of the joint coordinate system according to requirements; Based on the expert-defined joint coordinate system key point template, only the corresponding key points are retained from the vertices of the human body surface mesh model, and their three-dimensional coordinates are obtained.

6. The method for constructing a joint coordinate system based on human body surface mesh model reconstruction according to claim 1, characterized in that, It also includes a step of smoothing the time-series data of the selected joint coordinate system key points. Specifically, a low-pass filter is used to smooth the trajectory of the joint coordinate system key points. By adjusting the cutoff frequency, the jitter of the joint coordinate system key point positions is suppressed while maintaining motion details.

7. The method for constructing a joint coordinate system based on human body surface mesh model reconstruction according to claim 1, characterized in that, Establish a local coordinate system using the selected joint coordinate system key points, including: Based on anatomical features, the connection relationships between key points in the joint coordinate system are determined to form a complete skeleton structure. The skeleton structure is represented as an ordered graph structure with a set of directed edges connecting the key point nodes of the joint coordinate system. Based on the key points of the joint coordinate system in the above skeleton structure, a local coordinate system can be defined at each joint. Specifically, at a certain joint, a right-handed Cartesian coordinate system is defined, with its origin located at the current joint position. The coordinate axis direction is obtained by calculating the orientation of adjacent bone segments, thereby realizing the expression of local spatial posture.

8. A joint coordinate system construction device based on human body surface mesh model reconstruction, characterized in that, include: The image acquisition unit is configured to acquire human images of the target area; The image processing unit is configured to preprocess human images to obtain standard image blocks; The mesh model reconstruction unit is configured to reconstruct standard image patches based on a parametric human body mesh model to obtain a human body surface mesh model. The key point filtering unit is configured to obtain key points in the joint coordinate system based on the reconstructed human body surface mesh model and perform automatic filtering of key points in the joint coordinate system. The coordinate system reconstruction unit is configured to establish a local coordinate system using the selected joint key points.

9. An electronic device, characterized in that, include: At least one processor; And a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform the method according to any one of claims 1-7.

10. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include computer instructions for causing a computer to perform the method according to any one of claims 1-7.

Images