A model training method and device for extracting human skeleton features

Abstract

The present application provides a model training method and device for extracting human skeleton features. The method includes: obtaining original computed tomography (CT) data of human skeletons of different target objects belonging to the same group for target parts; reconstructing three-dimensional models of target objects for target parts based on the original CT data, and obtaining model files corresponding to each three-dimensional model; extracting point cloud data of corresponding three-dimensional models from each model file, and batch generating label data of each voxel in the corresponding three-dimensional model; using the point cloud data of all three-dimensional models and the label data of each voxel in all three-dimensional models as training sets, training a deep learning three-dimensional model that introduces dynamic graph convolution, until a preset stop condition is met, and obtaining a human skeleton landmark point recognition model. The technical solution provided by the present application can improve the accuracy of the human skeleton landmark point recognition model in identifying human skeleton landmark points.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of medical image processing technology, and in particular to a model training method and device for extracting human skeleton features. Background Art

[0002] With the continuous development of medical technology, human skeleton landmark recognition has important application value in the fields of medical image analysis and orthopedic surgery planning. In many medical image processing tasks, accurately identifying and locating human skeleton landmarks is crucial for correct diagnosis and treatment. However, due to the complexity of human skeleton structure and factors such as noise and deformation in images, traditional automatic landmark recognition methods and manual selection methods have certain limitations.

[0003] Currently, there are many methods for extracting and identifying human bone features from computed tomography (CT) data, such as deep learning-based methods.

[0004] However, existing methods lack the processing of local details, so there are still some challenges in terms of accuracy, robustness and efficiency. Summary of the invention

[0005] The embodiments of the present application provide a model training method and device for extracting human skeleton features, so as to solve the problem of low accuracy in identifying human skeleton features in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a model training method for extracting human skeleton features, comprising:

[0007] Acquire original computed tomography (CT) data of human bones of different target subjects belonging to the same group at target locations;

[0008] Reconstructing a three-dimensional model of a target part of each target object based on the original CT data to obtain a model file corresponding to each three-dimensional model;

[0009] Extracting point cloud data corresponding to the three-dimensional model from each of the model files, and generating label data of each voxel in the corresponding three-dimensional model in batches; wherein the label data includes identification data indicating whether the voxel is a human skeleton landmark point, and if the identification data indicates that the voxel is a human skeleton landmark point, the label data also includes: position data of the labeled human skeleton landmark point, labeled depth data, labeled area size and labeled morphological information;

[0010] The point cloud data of all three-dimensional models and the label data of each voxel in all three-dimensional models are used as training sets, and an improved deep learning three-dimensional model is trained until a preset stop condition is met, so as to obtain a human skeleton landmark point recognition model, wherein the improved deep learning three-dimensional model is a deep learning three-dimensional model that introduces dynamic graph convolution, and the human skeleton landmark point recognition model is used to extract human skeleton features and identify human skeleton landmark points.

[0011] Optionally, the improved deep learning three-dimensional model includes a sampling layer, an improved local area division layer, a local feature extraction layer, a global feature extraction layer and an output layer connected in sequence; wherein the improved local area division layer is a local area division layer that introduces dynamic graph convolution;

[0012] The sampling layer selects a plurality of acquisition points from the point cloud data of the three-dimensional model, wherein the plurality of acquisition points are evenly distributed in the three-dimensional space;

[0013] The improved local area division layer uses a minimum bounding sphere fitting method to group points around each acquisition point to form a local area; obtains a three-dimensional human skeleton template corresponding to the group to which the target object belongs, and if the similarity between the multiple acquisition points and multiple human skeleton landmarks in the three-dimensional human skeleton template is less than a preset threshold, a dynamic graph convolution operation is performed on the local area to obtain updated node features of the local area;

[0014] The local feature extraction layer extracts low-level features in each local area based on the updated node features of each local area through a multi-layer perceptron, and aggregates the low-level features in each local area into a global feature vector through a pooling operation;

[0015] The global feature extraction layer extracts global features from the global feature vector;

[0016] The output layer maps the global features to predicted data of human skeleton landmarks, wherein the predicted data includes: predicted position data of human skeleton landmarks, predicted depth data, predicted area size and predicted morphological information.

[0017] Optionally, the points around the acquisition point are grouped using a minimum bounding sphere fitting method to form a local area, including:

[0018] For the collection point, an initial radius is set;

[0019] Based on the initial radius, the minimum bounding sphere fitting method is used to group the points around the collection point to obtain a point set;

[0020] A dynamic radius adjustment strategy is introduced, so that when the number of acquisition points in the point set is less than a preset minimum point number threshold, the points around the acquisition points are grouped by using a minimum bounding sphere fitting method based on an increased radius; when the number of acquisition points in the point set is greater than a preset maximum point number threshold, the points around the acquisition points are grouped by using a minimum bounding sphere fitting method based on a reduced radius; the increased radius is greater than the initial radius, and the reduced radius is less than the initial radius;

[0021] When the number of collected points in the point set is greater than or equal to a preset minimum point number threshold and less than or equal to a maximum point number threshold, a local area is determined based on the point set.

[0022] Optionally, the dynamic radius adjustment strategy includes a dynamic radius adjustment strategy based on point cloud density, and the dynamic radius adjustment strategy adopts the following formula:

[0023] ;

[0024] in, is the increased radius, is the current radius. When the dynamic radius is adjusted for the first time, the current radius is the initial radius. is the adjustment factor, are weights of different scales, is a set of points The point cloud density at scale s, , is a set of points The number of collection points at scale s, is a set of points The volume on scale s, is the average density of all points on scale s.

[0025] Optionally, the step of grouping points around the acquisition point using a minimum bounding sphere fitting method based on the initial radius to obtain a point set includes:

[0026] Taking the collection point as the initial sphere center, and based on the initial sphere center and the initial radius, select all points whose distance from the initial sphere center is less than or equal to the initial radius as points around the collection point, and store the points around the collection point and the collection point together in an empty set to obtain a set to be updated;

[0027] Construct the objective function used by the minimum bounding sphere fitting method, and minimize the objective function based on the set to be updated to find the optimized sphere center and radius;

[0028] Based on the optimized sphere center and radius, all points whose distances to the optimized sphere center are less than or equal to the optimized radius are selected, and all points whose distances to the optimized sphere center are less than or equal to the optimized radius are added to the set to be updated to obtain a point set.

[0029] Optionally, the minimum bounding sphere fitting method includes a weighted minimum bounding sphere fitting method; the objective function adopted by the weighted minimum bounding sphere fitting method is:

[0030] ;

[0031] Among them, J1 is the objective function used by the minimum bounding sphere fitting method, It is i The coordinates of the points, is the center of the smallest enclosing sphere, is the radius of the minimum enclosing sphere, It is i The weight of the point, n is the number of collection points in the set to be updated.

[0032] Optionally, the minimum bounding sphere fitting method includes a multi-objective optimization method; the objective function adopted by the multi-objective optimization method is:

[0033] ;

[0034] Among them, J2 is the objective function used in the multi-objective optimization method, is the weight that minimizes the radius, is the weight of the uniformity of the distribution of the collection points, k is the number of iterations, It is i The coordinates of the points, is the center of the smallest enclosing sphere, is the radius of the minimum enclosing sphere, n is the number of collection points in the set to be updated.

[0035] Optionally, after the improved local area division layer obtains the three-dimensional human skeleton template corresponding to the group to which the target object belongs, the method further includes:

[0036] If the similarity between the multiple acquisition points and the multiple human skeleton landmark points in the human skeleton three-dimensional template is greater than or equal to a preset threshold, the feature information of the multiple human skeleton landmark points in the human skeleton three-dimensional template is determined as the updated node features of the local area.

[0037] Optionally, the original CT data includes original image data and metadata, and the metadata includes target object information and scanning parameters; the target object information includes: object number and human skeleton history data;

[0038] The reconstructing of the three-dimensional model of the target part of each target object based on the original CT data to obtain the model file corresponding to each three-dimensional model includes:

[0039] For the original CT data of the human skeleton at the target part of each target object, the following reconstruction process is performed:

[0040] Preprocessing the original image data to obtain preprocessed image data;

[0041] Based on the preprocessed image data, the target part is separated by using a threshold segmentation or a region growing algorithm to segment the image;

[0042] Converting data related to the target part in the preprocessed image data into voxel data;

[0043] Generate a surface mesh of a three-dimensional model of a target part of the target object using a surface reconstruction method based on the voxel data;

[0044] The three-dimensional model is optimized based on the metadata to obtain a model file corresponding to the three-dimensional model.

[0045] In a second aspect, an embodiment of the present application provides a model training device for extracting human skeleton features, comprising:

[0046] An acquisition module, used for acquiring original CT data of human bones of target parts of different target objects belonging to the same group;

[0047] A reconstruction module, used to reconstruct a three-dimensional model of a target part of each target object based on the original CT data, and obtain a model file corresponding to each three-dimensional model;

[0048] An extraction and generation module is used to extract point cloud data corresponding to the three-dimensional model from each of the model files, and batch generate label data of each voxel in the corresponding three-dimensional model; wherein the label data includes identification data indicating whether the voxel is a human skeleton landmark point, and if the identification data indicates that the voxel is a human skeleton landmark point, the label data also includes: position data of the labeled human skeleton landmark point, labeled depth data, labeled area size and labeled morphological information;

[0049] A training module is used to use the point cloud data of all three-dimensional models and the label data of each voxel in all three-dimensional models as a training set, train the improved deep learning three-dimensional model until the preset stopping conditions are met, and obtain a human skeleton landmark point recognition model, wherein the improved deep learning three-dimensional model is a deep learning three-dimensional model that introduces dynamic graph convolution, and the human skeleton landmark point recognition model is used to extract human skeleton features and identify human skeleton landmark points.

[0050] In an embodiment of the present application, a model training method for extracting human skeleton features is provided, comprising: obtaining original computed tomography (CT) data of human skeletons at target parts of different target objects belonging to the same group; reconstructing three-dimensional models of target parts of each target object based on the original CT data to obtain model files corresponding to each three-dimensional model; extracting point cloud data of the corresponding three-dimensional model from each model file, and batch generating label data of each voxel in the corresponding three-dimensional model; wherein the label data comprises identification data indicating whether the voxel is a human skeleton landmark point, and if the identification data indicates that the voxel is a human skeleton landmark point, the label data further comprises: position data of the labeled human skeleton landmark point, labeled depth data, labeled area size and labeled morphological information; taking the point cloud data of all three-dimensional models and the label data of each voxel in all three-dimensional models as training sets, training an improved deep learning three-dimensional model until a preset stop condition is met to obtain a human skeleton landmark point recognition model, wherein the improved deep learning three-dimensional model is a deep learning three-dimensional model that introduces dynamic graph convolution, and the human skeleton landmark point recognition model is used to extract human skeleton features and recognize human skeleton landmark points.

[0051] The embodiments of the present application select data from the same group and high-resolution original CT data to ensure the homogeneity and integrity of the data, thereby improving the generalization ability and recognition accuracy of the improved deep learning 3D model; generate high-quality 3D models by using 3D reconstruction technology, thereby providing a reliable basis for subsequent point cloud data extraction and label generation; batch generate label data containing position data, depth data, area size and morphological information, thereby providing a more comprehensive feature description, which helps to improve the recognition accuracy and robustness of the improved deep learning 3D model; and introduce a deep learning model with dynamic graph convolution to effectively process the irregularity and local details of point cloud data, thereby improving the model's recognition ability for complex geometric structures, thereby improving the accuracy of identifying human skeletal landmarks.

[0052] These and other aspects of the present application will become more clearly understood in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0054] Figure 1 A flow chart of a model training method for extracting human skeleton features provided in an embodiment of the present application;

[0055] Figure 2 A schematic diagram of the structure of an improved deep learning three-dimensional model provided in an embodiment of the present application;

[0056] Figure 3 A schematic diagram of the structure of a model training device for extracting human skeleton features provided in an embodiment of the present application;

[0057] Figure 4 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0059] With the continuous development of medical technology, human skeleton landmark recognition has important application value in the fields of medical image analysis and orthopedic surgery planning. In many medical image processing tasks, accurately identifying and locating human skeleton landmarks is crucial for correct diagnosis and treatment. However, due to the complexity of human skeleton structure and factors such as noise and deformation in images, traditional automated human skeleton landmark recognition methods and manual selection methods have certain limitations.

[0060] At present, some methods based on computer vision and machine learning have been proposed for the automatic recognition of human skeleton landmarks. These existing methods generally include steps such as feature extraction, classifier design and matching algorithm. However, the existing methods still have some challenges in terms of accuracy, robustness and efficiency. In order to improve the accuracy of human skeleton feature recognition, the present application embodiment proposes a model training method and device for extracting human skeleton features, which effectively processes the irregularity and local details of point cloud data by introducing a deep learning model of dynamic graph convolution, improves the recognition ability of the model to complex geometric structures, and then improves the accuracy of identifying human skeleton landmarks.

[0061] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel. The sequence numbers of the operations, such as 11, 12, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to different types.

[0062] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0063] Figure 1 A flow chart of a model training method for extracting human skeleton features provided in an embodiment of the present application, such as Figure 1 As shown, the method includes:

[0064] S11, obtaining original computed tomography (CT) data of human bones at target locations of different target objects belonging to the same group. Computed tomography, also known as CT scan or CAT scan, is a medical imaging technology that generates detailed cross-sectional images of the human body by using X-rays and computer algorithms.

[0065] It should be understood that belonging to the same group refers to a target object with similar characteristics, and the target object may refer to a human body. These objects have commonalities in some aspects, so that their data can be put together for analysis and modeling. Common similar characteristics of belonging to the same group include: age, gender, health status, living area, exercise status, occupation, and lifestyle. Similar characteristics may be one or more. When the similar characteristics are one, for example, four groups can be divided according to age: children's group (0 to 12 years old), adolescent group (13 to 18 years old), adult group (19 to 60 years old) and elderly group (61 years old and above). This embodiment can be divided into two groups according to gender: male group and female group. This embodiment can be divided into healthy people, people with specific diseases (such as osteoporosis, arthritis, etc.) according to health status. This embodiment can divide people by region according to living area. This embodiment can divide people by occupation type according to occupation. This embodiment can divide people according to lifestyle such as exercise status and whether they smoke.

[0066] It should also be understood that in the application of human skeleton feature extraction and human skeleton landmark point recognition, the target part can cover multiple skeletal parts of the human body, and common target parts include but are not limited to: knee joint, hip joint, spine, shoulder joint, wrist joint, ankle joint, foot, foot, skull. Among them, the knee joint includes the distal femur, proximal tibia, patella, etc. The hip joint includes the femoral head, femoral neck, acetabulum, etc. The spine includes the cervical vertebrae, thoracic vertebrae, lumbar vertebrae, sacral vertebrae, etc. The shoulder joint includes the humeral head, scapula, clavicle, etc. The wrist joint includes the distal radius, distal ulna, carpal bones, etc. The ankle joint includes the distal tibia, distal fibula, talus, etc. The hand includes metacarpal bones, phalanges, etc. The foot includes metatarsal bones, phalanges, etc. The skull includes the frontal bone, parietal bone, occipital bone, etc. Human skeleton landmarks are also called skeletal anatomical landmarks, landmarks, etc.

[0067] The original CT data includes original image data and metadata. The original image data refers to the two-dimensional slice image data generated by CT scanning. Each image represents the X-ray absorption of a certain cross section of the human body. Metadata is additional information related to image data, including target object information and scanning parameters; target object information includes: object number and human skeleton history data, and target object information is used to identify and record the information of each target object. Scanning parameters include the model of the scanning equipment, scanning parameter settings (such as voltage, current, layer thickness, etc.), which are used to ensure data consistency and traceability.

[0068] S12. Reconstructing a three-dimensional model of each target object for a target part based on the original CT data to obtain a model file corresponding to each three-dimensional model.

[0069] It should be understood that a 3D model is a 3D geometric model generated by processing the original CT data containing the original image data, which can intuitively display the structure of the human skeleton. A model file stores a file of a 3D model, which usually contains geometric information (such as vertices, edges, and faces) and texture information (such as color and material).

[0070] S13. Extract the point cloud data of the corresponding three-dimensional model from each model file, and batch generate label data for each voxel in the corresponding three-dimensional model; wherein the label data includes identification data indicating whether the voxel is a human skeleton landmark point. If the identification data indicates that the voxel is a human skeleton landmark point, the label data also includes: position data of the marked human skeleton landmark point, marked depth data, marked area size and marked morphological information.

[0071] It should be understood that the three-dimensional model uses voxels as the basic unit, and each voxel represents a three-dimensional pixel point in the three-dimensional image. Each voxel contains X-ray absorption information about the area, and detailed information about tissues, organs and bones can be obtained through the point cloud data and label data corresponding to the voxel. Point cloud data is a collection of data consisting of a large number of three-dimensional points (or voxels). The data of a three-dimensional point contains three-dimensional coordinates (x, y, z) and other attributes (such as color, normal direction, etc.), which are used to represent the surface shape of the target object for the target part. In the label data, the identification data indicates whether the voxel is a human skeleton landmark point. The position data indicates the three-dimensional coordinates of the annotated human skeleton landmark point. The depth data indicates the depth information of the annotated human skeleton landmark point. The area size indicates the area of ​​the annotated human skeleton landmark point. The morphological information indicates the morphological features of the annotated human skeleton landmark point (such as curvature, normal direction, etc.). The batch generation method can be implemented based on a template matching algorithm, based on deep learning, and / or based on a rule-based automatic labeling method, which is not specifically limited in the embodiments of the present application. The batch generation method can significantly reduce the time and workload of manual annotation and improve the efficiency of data preparation. Therefore, this embodiment can quickly generate a large amount of annotated data and accelerate the model training and verification process.

[0072] S14. Use the point cloud data of all three-dimensional models and the label data of each voxel in all three-dimensional models as a training set, train the improved deep learning three-dimensional model until the preset stopping conditions are met, and obtain a human skeleton landmark point recognition model, wherein the improved deep learning three-dimensional model is a deep learning three-dimensional model that introduces dynamic graph convolution, and the human skeleton landmark point recognition model is used to extract human skeleton features and identify human skeleton landmark points.

[0073] It should be understood that the deep learning 3D model uses a recognition algorithm based on deep learning and point clouds, which is used to handle the analysis and processing tasks of point cloud data. Unlike traditional image-based deep learning methods, the deep learning 3D model is suitable for processing disordered point cloud data in three-dimensional space and is often used for point cloud component segmentation, for example, segmenting human bone landmarks from the target part.

[0074] It should also be understood that the present embodiment reconstructs the target part into a three-dimensional model based on the original CT data. Point sequences are extracted from the three-dimensional model to form a point cloud. In the subsequent process, the local area of ​​the human skeleton landmarks can be fitted based on the trained improved deep learning three-dimensional model, and then local feature extraction, global feature extraction, prediction and other operations are performed to obtain the predicted data of the human skeleton landmarks.

[0075] The embodiments of the present application select data from the same group and high-resolution original CT data to ensure the homogeneity and integrity of the data, thereby improving the generalization ability and recognition accuracy of the improved deep learning 3D model; generate high-quality 3D models by using 3D reconstruction technology, thereby providing a reliable basis for subsequent point cloud data extraction and label generation; batch generate label data containing position data, depth data, area size and morphological information, thereby providing a more comprehensive feature description, which helps to improve the recognition accuracy and robustness of the improved deep learning 3D model; and introduce a deep learning model with dynamic graph convolution to effectively process the irregularity and local details of point cloud data, thereby improving the model's recognition ability for complex geometric structures, thereby improving the accuracy of identifying human skeletal landmarks.

[0076] In some possible embodiments, the original CT data includes original image data and metadata, and the metadata includes target object information and scanning parameters; the target object information includes: object number and human skeleton history data. Therefore, S12, based on the original CT data, reconstructs the three-dimensional model of each target object for the target part to obtain the model file corresponding to each three-dimensional model, which can be achieved through the following exemplary reconstruction process. That is: for the original CT data of the human skeleton of each target object for the target part, the following reconstruction process is performed:

[0077] Step 121: pre-process the original image data to obtain pre-processed image data.

[0078] It should be understood that preprocessing includes but is not limited to: denoising, normalization, resampling, etc., wherein denoising can remove noise data in the original image data, thereby improving the image quality. Normalization refers to normalizing the grayscale value of the original image data to a fixed range, such as 0 to 1. Resampling refers to adjusting the image resolution. Exemplarily, resampling can adjust the image resolution to 0.5mm x 0.5mm x 0.5mm.

[0079] Step 122: Based on the preprocessed image data, use a threshold segmentation or a region growing algorithm to separate the target part to segment the image.

[0080] It should be understood that the threshold segmentation algorithm can divide the image into two parts, the target part and the background, according to the gray value threshold and the preprocessed image data. The region growing algorithm starts from one or more seed points and gradually expands to adjacent similar regions until the corresponding stop condition is met. The above two algorithms can achieve the separation of the target part from the image.

[0081] Exemplarily, a grayscale value threshold of 150 is selected to separate the knee joint area from the background.

[0082] Step 123: Convert the data related to the target part in the preprocessed image data into voxel data.

[0083] It should be understood that voxel is a basic unit in three-dimensional space, similar to a pixel in a two-dimensional image, and each voxel data includes three-dimensional coordinates and a grayscale value.

[0084] Step 124: Generate a surface mesh of a three-dimensional model of a target part of the target object using a surface reconstruction method based on the voxel data.

[0085] It should be understood that the surface mesh of a three-dimensional model refers to a polygonal mesh used to represent the geometric shape of the surface of human bones. It is a data structure composed of vertices, edges, and faces, which is used to accurately describe the external contours and surface features of human bones. The following is a detailed explanation of these components: Vertices are the basic elements that make up the polygonal mesh, and each vertex has a specific three-dimensional position, which is used to define the geometric shape of the bone surface. Edges are used to define the connectivity of the polygonal mesh, connecting vertices to form polygonal faces. Faces are used to represent local areas on the surface of human bones, and multiple faces are combined to form a complete surface mesh.

[0086] In the present application embodiment, the surface mesh can be used for three-dimensional visualization to help doctors and researchers intuitively observe and analyze the structure of the human skeleton. The surface mesh can be used to extract the geometric features of the skeleton, such as curvature, normal direction, etc., for further analysis and modeling. The surface mesh can be used for surgical planning to help doctors formulate personalized surgical plans to improve the accuracy and success rate of the operation. The surface mesh can also be used for biomechanical simulation and simulation to study the behavior and response of the human skeleton under different conditions.

[0087] Step 125: Optimize each 3D model based on the metadata to obtain a model file corresponding to each 3D model.

[0088] This embodiment uses the target object information and scanning parameters in the metadata to optimize the generated 3D model, which can improve the accuracy and completeness of the 3D model. For example, the smoothness and details of the 3D model can be adjusted according to the patient's history of bone hyperplasia.

[0089] It should be noted that the accurate construction of the human skeleton landmark recognition model seriously affects the recognition accuracy of the human skeleton landmarks. Therefore, in order to improve the recognition accuracy of the human skeleton landmarks, the present application embodiment constructs an improved deep learning three-dimensional model. In some possible embodiments, such as Figure 2 As shown, the improved deep learning three-dimensional model includes a sampling layer 21, an improved local area division layer 22, a local feature extraction layer 23, a global feature extraction layer 24 and an output layer 25 connected in sequence; wherein the improved local area division layer 22 is a local area division layer that introduces dynamic graph convolution.

[0090] In an optional embodiment, the improved deep learning 3D model may include a set of sampling layers 21, an improved local area division layer 22 and a local feature extraction layer 23. A set of sampling layers 21, an improved local area division layer 22 and a local feature extraction layer 23 may constitute a feature extraction module.

[0091] In another optional embodiment, the improved deep learning 3D model can introduce a hierarchical structure, that is, it includes multiple groups of sampling layers 21, improved local area division layers 22 and local feature extraction layers 23. The hierarchical structure indicates that the improved deep learning 3D model realizes layer-by-layer sampling, division and aggregation operations through multiple feature extraction modules, and extracts the features of point cloud data from coarse to fine granularity. A more global feature representation is constructed by utilizing the features of the local area. Therefore, the improved deep learning 3D model improves the feature extraction and representation capabilities of point cloud data by introducing a hierarchical structure.

[0092] In the improved deep learning 3D model, the feature extraction module is used to extract local features from point cloud data. Each feature extraction module can be divided into three stages: sampling, partitioning, and aggregation. The sampling stage performs a sampling operation, the purpose of which is to select some acquisition points (or key points, center points, etc.) from the current point set, and for each sampling point, a set of representative points is selected as the local area of ​​the sampling point by using the minimum bounding sphere fitting method (i.e., spherical neighborhood query method). In the aggregation stage, a multi-layer perceptron can be used to extract the features of points in the local area.

[0093] The following is a detailed description of the functions of each layer:

[0094] The sampling layer 21 selects a plurality of acquisition points from the point cloud data of the three-dimensional model, wherein the plurality of acquisition points are evenly distributed in the three-dimensional space. The sampling layer 21 can ensure that the acquisition points are evenly distributed in the three-dimensional space, avoid oversampling or undersampling in local areas, and improve the robustness of the improved deep learning three-dimensional model.

[0095] The improved local area division layer 22 uses the minimum bounding sphere fitting method to group the points around the collection point for each collection point to form a local area; obtain the human skeleton three-dimensional template corresponding to the group to which the target object belongs, and if the similarity between multiple collection points and multiple human skeleton landmark points in the human skeleton three-dimensional template is less than a preset threshold, a dynamic graph convolution operation is performed on the local area to obtain the updated node features of the local area. It should be understood that the preset threshold can be 90%, 92%, 95%, etc., and the specific value of the preset threshold is not specifically limited in the embodiment of the present application. Dynamic graph convolution can capture the local features and geometric structures of point cloud data, especially in irregular point cloud data. The dynamic graph convolution operation can capture features of different scales, and enhance the expression ability of the model through the fusion of multi-scale features. Through dynamic graph convolution, the improved deep learning three-dimensional model can better understand the geometric relationships and features of local areas.

[0096] On the contrary, if the similarity between the multiple acquisition points and the multiple human skeleton landmarks in the human skeleton three-dimensional template is greater than or equal to the preset threshold, the feature information of the multiple human skeleton landmarks in the human skeleton three-dimensional template is determined as the updated node feature of the local area. In this case, the embodiment of the present application utilizes the feature information of the human skeleton landmarks in the human skeleton three-dimensional template, skips the dynamic graph convolution, and reduces unnecessary computational overhead.

[0097] The local feature extraction layer 23 extracts low-level features in each local area based on the updated node features of each local area through a multi-layer perceptron, and aggregates the low-level features in each local area into a global feature vector through a pooling operation.

[0098] The global feature extraction layer 24 extracts global features from the global feature vector.

[0099] The output layer 25 maps the global features to the predicted data of the human skeleton landmarks, and the predicted data includes: the predicted position data of the human skeleton landmarks, the predicted depth data, the predicted area size and the predicted morphological information.

[0100] The embodiment of the present application enables the improved deep learning three-dimensional model to better capture local features and geometric structures and improve the accuracy of feature extraction through dynamic graph convolution operations. Through the setting of similarity thresholds, the improved deep learning three-dimensional model can adaptively choose whether to perform dynamic graph convolution operations, reduce computational overhead, and improve the robustness of the improved deep learning three-dimensional model. The fusion of multi-scale features and the use of multi-layer perceptrons enhance the expressive power of the improved deep learning three-dimensional model and can handle complex geometric structures and features. Through the setting of similarity thresholds, the model can adaptively choose whether to perform dynamic graph convolution operations, which improves the flexibility and adaptability of the improved deep learning three-dimensional model. The ability of multi-task learning enables the improved deep learning three-dimensional model to predict multiple attributes at the same time, improving the versatility and practicality of the improved deep learning three-dimensional model.

[0101] Based on the above design of the improved deep learning 3D model, in order to retain the detailed information of local features, the improved deep learning 3D model also introduces upsampling and fusion operations. The upsampling operation restores the detailed information by mapping the coarser features to the positions of the original input points. The fusion operation fuses features from different resolutions to obtain richer feature representations.

[0102] In order to deal with the disorder of point cloud data, this embodiment can introduce an operation called a perception function in the improved deep learning 3D model. In this embodiment, the perception function is a permutation invariant function (PIF). The perception function aggregates the feature representation of the input point cloud data to generate a global feature representation. In this way, the improved deep learning 3D model can ensure that when processing any arrangement of point cloud data, it will not be affected by changes in the order of points.

[0103] Through the above hierarchical structure, upsampling and fusion operations, and the design of perception functions, the improved deep learning 3D model can effectively process point cloud data and extract rich feature representations. In the task of human skeletal landmark recognition, the improved deep learning 3D model can learn the local and global features in the point cloud data, and then accurately locate and identify human skeletal landmarks, providing more accurate medical image analysis and surgical planning support.

[0104] The embodiment of the present application takes the target part as the femur, and the human bone landmarks as the anatomical landmarks on the femur as an example, to provide a specific example. The main function of the improved deep learning three-dimensional model is to segment the local area and finally identify the anatomical landmarks on the femur. In the embodiment of the present application, the acquisition point (here refers to the point located on the bone surface and may serve as an anatomical landmark) can be used as the center of the sphere and a sphere with a radius of 15 mm. The bone surface area within the range of the sphere is determined as a local area, and the local area can be segmented out by the improved deep learning three-dimensional model. After segmentation, this embodiment can obtain the center of the sphere by fitting the minimum enclosing sphere, and the final required anatomical landmarks are obtained. The following is the specific training process of the improved deep learning three-dimensional model:

[0105] (1) Generate the data required for improved deep learning 3D model training.

[0106] The data required for the improved deep learning three-dimensional model training are point cloud data and label data, and the point cloud data is obtained based on CT data. CT data can present tissues of different densities, such as bones, muscles, fat and organs, etc. Therefore, the present embodiment can reconstruct the three-dimensional model of the femur based on CT data. The threshold segmentation method and the manual outlining method can be used in the reconstruction process. After the reconstruction is completed, the femur model file can be obtained. The present application embodiment can make a total of about 100 femur model files. The femur model file can contain the data content such as the geometric data, topological structure, point attributes, unit attributes of the model, among which the point attributes are the key to the point cloud file. The three-dimensional model of the femur enables doctors to make accurate diagnosis and treatment planning.

[0107] After obtaining the femur model file, the present application embodiment can extract point cloud data. The present embodiment can adopt the mode that the points on the model surface are resampled, and 10000 points are sampled as the original point cloud. Subsequently, it is necessary to make the label file corresponding to each point. According to the femur model file obtained in the previous step, the required marking points can be marked on the femur model, and the coordinates of the required marking points are recorded as P1, P2, ..., Pn, n=1000 respectively. The present embodiment can calculate the distance between the femur point cloud and the marking points, and the femur point cloud less than 15mm constitutes a group of point clouds, which are Label1, ..., Labelm, m is the number of point clouds, and is labeled, 1, 2, ..., m, and the point cloud data corresponding to the point cloud and the label data are stored as txt format for constituting the required data for training.

[0108] (2) Based on the data required for training, train the improved deep learning 3D model.

[0109] The embodiment of the present application performs initialization design: (2.1) For network initialization, the embodiment of the present application uses a random method to initialize the parameters of the improved deep learning three-dimensional model. (2.2) For the design of the loss function, the embodiment of the present application can define a loss function for training. In the improved deep learning three-dimensional model, the loss function used in the present embodiment can be a cross entropy loss function, which is used to measure the difference between the predicted data and the real data. (2.3) For the setting of the optimizer, the embodiment of the present application can select and configure an optimizer suitable for the application scenario so as to update the parameters of the improved deep learning three-dimensional model during the training process.

[0110] After the initialization design is performed in the embodiment of the present application, iterative training is performed: iterative training is performed on each sample in the training set. For each sample, the point cloud data is input into the improved deep learning 3D model to obtain predicted data. The loss between the predicted data and the real data is calculated, and the parameters of the improved deep learning 3D model are updated according to the loss value. Repeat the above steps until the training of all training samples is completed.

[0111] During the training process, the present embodiment can periodically use the validation set to evaluate the performance of the improved deep learning 3D model. According to the performance indicators on the validation set, such as accuracy, intersection-over-union ratio, etc., the improved deep learning 3D model is adjusted and optimized. After the training is completed, the model parameters obtained by the training are saved to obtain the corresponding human skeleton landmark recognition model for subsequent prediction and application.

[0112] (3) Use the trained improved deep learning 3D model to make predictions and obtain prediction data.

[0113] The 3D model is reconstructed based on the collected original CT data, and the point cloud data of 10,000 points in the 3D model and the label data are used as the input of the human skeleton landmark recognition model. The human skeleton landmark recognition model makes predictions based on the point cloud data of 10,000 points in the 3D model and the label data to obtain the prediction data related to the human skeleton landmarks. The identified human skeleton landmarks enable doctors to make accurate diagnoses and treatment plans.

[0114] The local area formed based on the existing minimum bounding sphere fitting method has a large deviation from the actual area. In order to reduce the deviation between the local area and the actual area, the embodiment of the present application introduces a dynamic radius adjustment strategy in the process of forming the local area. In some possible embodiments, the minimum bounding sphere fitting method is used to group the points around the acquisition point to form a local area, including:

[0115] Step a1: For the collection point, set an initial radius. The initial radius can be customized, for example, the initial radius is 0.1 meters.

[0116] Step a2: Based on the initial radius, the points around the collection point are grouped using the minimum bounding sphere fitting method to obtain a point set. For example, for a certain collection point, with a radius of 0.1 meter, all points that are no more than 0.1 meter away from the collection point are found to form a point set.

[0117] Step a3, introduce a dynamic radius adjustment strategy, so that when the number of collection points in the point set is less than the preset minimum point number threshold, the points around the collection points are grouped using the minimum bounding sphere fitting method based on the increased radius; when the number of collection points in the point set is greater than the preset maximum point number threshold, the points around the collection points are grouped using the minimum bounding sphere fitting method based on the reduced radius; the increased radius is greater than the initial radius, and the reduced radius is less than the initial radius.

[0118] For example, the minimum point number threshold is 10 points, and the maximum point number threshold is 50 points. If the number of points in the point set is less than 10 points, increase the radius to 0.15 meters and re-fit the minimum bounding sphere; if the number of points in the point set is more than 50 points, reduce the radius to 0.08 meters and re-fit the minimum bounding sphere.

[0119] As a possible implementation method, the dynamic radius adjustment strategy includes a dynamic radius adjustment strategy based on point cloud density. The dynamic radius adjustment strategy adopts the following formula:

[0120] ;

[0121] in, is the increased radius, is the current radius. When the dynamic radius is adjusted for the first time, the current radius is the initial radius. is the adjustment factor, are weights of different scales, is a set of points The point cloud density at scale s, , is a set of points The number of collection points at scale s, is a set of points The volume on scale s, is the average density of all points on scale s.

[0122] The dynamic radius adjustment strategy based on point cloud density is a method that dynamically adjusts the radius according to the density of the point cloud when forming a local area. This method ensures that the number of points in the local area is within a reasonable range by considering the distribution density of the point cloud, thereby improving the accuracy and robustness of the local area. By dynamically adjusting the radius, it is also possible to avoid overfitting or underfitting problems caused by too many or too few points, and improve the robustness of the improved deep learning 3D model.

[0123] Step a4: When the number of collected points in the point set is greater than or equal to a preset minimum point number threshold and less than or equal to a maximum point number threshold, a local area is determined based on the point set.

[0124] Exemplarily, the number of points in the point set is 30 points, which meets the requirement of being greater than the minimum point number threshold of 10 and less than the maximum point number threshold of 50, then the set consisting of these 30 points is the final determined local area.

[0125] Through the above-mentioned dynamic radius adjustment strategy, the local area can be formed more accurately, the deviation between the local area and the actual area can be reduced, and the robustness and accuracy of the improved deep learning 3D model can be improved.

[0126] The fitting effect of the existing minimum bounding sphere fitting method needs to be improved. Therefore, in order to improve the fitting effect of the minimum bounding sphere fitting method, the embodiment of the present application constructs an objective function that is different from the minimum bounding sphere fitting method used in the prior art. In some possible embodiments, step a2, based on the initial radius, uses the minimum bounding sphere fitting method to group the points around the acquisition point to obtain a point set, including:

[0127] Step a21, taking the collection point as the initial sphere center, and based on the initial sphere center and the initial radius, selecting all points whose distance from the initial sphere center is less than or equal to the initial radius as points around the collection point, and storing the points around the collection point and the collection point together in an empty set to obtain a set to be updated.

[0128] Step a22: construct the objective function adopted by the minimum bounding sphere fitting method, and minimize the objective function based on the set to be updated to find the optimized sphere center and radius.

[0129] The minimum bounding sphere fitting method provided in the embodiment of the present application may be a weighted minimum bounding sphere fitting method, a multi-objective optimization method, etc. Accordingly, as a possible implementation, the minimum bounding sphere fitting method is a weighted minimum bounding sphere fitting method; the objective function adopted by the weighted minimum bounding sphere fitting method is:

[0130] ;

[0131] Among them, J1 is the objective function used by the minimum bounding sphere fitting method, It isi The coordinates of the points, is the center of the smallest enclosing sphere, is the radius of the minimum enclosing sphere, It is i The weight of the point, n is the number of collection points in the set to be updated.

[0132] In the embodiment of the present application, the weighted minimum bounding sphere fitting method introduces weights into the objective function so that certain points have a greater impact on the optimization result. For example, different weights can be assigned based on the importance or reliability of the points.

[0133] As another possible implementation method, the minimum bounding sphere fitting method is a multi-objective optimization method; the objective function used in the multi-objective optimization method is:

[0134] ;

[0135] Among them, J2 is the objective function used in the multi-objective optimization method, is the weight that minimizes the radius, is the weight of the uniformity of the distribution of the collection points, k is the number of iterations, It is i The coordinates of the points, is the center of the smallest enclosing sphere, is the radius of the minimum enclosing sphere, n is the number of collection points in the set to be updated.

[0136] The above multi-objective optimization method can define multiple objective functions, each objective function measuring a different optimization goal. For example, one objective function can measure the radius of the bounding sphere, and another objective function can measure the number of points within the bounding sphere.

[0137] Step a23: Based on the optimized sphere center and radius, select all points whose distance from the optimized sphere center is less than or equal to the optimized radius, and add all points whose distance from the optimized sphere center is less than or equal to the optimized radius to the set to be updated to obtain a point set.

[0138] The embodiment of the present application finds the optimized center and radius by minimizing the objective function, ensures that the number of points in the local area is within a reasonable range, and reduces the deviation between the local area and the actual area. The weighted minimum enclosing sphere fitting method and the multi-objective optimization method can adapt to point cloud data of different densities and improve the accuracy of the local area. By optimizing the center and radius, the overfitting or underfitting problems caused by too many or too few points can be avoided, and the robustness of the model can be improved. By optimizing the center and radius, features can be extracted at different scales, the expressive power of the model can be enhanced, and the geometric and topological information of the local area can be better captured. By optimizing the center and radius, unnecessary computational overhead can be reduced and the computational efficiency of the model can be improved.

[0139] Figure 3 A schematic diagram of the structure of a model training device for extracting human skeleton features provided in an embodiment of the present application, such as Figure 3 As shown, the device comprises:

[0140] The acquisition module 31 is used to acquire original CT data of human bones at target parts of different target objects belonging to the same group.

[0141] The reconstruction module 32 is used to reconstruct the three-dimensional model of each target object for the target part based on the original CT data, and obtain the model file corresponding to each three-dimensional model.

[0142] The extraction and generation module 33 is used to extract the point cloud data of the corresponding three-dimensional model from each model file, and batch generate label data of each voxel in the corresponding three-dimensional model; wherein the label data includes identification data indicating whether the voxel is a human skeleton landmark point. If the identification data indicates that the voxel is a human skeleton landmark point, the label data also includes: the position data of the marked human skeleton landmark point, the marked depth data, the marked area size and the marked morphological information.

[0143] The training module 34 is used to use the point cloud data of all three-dimensional models and the label data of each voxel in all three-dimensional models as a training set, train the improved deep learning three-dimensional model until the preset stopping conditions are met, and obtain the human skeleton landmark point recognition model, wherein the improved deep learning three-dimensional model is a deep learning three-dimensional model that introduces dynamic graph convolution, and the human skeleton landmark point recognition model is used to extract human skeleton features and identify human skeleton landmark points.

[0144] Figure 3 The model training device for extracting human skeleton features can be performed Figure 1The implementation principle and technical effect of the model training method for extracting human skeleton features described in the illustrated embodiment will not be described in detail. The specific manner in which each module and unit performs operations in the model training device for extracting human skeleton features in the above embodiment has been described in detail in the embodiment of the method, and will not be described in detail here.

[0145] In one possible design, Figure 3 The model training device for extracting human skeleton features in the illustrated embodiment can be implemented as a computing device, such as Figure 4 As shown, the computing device may include a storage component 41 and a processing component 42 .

[0146] The storage component 41 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 42 .

[0147] The processing component 42 is used to: obtain the original computer tomography CT data of the human skeleton at the target part of different target objects belonging to the same group; reconstruct the three-dimensional model of each target object at the target part based on the original CT data, and obtain the model file corresponding to each three-dimensional model; extract the point cloud data of the corresponding three-dimensional model from each model file, and batch generate the label data of each voxel in the corresponding three-dimensional model; wherein the label data includes identification data indicating whether the voxel is a human skeleton landmark point, if the identification data indicates that the voxel is a human skeleton landmark point, then the label data also includes: the position data of the labeled human skeleton landmark point, the labeled depth data, the labeled area size and the labeled morphological information; use the point cloud data of all three-dimensional models and the label data of each voxel in all three-dimensional models as training sets, train the improved deep learning three-dimensional model until the preset stop condition is met, and obtain the human skeleton landmark point recognition model, wherein the improved deep learning three-dimensional model is a deep learning three-dimensional model that introduces dynamic graph convolution, and the human skeleton landmark point recognition model is used to extract human skeleton features and recognize human skeleton landmark points.

[0148] The processing component 42 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented by one or more application-specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0149] The storage component 41 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as random access memory (RAM), static random access memory (SRAM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0150] Of course, the computing device may also include other components, such as input / output interfaces, display components, communication components, etc.

[0151] The input / output interface provides an interface between the processing component and the peripheral interface module, which may be an output device, an input device, etc.

[0152] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.

[0153] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0154] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 The model training method for extracting human skeleton features in the illustrated embodiment.

[0155] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0156] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0157] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A model training method for extracting human skeleton features, characterized in that: include: Acquire original computed tomography (CT) data of human bones of different target subjects belonging to the same group at target locations; Reconstructing a three-dimensional model of a target part of each target object based on the original CT data to obtain a model file corresponding to each three-dimensional model; Extracting point cloud data corresponding to the three-dimensional model from each of the model files, and generating label data of each voxel in the corresponding three-dimensional model in batches; wherein the label data includes identification data indicating whether the voxel is a human skeleton landmark point, and if the identification data indicates that the voxel is a human skeleton landmark point, the label data also includes: position data of the labeled human skeleton landmark point, labeled depth data, labeled area size and labeled morphological information; The point cloud data of all three-dimensional models and the label data of each voxel in all three-dimensional models are used as a training set to train the improved deep learning three-dimensional model until a preset stop condition is met, thereby obtaining a human skeleton landmark point recognition model, wherein the human skeleton landmark point recognition model is used to extract human skeleton features and identify human skeleton landmark points; The improved deep learning three-dimensional model includes a sampling layer, an improved local area division layer, a local feature extraction layer, a global feature extraction layer and an output layer connected in sequence; The sampling layer selects a plurality of acquisition points from the point cloud data of the three-dimensional model, wherein the plurality of acquisition points are evenly distributed in the three-dimensional space; The improved local area division layer uses a minimum bounding sphere fitting method to group points around each acquisition point to form a local area; obtains a three-dimensional human skeleton template corresponding to the group to which the target object belongs, and if the similarity between the multiple acquisition points and multiple human skeleton landmarks in the three-dimensional human skeleton template is less than a preset threshold, a dynamic graph convolution operation is performed on the local area to obtain updated node features of the local area; The local feature extraction layer extracts low-level features in each local area based on the updated node features of each local area through a multi-layer perceptron, and aggregates the low-level features in each local area into a global feature vector through a pooling operation; The global feature extraction layer extracts global features from the global feature vector; The output layer maps the global features to predicted data of human skeleton landmarks, wherein the predicted data includes: predicted position data of human skeleton landmarks, predicted depth data, predicted area size and predicted morphological information.

2. The method according to claim 1, characterized in that The minimum bounding sphere fitting method is used to group the points around the acquisition point to form a local area, including: For the collection point, an initial radius is set; Based on the initial radius, the minimum bounding sphere fitting method is used to group the points around the collection point to obtain a point set; A dynamic radius adjustment strategy is introduced, so that when the number of acquisition points in the point set is less than a preset minimum point number threshold, the points around the acquisition points are grouped by using a minimum bounding sphere fitting method based on an increased radius; when the number of acquisition points in the point set is greater than a preset maximum point number threshold, the points around the acquisition points are grouped by using a minimum bounding sphere fitting method based on a reduced radius; the increased radius is greater than the initial radius, and the reduced radius is less than the initial radius; When the number of collected points in the point set is greater than or equal to a preset minimum point number threshold and less than or equal to a maximum point number threshold, a local area is determined based on the point set.

3. The method according to claim 2, characterized in that The dynamic radius adjustment strategy includes a dynamic radius adjustment strategy based on point cloud density. The dynamic radius adjustment strategy adopts the following formula: ； in, is the increased radius, is the current radius. When the dynamic radius is adjusted for the first time, the current radius is the initial radius. is the adjustment factor, are weights of different scales, is a set of points The point cloud density at scale s, , is a set of points The number of collection points at scale s, is a set of points The volume on scale s, is the average density of all points on scale s.

4. The method according to claim 2, characterized in that: Based on the initial radius, the points around the acquisition point are grouped using the minimum bounding sphere fitting method to obtain a point set, including: Taking the collection point as the initial sphere center, and based on the initial sphere center and the initial radius, select all points whose distance from the initial sphere center is less than or equal to the initial radius as points around the collection point, and store the points around the collection point and the collection point together in an empty set to obtain a set to be updated; Construct the objective function used by the minimum bounding sphere fitting method, and minimize the objective function based on the set to be updated to find the optimized sphere center and radius; Based on the optimized sphere center and radius, all points whose distances to the optimized sphere center are less than or equal to the optimized radius are selected, and all points whose distances to the optimized sphere center are less than or equal to the optimized radius are added to the set to be updated to obtain a point set.

5. The method according to claim 4, characterized in that The minimum bounding sphere fitting method includes a weighted minimum bounding sphere fitting method; the objective function adopted by the weighted minimum bounding sphere fitting method is: ； Among them, J1 is the objective function used by the minimum bounding sphere fitting method, It is The coordinates of the points, is the center of the smallest enclosing sphere, is the radius of the minimum enclosing sphere, It is The weight of the point, is the number of collection points in the set to be updated.

6. The method according to claim 4, characterized in that The minimum bounding sphere fitting method includes a multi-objective optimization method; the objective function adopted by the multi-objective optimization method is: ； Among them, J2 is the objective function used in the multi-objective optimization method, is the weight that minimizes the radius, is the weight of the uniformity of the collection point distribution, is the number of iterations, It is The coordinates of the points, is the center of the smallest enclosing sphere, is the radius of the minimum enclosing sphere, is the number of collection points in the set to be updated.

7. The method according to claim 1, characterized in that After the improved local area division layer obtains the human skeleton three-dimensional template corresponding to the group to which the target object belongs, the method further includes: If the similarity between the multiple acquisition points and the multiple human skeleton landmark points in the human skeleton three-dimensional template is greater than or equal to a preset threshold, the feature information of the multiple human skeleton landmark points in the human skeleton three-dimensional template is determined as the updated node features of the local area.

8. The method according to claim 1, characterized in that The original CT data includes original image data and metadata, wherein the metadata includes target object information and scanning parameters; the target object information includes: object number and human skeleton history data; The reconstructing of the three-dimensional model of the target part of each target object based on the original CT data to obtain the model file corresponding to each three-dimensional model includes: For the original CT data of the human skeleton at the target part of each target object, the following reconstruction process is performed: Preprocessing the original image data to obtain preprocessed image data; Based on the preprocessed image data, the target part is separated by using a threshold segmentation or a region growing algorithm to segment the image; Converting data related to the target part in the preprocessed image data into voxel data; Generate a surface mesh of a three-dimensional model of a target part of the target object using a surface reconstruction method based on the voxel data; The three-dimensional model is optimized based on the metadata to obtain a model file corresponding to the three-dimensional model.

9. A model training device for extracting human skeleton features, characterized in that: include: An acquisition module, used for acquiring original CT data of human bones of target parts of different target objects belonging to the same group; A reconstruction module, used to reconstruct a three-dimensional model of a target part of each target object based on the original CT data, and obtain a model file corresponding to each three-dimensional model; An extraction and generation module is used to extract point cloud data corresponding to the three-dimensional model from each of the model files, and batch generate label data of each voxel in the corresponding three-dimensional model; wherein the label data includes identification data indicating whether the voxel is a human skeleton landmark point, and if the identification data indicates that the voxel is a human skeleton landmark point, the label data also includes: position data of the labeled human skeleton landmark point, labeled depth data, labeled area size and labeled morphological information; A training module is used to use the point cloud data of all three-dimensional models and the label data of each voxel in all three-dimensional models as a training set, train the improved deep learning three-dimensional model until a preset stop condition is met, and obtain a human skeleton landmark point recognition model, wherein the human skeleton landmark point recognition model is used to extract human skeleton features and identify human skeleton landmark points; The improved deep learning three-dimensional model includes a sampling layer, an improved local area division layer, a local feature extraction layer, a global feature extraction layer and an output layer connected in sequence; The sampling layer selects a plurality of acquisition points from the point cloud data of the three-dimensional model, wherein the plurality of acquisition points are evenly distributed in the three-dimensional space; The improved local area division layer uses a minimum bounding sphere fitting method to group points around each acquisition point to form a local area; obtains a three-dimensional human skeleton template corresponding to the group to which the target object belongs, and if the similarity between the multiple acquisition points and multiple human skeleton landmarks in the three-dimensional human skeleton template is less than a preset threshold, a dynamic graph convolution operation is performed on the local area to obtain updated node features of the local area; The local feature extraction layer extracts low-level features in each local area based on the updated node features of each local area through a multi-layer perceptron, and aggregates the low-level features in each local area into a global feature vector through a pooling operation; The global feature extraction layer extracts global features from the global feature vector; The output layer maps the global features to predicted data of human skeleton landmarks, wherein the predicted data includes: predicted position data of human skeleton landmarks, predicted depth data, predicted area size and predicted morphological information.

Images