Child hip joint development prediction method and device

Abstract

The invention relates to the technical field of child hip joint development prediction, in particular to a child hip joint development prediction method and device.In the embodiment of the invention, firstly, a hip joint two-dimensional ultrasonic image is input into a first prediction model, and the normal distribution probability and the prediction value of the child hip joint development week age number are predicted; a hip three-dimensional model is constructed through the hip joint two-dimensional ultrasonic image, and three groups of acetabulum dysplasia evaluation factors in three dimensions are obtained based on the hip three-dimensional model; and finally, integrating the children clinical information, the children hip joint development week age prediction result and the acetabular dysplasia evaluation factor to obtain a prediction result of the children hip joint development condition.

Description

Technical Field

[0001] The present invention relates to the technical field of prediction of children's hip joint development, and in particular to a method and device for predicting children's hip joint development. Background Art

[0002] Developmental dysplasia of the hip (DDH) is a condition characterized by an abnormality in the anatomical structure of the hip joint, either congenital or acquired during development. Its primary manifestation is an abnormal fit between the acetabulum and the femoral head. The overarching principle for diagnosis and treatment of this condition is "early detection, early treatment." This is especially true for infants aged 0-6 months, whose bones are still developing. Early diagnosis and intervention within 6 months can significantly improve treatment outcomes and reduce the risk of subsequent complications. Therefore, early screening and targeted intervention for DDH are crucial for improving patient outcomes.

[0003] Ultrasound examination is the primary diagnostic method for DDH screening in infants under 6 months of age. Its advantages include the ability to clearly display the cartilage and bony structures around the hip joint, the lack of radiation damage, and the ability to dynamically observe continuous images. However, ultrasound examination is highly dependent on standardized images, and the process of obtaining standard images is highly dependent on the operator's experience. There are also large differences between different standard images. In particular, scanning angle errors may lead to inaccurate measurements of key indicators such as anatomical angles and acetabular coverage, thereby affecting the accuracy of the final diagnosis. Relying solely on manual interpretation of two-dimensional ultrasound images, it is almost impossible to obtain standardized and highly consistent diagnostic conclusions.

[0004] In order to improve the accuracy of DDH diagnosis, artificial intelligence methods have been applied in the diagnosis of DDH, and the standardization of diagnostic methods is achieved by automatically judging standard ultrasound images, such as CN117297665A, CN118411373A, CN112907507A, CN110604594B, CN106667525A, etc. However, due to the large errors between different standard images, the accuracy of key anatomical angle measurements is still limited, which may lead to misjudgment of acetabular development and affect diagnosis and treatment decisions. In addition, this problem also means that the ultrasound diagnostic results of the same child at different times cannot be used as a reference for each other, and it is impossible to construct a method to predict the future development of the disease. Therefore, how to evaluate the growth and development of DDH patients by establishing a reliable prediction model based on ultrasound images has become an urgent problem to be solved. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a method and device for predicting children's hip joint development to solve the technical problems in the prior art.

[0006] Based on the above objectives, as a first aspect of the present invention, the present invention provides a method for predicting hip joint development in children, comprising:

[0007] Obtain clinical information of children and two-dimensional ultrasound images of the hip joint with spatial coordinates;

[0008] Inputting the two-dimensional ultrasound image of the hip joint into a pre-trained first prediction model to predict the normal distribution probability and predicted value of the child's hip joint development age in weeks;

[0009] constructing a three-dimensional model of the hip based on the two-dimensional ultrasound image of the hip joint, measuring hip anatomical parameters in the three-dimensional model of the hip, and obtaining three sets of three-dimensional acetabular dysplasia evaluation factors;

[0010] The child's clinical information, the predicted result of the child's hip joint development age in weeks, and the acetabular dysplasia evaluation factor are input into a pre-trained second prediction model to predict the child's hip joint development.

[0011] As an optional implementation, the training method of the first prediction model is:

[0012] Select multiple sets of hip joint ultrasound images passing near the centroid of the femoral head, and record the age of the child in weeks when the images were collected, to form a hip joint ultrasound image dataset with age labels;

[0013] Dividing the hip joint ultrasound image dataset into a training set and a test set;

[0014] An initial first prediction model is constructed, which takes a single-channel hip joint ultrasound image as input and outputs the probability distribution and predicted age value of the hip joint.

[0015] The training set is input into the initial first prediction model to train the model, and the model is tested with the test set until the model converges.

[0016] As an optional embodiment, the acetabular dysplasia evaluation factors include a hip joint characteristic angle change curve, a three-dimensional acetabular coverage rate, and a characteristic distance.

[0017] As an optional implementation, the calculation method of the hip joint characteristic angle change curve is:

[0018] Based on anatomical principles, the iliac ramus anatomical region and the acetabulum anatomical region are obtained from the three-dimensional hip model by dissection, and the iliac ramus anatomical region and the acetabulum anatomical region are respectively plane-fitted using the least squares method to obtain the closest planes n1 and n2;

[0019] Solving for an anatomical axis n, which is perpendicular to the planes n1 and n2;

[0020] Multiple cross-sections of the iliac ramus and acetabulum anatomical regions were taken in the direction perpendicular to the anatomical axis n. Linear fitting was performed on each cross-section using the least squares method to obtain the fitting lines L1 and L2 that were closest to the cross-section. The angle between the two fitting lines L1 and L2 was the characteristic angle α of the iliac ramus and acetabulum anatomical regions on the cross-section.

[0021] By calculating the characteristic angle on each cross section, the variation curve of the characteristic angle α along the anatomical axis n, as well as the maximum and minimum values ​​of the characteristic angle α can be obtained.

[0022] As an optional embodiment, the three-dimensional acetabulum coverage is calculated as follows:

[0023] Based on anatomical principles, the anatomical region of the iliac ramus and the anatomical region of the femoral head are obtained from the three-dimensional hip model;

[0024] The least squares method was used to fit the plane of the iliac ramus anatomical region to obtain the closest plane n1;

[0025] The volume ratio of the femoral head anatomical area located inside plane n1 was calculated as the acetabulum coverage rate.

[0026] As an optional implementation, the characteristic distance is calculated as follows:

[0027] Based on anatomical principles, the femoral head anatomical region is obtained from the three-dimensional hip model, and its center of mass C is acquired;

[0028] Using a straight line passing through the center of mass C, the straight line intersects the surface of the anatomical region of the femoral head and the surface of the anatomical region of the pubic bone or ischium;

[0029] Among all possible intersecting straight lines, the straight line segment with the shortest distance between the surface of the femoral head anatomical region and the surface of the pubic bone or ischium anatomical region is obtained, which is the characteristic distance.

[0030] As an optional implementation, the method for solving the straight line segment with the shortest distance is a gradient descent method:

[0031] Let the line passing through the centroid C be θ, and the distance function be D(θ), which represents the distance between the surface of the anatomical region of the femoral head and the surface of the anatomical region of the pubic bone or ischium. The line with the minimum distance is found by iterating the following formula:

[0032]

[0033] As an optional embodiment, the step of inputting the child's clinical information, the predicted result of the child's hip joint development age in weeks, and the acetabular dysplasia evaluation factor into a pre-trained second prediction model to predict the child's hip joint development includes:

[0034] Acquiring an ultrasonic medical image passing through the centroid of the femoral head, inputting the ultrasonic medical image into a convolutional neural network to obtain a one-dimensional image feature vector with multi-scale features;

[0035] Encode clinical information to form a one-dimensional clinical feature vector;

[0036] Encode the patient follow-up time to form a one-dimensional time feature vector;

[0037] fusing the image feature vector, the clinical feature vector, and the time feature vector into a unified fused feature vector;

[0038] The fused feature vector is used as input into a second prediction model to predict the development of the child's hip joint at a preset time node in the future.

[0039] As an optional implementation, the training method of the second prediction model is:

[0040] Obtain a labeled fused feature vector dataset;

[0041] Dividing the fused feature vector dataset into a training set and a test set;

[0042] Construct an initial second prediction model, taking the single-channel fusion feature vector as input and the child's hip joint development as output;

[0043] The training set is input into the initial second prediction model to train the model, and the model is tested with the test set until the model converges.

[0044] As a second aspect of the present invention, corresponding to the method for predicting the development of the child's hip joint, a device for predicting the development of the child's hip joint is also provided, comprising:

[0045] an acquisition module, used for acquiring clinical information of the child and a two-dimensional ultrasound image of the child's hip joint with spatial coordinates;

[0046] A first prediction module is configured to input the two-dimensional ultrasound image of the hip joint into a pre-trained first prediction model to predict the normal distribution probability and predicted value of the child's hip joint development age in weeks;

[0047] a measurement module, configured to construct a three-dimensional model of the hip based on the two-dimensional ultrasound image of the hip joint, measure hip anatomical parameters in the three-dimensional model of the hip, and obtain three sets of three-dimensional acetabular dysplasia evaluation factors;

[0048] The second prediction module is used to input the child's clinical information, the prediction result of the child's hip joint development age in weeks, and the acetabular dysplasia evaluation factor into the pre-trained second prediction model to predict the child's hip joint development.

[0049] Beneficial effects of the present invention: In an embodiment of the present invention, first, a two-dimensional ultrasound image of the hip joint is input into a first prediction model to predict the normal distribution probability and predicted value of the child's hip joint development age in weeks; a three-dimensional hip model is constructed using the two-dimensional ultrasound image of the hip joint, and three groups of three-dimensional acetabular dysplasia evaluation factors are obtained based on the three-dimensional hip model; finally, the child's clinical information, the predicted results of the child's hip joint development age in weeks, and the acetabular dysplasia evaluation factors are integrated to obtain a prediction result of the child's hip joint development. Compared with traditional prediction methods, the embodiment of the present invention, on the one hand, uses the femoral head of two-dimensional ultrasound as the ROI, extracts features through the characteristic texture and morphology of the femoral head, judges the child's actual bone age, and aligns the development time; on the other hand, a three-dimensional model is constructed using the marked two-dimensional ultrasound image, and three groups of three-dimensional acetabular dysplasia evaluation factors are obtained in the three-dimensional image. Finally, the two factors are combined to predict the child's development and obtain better prediction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 Schematic diagram of a prediction method according to an embodiment of the present invention;

[0052] Figure 2 Schematic diagram of the flow of the first prediction model according to an embodiment of the present invention;

[0053] Figure 3 A schematic diagram of three-dimensional acetabulum coverage according to an embodiment of the present invention;

[0054] Figure 4 Schematic diagram of a prediction device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to specific embodiments.

[0056] It should be noted that, unless otherwise defined, technical terms or scientific terms used in the present application should be understood as their common meanings to those skilled in the art to which the present application pertains. The terms "first", "second" and similar terms used in the present application do not indicate any order, number or importance, but are only used to distinguish different components. The terms "comprise", "include" and similar terms mean that the elements or objects before the terms encompass the elements or objects listed after the terms and their equivalents, without excluding other elements or objects. The terms "connect" or "connected" and similar terms do not mean physical or mechanical connection, but can include electrical connection, whether direct or indirect. The terms "upper", "lower", "left", "right" and the like are only used to indicate relative positional relationships, and when the absolute positions of the described objects change, the relative positional relationships can also change accordingly.

[0057] To solve the above technical problems, as shown in the technical scheme of the present application, Figure 1 The present application provides a child hip joint development prediction method, comprising:

[0058] S100, acquiring clinical information of a child and a two-dimensional ultrasound image of a hip joint of the child with spatial coordinates;

[0059] S200, inputting the two-dimensional ultrasound image of the hip joint into a first prediction model pre-trained, to predict a normal distribution probability and a prediction value of a child hip joint development age;

[0060] S300, based on the two-dimensional ultrasound image of the hip joint, constructing a hip three-dimensional model, measuring hip anatomical parameters in the hip three-dimensional model, and obtaining three groups of hip dysplasia evaluation factors under three dimensions;

[0061] S400, inputting the clinical information of the child, the child hip joint development age prediction result and the hip dysplasia evaluation factors into a second prediction model pre-trained, to predict the child hip joint development situation.

[0062] In an embodiment of the present invention, a two-dimensional ultrasound image of the hip joint is first input into a first prediction model to predict the normal distribution probability and predicted value of the child's hip joint development age in weeks; a three-dimensional hip model is constructed using the two-dimensional ultrasound image of the hip joint, and three sets of three-dimensional acetabular dysplasia evaluation factors are obtained based on the three-dimensional hip model. Finally, the child's clinical information, the predicted results of the child's hip joint development age in weeks, and the acetabular dysplasia evaluation factors are combined to obtain a prediction result of the child's hip joint development. Compared with traditional prediction methods, the embodiment of the present invention, on the one hand, uses the femoral head of the two-dimensional ultrasound as the region of interest (ROI), extracts features based on the characteristic texture and morphology of the femoral head, judges the child's actual bone age, and aligns the development time; on the other hand, a three-dimensional model is constructed using the marked two-dimensional ultrasound image, and three sets of three-dimensional acetabular dysplasia evaluation factors are obtained in the three-dimensional image. Finally, the two factors are combined to predict the child's development and obtain better prediction results.

[0063] Optionally, the method for acquiring the two-dimensional ultrasound image of the child's hip joint with spatial coordinates in step S100 is:

[0064] First, a set of two-dimensional ultrasound images with spatial coordinates is acquired at the hip joint. These spatial coordinates can be obtained using sensors located in each degree of freedom on a mechanical body frame connected to the ultrasound probe, or using an optical navigation system, a magnetic navigation target, or a robotic arm fixed to the ultrasound probe. Alternatively, the two-dimensional ultrasound images with spatial coordinates can be directly acquired using a three-dimensional ultrasound probe.

[0065] Correct and align various image parameters based on the image source.

[0066] As an optional embodiment of the present invention, the training method of the first prediction model is:

[0067] Select multiple sets of hip joint ultrasound images passing near the centroid of the femoral head, and record the age of the child in weeks when the images were collected, to form a hip joint ultrasound image dataset with age labels;

[0068] Dividing the hip joint ultrasound image dataset into a training set and a test set;

[0069] An initial first prediction model is constructed, which takes a single-channel hip joint ultrasound image as input and outputs the probability distribution and predicted age value of the hip joint.

[0070] The training set is input into the initial first prediction model to train the model, and the model is tested with the test set until the model converges.

[0071] As an example, Figure 2As shown in the figure, the architecture of the convolutional neural network used in the first prediction model mainly consists of four parts: input layer, feature extraction module, multi-scale feature fusion module, and dual-task output layer.

[0072] The input layer is a single-channel grayscale image of 224×224 pixels, which is a two-dimensional ultrasound image of the femoral head anatomical area (i.e., the region of interest) extracted in advance by a U-Net segmentation network.

[0073] The feature extraction module adopts an improved design based on the EfficientNet architecture and is divided into multiple stages to extract image features according to the following modules:

[0074] Backbone network: 32-channel Conv module, using 3×3 convolution and stride 2, with an output feature map size of 112×112×32;

[0075] Module 1: 16-channel MBConv1 module, using 3×3 depthwise separable convolution and stride 1, with an output feature map size of 112×112×16;

[0076] Module 2: Multiple 24-channel MBConv6 modules using 3×3 depthwise separable convolutions with an output feature map size of 56×56×24;

[0077] Module 3: Multiple 40-channel MBConv6 modules using 5×5 depthwise separable convolutions and stride 2, with an output feature map size of 28×28×40;

[0078] Module 4: Multiple 80-channel MBConv6 modules using 3×3 depthwise separable convolutions and stride 1, with an output feature map size of 28×28×80;

[0079] Module 5: Multiple 112-channel MBConv6 modules using 5×5 depthwise separable convolutions and stride 2, with an output feature map size of 14×14×112;

[0080] Module 6: Multiple 192-channel MBConv6 modules using 5×5 depthwise separable convolutions and stride 2, with an output feature map size of 7×7×192;

[0081] Module 7: 320-channel MBConv6 module, using 5×5 depthwise separable convolution and stride 1, with an output feature map size of 7×7×320;

[0082] Final convolutional layer module: A 1280-channel Conv module that uses 1×1 convolution and stride 1, performs global average pooling and a fully connected layer, and outputs a 1280-dimensional feature vector.

[0083] The multi-scale feature fusion module reduces the dimensionality of features at different scales in the feature extraction module and concatenates them with the final convolutional layer module to generate a one-dimensional feature vector containing multi-scale features. Specifically, it includes:

[0084] Low-level features: The 56×56×24 feature map output by module 2 is used to output a 24-dimensional vector after global average pooling or global maximum pooling to capture the detailed features of the ultrasound image, such as the texture of the cartilage.

[0085] Middle-level features: The 28×28×80 feature map output by module 4 is used to output an 80-dimensional vector after global average pooling or global maximum pooling to capture local features of the ultrasound image, such as the morphological structure of bones.

[0086] High-level features: 1280-dimensional feature vectors output by the final convolutional layer module, used to capture semantic-level abstract features of ultrasound images.

[0087] The dual-task output layer connects the multi-scale fused feature vectors to two branches. The first branch is used to predict the probability distribution of age, specifically including:

[0088] Fully connected layer 1: Receives a 1384-dimensional feature vector from the multi-scale feature fusion module and outputs a 512-dimensional feature vector. It uses the ReLU activation function and optionally sets a dropout mechanism to enhance the generalization ability of the model.

[0089] Fully connected layer 2: Input 512-dimensional feature vector, output 28-dimensional feature vector, representing the age of the baby from 7 to 34 weeks. The output is normalized using the Softmax activation function, and the final discrete probability distribution P(y) with an age range of 7 to 34 weeks is obtained, which satisfies the probability constraint condition, that is,

[0090] P(y)∈[0,1],

[0091] The second branch is used to predict the expected value of the age in weeks, including:

[0092] Fully connected layer 1: receives the 1384-dimensional feature vector from the multi-scale feature fusion module, outputs a 64-dimensional feature vector, and uses the ReLU activation function;

[0093] Fully connected layer 2: Inputs a 64-dimensional feature vector, outputs a scalar node, uses a linear activation function, and outputs a continuous value, which is the expected value of the age in weeks.

[0094] The loss function of the convolutional neural network is the weighted sum of the KL divergence loss of the probability distribution and the deviation loss of the expected value of the age in weeks. The advantage of this loss function definition is that it can predict both the probability distribution and the expected value at the same time, giving both a predicted age and a prediction range, providing more reference for doctors. The loss function can be expressed as:

[0095] L total =α·L KL +(1-α)·L MAE

[0096] Among them: KL divergence loss is the predicted probability distribution and actual age Standard normal distribution near The difference between. ; ; The expected age deviation loss is the absolute error between the predicted age and the actual age:

[0097] in Is the weight of the KL divergence loss, which is a hyperparameter.

[0098] Since the ultrasound images of the femoral head of infants show different morphological characteristics at different stages of development, the images go through the stages of no development, cord-like development, formation of bright ossification centers, and complete ossification in sequence. The morphological changes of the femoral head ossification center are highly correlated with the developmental process of the child's hip joint. This application can predict this feature by establishing an artificial intelligence model, which serves as the basis for evaluating the development rate and bone age through hip ultrasound images. It can achieve accurate prediction of the child's actual developmental age, thereby providing a scientific basis for early screening, diagnosis and intervention.

[0099] As an optional embodiment, the acetabular dysplasia evaluation factors include a hip joint characteristic angle change curve, a three-dimensional acetabular coverage rate, and a characteristic distance.

[0100] The calculation method of the hip joint characteristic angle change curve is:

[0101] Based on anatomical principles, the iliac ramus anatomical region and the acetabulum anatomical region are obtained from the three-dimensional model of the hip. The least squares method is used to perform plane fitting on the iliac ramus anatomical region and the acetabulum anatomical region to obtain the closest planes n1 and n2; that is, solve an equation of n p The plane with r + d = 0 minimizes the sum of the squares of the distances from all points in the anatomical region to the plane:

[0102]

[0103] Among them, n p is the normal vector of the plane, d is the intercept of the plane, so that all points r in the space passing through the plane satisfy n p · r+d=0.

[0104] Solve for an anatomical axis n, which is perpendicular to the planes n1 and n2. Specifically, if the normal vectors of the two planes are n1 and n2, the normal vector n of the anatomical axis is obtained by the following cross product:

[0105] n=n1×n2

[0106] Multiple cross-sections of the iliac ramus and acetabulum anatomical regions were taken in the direction perpendicular to the anatomical axis n. Linear fitting was performed on each cross-section using the least squares method to obtain the fitting lines L1 and L2 that were closest to the cross-section. The angle between the two fitting lines L1 and L2 was the characteristic angle α of the iliac ramus and acetabulum anatomical regions on the cross-section.

[0107] By calculating the characteristic angle on each cross section, the variation curve of the characteristic angle α along the anatomical axis n, as well as the maximum and minimum values ​​of the characteristic angle α can be obtained.

[0108] The characteristic angle α is the angle between the anatomical regions of the iliac ramus and the acetabulum. The characteristic angle's curve along the anatomical axis captured by 3D imaging can avoid errors in measuring the characteristic angle caused by deviations such as displacement and rotation between the ultrasound probe's cross-section and the standard cross-section when using only 2D images for diagnosis. This angle curve allows doctors to comprehensively assess the development of a child's acetabulum.

[0109] The calculation method of the three-dimensional acetabulum coverage is:

[0110] Based on anatomical principles, the anatomical region of the iliac ramus and the anatomical region of the femoral head are obtained from the three-dimensional hip model;

[0111] The least squares method was used to fit the plane of the iliac ramus anatomical region to obtain the closest plane n1;

[0112] The volume ratio of the femoral head anatomical area located inside plane n1 was calculated as the acetabulum coverage rate.

[0113] This coverage index helps doctors assess whether the child's femoral head is covered by the acetabulum and determine whether the femoral head is dislocated.

[0114] The characteristic distance is calculated as follows:

[0115] Based on anatomical principles, the femoral head anatomical region is obtained from the three-dimensional hip model, and its center of mass C is acquired;

[0116] Using a straight line passing through the center of mass C, the straight line intersects the surface of the anatomical region of the femoral head and the surface of the anatomical region of the pubic bone or ischium;

[0117] Among all possible intersecting straight lines, the straight line segment with the shortest distance between the surface of the femoral head anatomical region and the surface of the pubic bone or ischium anatomical region is obtained, which is the characteristic distance.

[0118] The method for finding the shortest straight line segment is the gradient descent method:

[0119] Let the line passing through the centroid C be θ, and the distance function be D(θ), which represents the distance between the surface of the anatomical region of the femoral head and the surface of the anatomical region of the pubic bone or ischium. The line with the minimum distance is found by iterating the following formula:

[0120]

[0121] Among them, θ k+1 is the parameter value at the kth iteration, α is the learning rate, is the gradient of the distance function D(θ).

[0122] Measuring the distance from the edge of the femoral head to the pubic bone or ischium can help doctors evaluate whether a child's acetabulum is dislocated.

[0123] As an optional implementation, Figure 3 As shown, the child's clinical information, the predicted result of the child's hip joint development age in weeks, and the acetabular dysplasia evaluation factor are input into the pre-trained second prediction model to predict the child's hip joint development, including:

[0124] Acquiring an ultrasonic medical image passing through the centroid of the femoral head, inputting the ultrasonic medical image into a convolutional neural network to obtain a one-dimensional image feature vector with multi-scale features;

[0125] Encode clinical information to form a one-dimensional clinical feature vector;

[0126] Encode the patient follow-up time to form a one-dimensional time feature vector;

[0127] fusing the image feature vector, the clinical feature vector, and the time feature vector into a unified fused feature vector;

[0128] The fused feature vector is used as input into a second prediction model to predict the development of the child's hip joint at a preset time node in the future.

[0129] Preferably, the convolutional neural network can be the neural network used in the first prediction model, including an input layer, a feature extraction module, and a multi-scale feature fusion module, and can also adopt a deep convolutional neural network structure such as ResNet, DenseNet or Inception Net.

[0130] Preferably, the clinical information includes but is not limited to basic information such as the child's age, gender, race, disease classification, and the three-dimensional hip dysplasia evaluation factor, three-dimensional acetabular coverage, three-dimensional image feature distance and other diagnostic information calculated in the previous step, as well as disease risk factor information such as family history and abnormal fetal position. The encoding method can use one-hot encoding for variables that can be divided into two or more categories (such as gender,

[0131] (One-hot Encoding), that is, assuming that an information can be divided into C categories, for category c i The one-hot encoding of can be expressed as

[0132] x=[x1,x2,…,x C ],

[0133] For variables represented by numerical values ​​(such as feature angle, coverage, etc.), normalized coding can be used. Assuming that the input data is x, the numerical range of the diagnostic information is [x min ,x max ], then the normalized parameter value x' is

[0134]

[0135] In the case of pre-training data, Embedding encoding can also be used as a whole to represent the correlation between different clinical information. i The embedded feature vector e i The embedding matrix W∈R N×d , the embedding vector of each category is iteratively optimized through the back-propagation algorithm to minimize the loss function And update the embedding matrix by gradient descent Finally, the optimized embedding representation is obtained.

[0136] The patient follow-up time includes the current age of the infant and the age of the infant when the infant was last examined. The encoding method of the infant's current age can optionally include: absolute time encoding, that is, the total time length T = t max -t min Directly map it to a continuous value so that it is in a uniform range. The normalized parameter value t' is

[0137]

[0138] Time decay coding, that is, using two different decay factors α and β to weight time so that it is close to the weighted average t mean The time region will have a stronger weight, and the attenuation method is

[0139]

[0140] Or sine / cosine periodic encoding, that is, mapping time information into a periodic two-dimensional vector through sine and cosine functions. The specific method is

[0141]

[0142] For the age at the last examination, the time interval Δt between two examinations is normalized and coded. If the last diagnostic data is missing, Δt=-1 or Δt=1 can be optionally set.

[0143] The second prediction model is a Transformer model or other deep learning model. Using the Transformer model, the self-attention mechanism can be used to calculate the correlation between different input features, thereby automatically assigning weights to different features to capture the correlation between cross-modal information such as time.

[0144] When using the second prediction model for prediction, the fused feature vector is mapped through a fully connected layer to predicted disease progression values ​​for the next 4, 8, and 12 weeks. Each predicted value is a continuous value or a multi-class prediction result, representing the degree of disease progression. To simultaneously predict disease progression at multiple time points, the model performs independent regression or classification predictions for each time point.

[0145] As an embodiment, in the predicted value, the continuous numerical value can be a positive probability value of the patient at a future time point; the multi-classification prediction result can be a disease classification of the patient at a future time point.

[0146] As an optional implementation, the training method of the second prediction model is:

[0147] Obtain a labeled fused feature vector dataset;

[0148] Dividing the fused feature vector dataset into a training set and a test set;

[0149] Construct an initial second prediction model, taking the single-channel fusion feature vector as input and the child's hip joint development as output;

[0150] The training set is input into the initial second prediction model to train the model, and the model is tested with the test set until the model converges.

[0151] Corresponding to the method for predicting children's hip joint development, Figure 4 As shown, an embodiment of the present invention further provides a device for predicting hip joint development in children, comprising:

[0152] An acquisition module 10 is used to acquire clinical information of the child and a two-dimensional ultrasound image of the child's hip joint with spatial coordinates;

[0153] A first prediction module 20 is configured to input the two-dimensional ultrasound image of the hip joint into a pre-trained first prediction model to predict the normal distribution probability and predicted value of the child's hip joint development age in weeks;

[0154] a measurement module 30 for constructing a three-dimensional hip model based on the two-dimensional ultrasound image of the hip joint, measuring hip anatomical parameters in the three-dimensional hip model, and obtaining three sets of three-dimensional acetabular dysplasia evaluation factors;

[0155] The second prediction module 40 is used to input the child's clinical information, the prediction result of the child's hip joint development age in weeks, and the acetabular dysplasia evaluation factor into the pre-trained second prediction model to predict the child's hip joint development.

[0156] In an embodiment of the present invention, a two-dimensional ultrasound image of the hip joint is first input into a first prediction model to predict the normal distribution probability and predicted value of the child's hip joint development age in weeks; a three-dimensional hip model is constructed using the two-dimensional ultrasound image of the hip joint, and three sets of three-dimensional acetabular dysplasia evaluation factors are obtained based on the three-dimensional hip model. Finally, the child's clinical information, the predicted results of the child's hip joint development age in weeks, and the acetabular dysplasia evaluation factors are integrated to obtain a prediction result of the child's hip joint development. Compared with traditional prediction methods, the embodiment of the present invention, on the one hand, uses the femoral head of the two-dimensional ultrasound as the ROI, extracts features through the characteristic texture and morphology of the femoral head, judges the child's actual bone age, and aligns the development time; on the other hand, a three-dimensional model is constructed using the marked two-dimensional ultrasound image, and three sets of three-dimensional acetabular dysplasia evaluation factors are obtained in the three-dimensional image. Finally, the two factors are combined to predict the child's development and obtain better prediction results.

[0157] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present invention (including the claims) is limited to these examples. Within the scope of the present invention, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the present invention as described above, which are not provided in detail for the sake of simplicity.

[0158] The present application is intended to cover all such alternatives, modifications, and variations as fall within the broad scope of the appended claims. Accordingly, any and all such alternatives, modifications, equivalents, improvements and the like are intended to be encompassed by the present application.

Claims

1. A method for predicting children's hip joint development, characterized in that: include: Obtain clinical information of children and two-dimensional ultrasound images of the hip joint with spatial coordinates; Inputting the two-dimensional ultrasound image of the hip joint into a pre-trained first prediction model to predict the normal distribution probability and predicted value of the child's hip joint development age in weeks; constructing a three-dimensional model of the hip based on the two-dimensional ultrasound image of the hip joint, measuring hip anatomical parameters in the three-dimensional model of the hip, and obtaining three sets of three-dimensional acetabular dysplasia evaluation factors; The child's clinical information, the predicted result of the child's hip joint development age in weeks, and the acetabular dysplasia evaluation factor are input into a pre-trained second prediction model to predict the child's hip joint development.

2. The method for predicting children's hip joint development according to claim 1, characterized in that: The training method of the first prediction model is: Select multiple sets of hip joint ultrasound images passing near the centroid of the femoral head, and record the age of the child in weeks when the images were collected, to form a hip joint ultrasound image dataset with age labels; Dividing the hip joint ultrasound image dataset into a training set and a test set; An initial first prediction model is constructed, which takes a single-channel hip joint ultrasound image as input and outputs the probability distribution and predicted age value of the hip joint. The training set is input into the initial first prediction model to train the model, and the model is tested with the test set until the model converges.

3. The method for predicting children's hip joint development according to claim 1, wherein: The acetabular dysplasia evaluation factors include a hip joint characteristic angle change curve, a three-dimensional acetabular coverage rate, and a characteristic distance.

4. The method for predicting children's hip joint development according to claim 3, characterized in that: The calculation method of the hip joint characteristic angle change curve is: Based on anatomical principles, the iliac ramus anatomical region and the acetabulum anatomical region are obtained from the three-dimensional hip model by dissection, and the iliac ramus anatomical region and the acetabulum anatomical region are respectively plane-fitted using the least squares method to obtain the closest planes n1 and n2; Solving for an anatomical axis n, which is perpendicular to the planes n1 and n2; Multiple cross-sections of the iliac ramus and acetabulum anatomical regions were taken in the direction perpendicular to the anatomical axis n. Linear fitting was performed on each cross-section using the least squares method to obtain the fitting lines L1 and L2 that were closest to the cross-section. The angle between the two fitting lines L1 and L2 was the characteristic angle α of the iliac ramus and acetabulum anatomical regions on the cross-section. By calculating the characteristic angle on each cross section, the variation curve of the characteristic angle α along the anatomical axis n, as well as the maximum and minimum values ​​of the characteristic angle α can be obtained.

5. The method for predicting children's hip joint development according to claim 3, characterized in that: The calculation method of the three-dimensional acetabulum coverage is: Based on anatomical principles, the anatomical region of the iliac ramus and the anatomical region of the femoral head are obtained from the three-dimensional hip model; The least squares method was used to fit the plane of the iliac ramus anatomical region to obtain the closest plane n1; The volume ratio of the femoral head anatomical area located inside plane n1 was calculated as the acetabulum coverage rate.

6. The method for predicting children's hip joint development according to claim 3, characterized in that: The calculation method of the characteristic distance is: Based on anatomical principles, the femoral head anatomical region is obtained from the three-dimensional hip model, and its center of mass C is obtained; Using a straight line passing through the center of mass C, the straight line intersects the surface of the anatomical region of the femoral head and the surface of the anatomical region of the pubic bone or ischium; Among all possible intersecting straight lines, the straight line segment with the shortest distance between the surface of the femoral head anatomical region and the surface of the pubic bone or ischium anatomical region is obtained, which is the characteristic distance.

7. The method for predicting children's hip joint development according to claim 6, characterized in that: The method for solving the shortest straight line segment is the gradient descent method: Let the line passing through the centroid C be θ, and the distance function be D(θ), which represents the distance between the surface of the anatomical region of the femoral head and the surface of the anatomical region of the pubic bone or ischium. The line with the minimum distance is found by iterating the following formula:

8. The method for predicting children's hip joint development according to claim 1, wherein: The step of inputting the child's clinical information, the predicted result of the child's hip joint development age in weeks, and the acetabular dysplasia evaluation factor into a pre-trained second prediction model to predict the child's hip joint development includes: Acquiring an ultrasonic medical image passing through the centroid of the femoral head, inputting the ultrasonic medical image into a convolutional neural network to obtain a one-dimensional image feature vector with multi-scale features; Encode clinical information to form a one-dimensional clinical feature vector; Encode the patient follow-up time to form a one-dimensional time feature vector; fusing the image feature vector, the clinical feature vector, and the time feature vector into a unified fused feature vector; The fused feature vector is used as input into a second prediction model to predict the development of the child's hip joint at a preset time node in the future.

9. The method for predicting children's hip joint development according to claim 8, characterized in that: The training method of the second prediction model is: Obtain a labeled fused feature vector dataset; Dividing the fused feature vector dataset into a training set and a test set; Construct an initial second prediction model, taking the single-channel fusion feature vector as input and the child's hip joint development as output; The training set is input into the initial second prediction model to train the model, and the model is tested with the test set until the model converges.

10. A device for predicting children's hip joint development, characterized in that: include: an acquisition module, used for acquiring clinical information of the child and a two-dimensional ultrasound image of the child's hip joint with spatial coordinates; A first prediction module is configured to input the two-dimensional ultrasound image of the hip joint into a pre-trained first prediction model to predict the normal distribution probability and predicted value of the child's hip joint development age in weeks; a measurement module, configured to construct a three-dimensional model of the hip based on the two-dimensional ultrasound image of the hip joint, measure hip anatomical parameters in the three-dimensional model of the hip, and obtain three sets of three-dimensional acetabular dysplasia evaluation factors; The second prediction module is used to input the child's clinical information, the prediction result of the child's hip joint development age in weeks, and the acetabular dysplasia evaluation factor into the pre-trained second prediction model to predict the child's hip joint development.

Images