Infant hip joint ultrasonic image segmentation method based on bicoordinate system fusion

Abstract

The invention discloses an infant hip joint ultrasonic image segmentation method based on bicoordinate system fusion, and belongs to the field of medical image processing and ultrasonic images. According to the method, the segmentation precision of the hip joint is improved by combining the advantages of the Cartesian coordinate system and the polar coordinate system. The method comprises the following specific steps: firstly, manually selecting polar coordinate origins of a starting slice and a terminating slice, and then calculating a polar coordinate origin of a middle frame by using a linear interpolation algorithm; then, converting the ultrasonic image of the hip joint of the infant from a Cartesian coordinate system to a polar coordinate system through linear transformation; and finally, taking the converted polar coordinate image and the original Cartesian coordinate image as double inputs, and further improving the segmentation precision of the hip joint by adopting feature screening, alignment and fusion strategies. According to the method, the complementary characteristics of the Cartesian coordinate system and the polar coordinate system are fully utilized, efficient and accurate segmentation of the complex hip joint structure is achieved, the segmentation result is highly consistent with manual labeling, and the method has important clinical application value.

Description

Technical Field

[0001] The present invention belongs to the field of medical image processing and ultrasonic imaging, and more specifically, relates to a method for segmenting ultrasonic images of infant hip joints based on dual coordinate system fusion. Background Art

[0002] As a non-invasive medical imaging technology, infant hip ultrasound has become an important means of diagnosing infant hip developmental diseases and evaluating hip dysplasia. Infancy is a critical period for hip development, and the morphology, structure and position of the hip directly affect the function and health of the hip. Accurately segmenting the boundaries of the hip joint can help doctors more accurately assess the development of the hip joint and promptly detect conditions such as hip dysplasia, acetabular dysplasia or femoral head dislocation, thereby providing infants with scientific and personalized treatment plans to avoid possible joint disorders or osteoarthritis in the future.

[0003] However, ultrasound image segmentation of infant hip joints still faces multiple challenges in practical applications. Traditional manual segmentation methods not only rely on expert experience, but are also time-consuming and easily interfered by subjective factors, making it difficult to ensure the accuracy and consistency of segmentation results. In recent years, automated segmentation methods have been widely studied, but because ultrasound images of infant hip joints usually have problems such as low image resolution, low contrast, and high noise, the existing automated segmentation methods often do not produce ideal segmentation results when processing these images.

[0004] Most of the current automated segmentation methods are based on traditional image processing techniques, such as threshold-based segmentation methods, edge detection methods, and traditional machine learning algorithms. Although these methods can extract the boundary information of the hip joint to a certain extent, due to the complex morphology of infant hip joints and their strong individual differences, these traditional methods are prone to produce erroneous segmentation results when dealing with irregular morphology, blurred local details, or large image noise, resulting in inaccurate division of the hip joint boundary. Image segmentation methods based on deep learning, such as convolutional neural networks (CNNs) and fully convolutional neural networks (FCNs), have significantly improved image segmentation accuracy, but still face the difficulty of reduced segmentation accuracy when dealing with low-resolution and heavily noisy ultrasound images. In addition, most of the current deep learning methods only extract features based on a single coordinate system (such as the Cartesian coordinate system), and fail to fully consider the local features in different coordinate systems, resulting in large errors in the segmentation effect when dealing with hip joint images with special morphology or complex structures. Summary of the invention

[0005] In view of the defects of the prior art and the need for improvement, the present invention provides a method for ultrasonic image segmentation of infant hip joints based on dual coordinate system fusion, the purpose of which is to combine the advantages of Cartesian coordinate system and polar coordinate system, make full use of the structural characteristics of the hip joint in different coordinate systems, and realize the complementarity and fusion of features in different coordinate systems, so as to improve the accuracy and robustness of hip joint segmentation, especially in the early diagnosis and treatment of infant hip joint diseases. Through the feature fusion of dual coordinate systems, the present invention can effectively cope with the challenges of low resolution and high noise of ultrasonic images, overcome the limitations of traditional methods in dealing with complex hip joint morphology, and provide more accurate and reliable segmentation results, thereby providing higher quality image support for clinical practice. This method can significantly improve the accuracy of ultrasonic segmentation of infant hip joints, and thus provide clinicians with a clearer and more accurate basis for pathological judgment.

[0006] To achieve the above object, according to the present invention, a method for segmenting infant hip joint ultrasound images based on dual coordinate system fusion is provided, which is characterized by comprising the following steps:

[0007] (1) Obtain three-dimensional ultrasound images of infant hip joints in a Cartesian coordinate system and divide the image data into a training set and a test set for model training and verification;

[0008] (2) cutting the three-dimensional ultrasound image obtained in step (1) into a plurality of two-dimensional cross-sectional slice images, manually selecting a starting slice and an ending slice and polar coordinate origins on the two slices, and using a linear interpolation algorithm to generate the polar coordinate origin of each slice between the starting slice and the ending slice;

[0009] (3) Generate a two-dimensional ultrasound image in a polar coordinate system using linear transformation based on the polar coordinate origin on each slice. The images in the Cartesian coordinate system and the images in the polar coordinate system constitute the final training set and validation set;

[0010] (4) Design a dual-input segmentation network, input a two-dimensional ultrasound image in a Cartesian coordinate system and a two-dimensional ultrasound image in a polar coordinate system, and output the hip joint segmentation result in a Cartesian coordinate system; and use a multi-loss function joint optimization strategy to train the network;

[0011] (5) In the application stage, the original Cartesian coordinate image and the linearly transformed polar coordinate image are taken as input, and the trained network is used to generate the final segmentation result of the target hip joint.

[0012] As a further preferred embodiment of the present invention, in step (2), the interpolation algorithm is to manually select the coordinates (x1, y1) and (x2, y2) of the polar coordinate origins on the starting slice and the ending slice in the Cartesian coordinate system, and then, for any intermediate frame k, calculate the coordinates of the polar coordinate origin of the intermediate frame by interpolation. Specifically, the interpolation calculation formula is:

[0013] Where N is the total number of frames, k is the index of the current frame, (x1, y1) is the polar coordinate origin coordinate of the starting slice, and (x2, y2) is the polar coordinate origin coordinate of the ending slice.

[0014] The interpolation algorithm automatically calculates the polar coordinate origin position of the intermediate frame between the polar coordinate origin coordinates on the manually selected start slice and the end slice. The polar coordinate origin needs to be manually selected one by one between each frame, which greatly simplifies the manual operation process and reduces the impact of human errors. The polar coordinate origin position of the intermediate frame is calculated by linear interpolation, which can ensure the smooth transition of the polar coordinate origin during the scanning process and avoid jumpy or discontinuous changes. The linear interpolation algorithm is based on the polar coordinate origin coordinates of the start frame and the end frame. The coordinate position in the middle of each frame is efficiently calculated through a simple mathematical formula. It has the advantages of small calculation amount, simple implementation, and strong adaptability. Combined with the characteristics of the three-dimensional ultrasound image of the hip joint, the structure of the image is usually continuous during the scanning process of the hip joint, and has a certain smoothness in space. During the entire scanning process, the geometric shape of the hip joint will gradually change with the change of the scanning angle. Through this interpolation algorithm, this structural continuity of the hip joint can be effectively captured, and the polar coordinate origin position is kept consistent between multiple slices, thereby ensuring the alignment and accuracy of each frame image during the hip joint segmentation process. This method effectively overcomes the image offset and discontinuity problems that may occur in traditional methods, and improves the segmentation accuracy and the accuracy of hip dysplasia assessment in infants and young children.

[0015] As a further preferred embodiment of the present invention, in step (3), the linear transformation is performed at a known polar coordinate origin coordinate (x k ,y k ), the following formula is used to transform the point (x, y) in the Cartesian coordinate system into polar coordinates to obtain the polar diameter ρ and polar angle θ of the point in the polar coordinate system. The specific calculation formula is as follows:

[0016] Where ρ is the polar radius, which represents the distance from the point (x, y) to the origin of the polar coordinates (x k ,y k ); θ is the polar angle, which represents the distance between the point (x, y) and the origin (x k ,y k). This transformation calculates the distance and direction between the Cartesian coordinate system and the polar origin to obtain the corresponding position in the polar coordinate system.

[0017] Linear transformation can accurately capture the distance and direction information from each point in the image to the origin of the polar coordinates, and can provide more accurate transformation results, especially when processing images with strong structural continuity. The simplicity of linear transformation ensures consistency between different frames and avoids calculation errors or image discontinuities that may be caused by other complex transformation methods (such as logarithmic transformation). In this way, the smoothness and continuity of the image can be better maintained, especially when segmenting the hip joint, the edge of the joint can be more accurately located, thereby improving the accuracy of the segmentation results. Compared with more complex transformation methods such as logarithmic transformation, linear transformation does not require additional parameter adjustments or complex calculation steps. It can reduce computational overhead while ensuring transformation accuracy, greatly improving processing speed. Especially in real-time ultrasound image segmentation applications, computational efficiency is crucial. Linear transformation can quickly convert data in the Cartesian coordinate system into the polar coordinate system, thereby effectively supporting rapid image analysis and processing.

[0018] As a further preferred embodiment of the present invention, in the step (4), the dual-input segmentation network is a U-shaped architecture, including an encoder, i.e., a downsampling part, and a decoder, i.e., an upsampling part; both the encoder and the decoder adopt a five-layer structure, wherein the first three layers are convolutional layers, and the last two layers are KAN layers; the dual-coordinate system images are respectively input into the same encoder to extract multi-scale features Cf in the Cartesian coordinate system and multi-scale features Pf in the polar coordinate system; in the feature extraction of each layer, the network first extracts the feature maps Cf in the Cartesian and polar coordinate systems from the dual-coordinate system images respectively. i and Pf i , where i represents the index of the current layer. The extracted dual-coordinate feature maps are superimposed along the channel dimension and input into the feature alignment module (CFSM module). Subsequently, the features processed by the CFSM module are weighted by the channel attention and spatial attention mechanisms through the CCAM module. The fused feature maps are combined with the corresponding upsampling layer through skip connections. The MSEF module is added to the bottleneck layer of the network to extract and fuse features at different scales.

[0019] The dual-input segmentation network used in the present invention combines the feature extraction method of Cartesian coordinate system and polar coordinate system, which effectively improves the accuracy and robustness of hip joint ultrasound image segmentation. Compared with the traditional single coordinate system feature extraction method, the present invention can make full use of the spatial information in different coordinate systems by processing the image features in Cartesian and polar coordinate systems at the same time, thereby improving the ability to accurately segment the hip joint. The feature alignment module (CFSM module) can effectively align the features in Cartesian and polar coordinate systems, remove redundant information, optimize the representation of features, and further improve the accuracy of image segmentation. At the same time, the CCAM module enhances the network's attention to key areas by introducing channel and spatial attention mechanisms, so that the network can better handle complex and subtle hip joint structures. In addition, the multi-scale feature fusion module (MSEF module) enhances the network's modeling ability at different scales through the extraction and fusion of multi-scale information, further improving the segmentation accuracy and the ability to restore image details. The introduction of the KAN layer further enhances the nonlinear modeling ability of the model, helps the network focus on key areas in the image, and improves the segmentation accuracy of complex targets. Through the synergistic effect of the above multiple modules, the present invention can more accurately extract the various levels of features of the hip joint when processing hip joint ultrasound images, ensuring the high quality of the segmentation results, especially when facing complex hip joint structures, showing higher robustness and accuracy.

[0020] In general, the above technical solution conceived by the present invention has the following beneficial effects compared with the prior art:

[0021] 1. The interpolation algorithm adopted in the present invention reduces manual intervention and makes full use of the structural continuity of the hip joint during the three-dimensional ultrasonic image scanning process, making the calculation of the polar coordinate origin smoother and more continuous, thereby effectively improving the quality and accuracy of hip joint segmentation, and has significant technical innovation and practicality.

[0022] 2. The linear transformation method adopted by the present invention not only avoids the computational overhead of complex algorithms through simple and direct calculation steps, but also can accurately and effectively process images with structural continuity, ensuring segmentation accuracy and efficiency. This makes the method have significant advantages in hip joint ultrasound image segmentation and related applications, can effectively improve processing speed, reduce errors, and has broad application prospects.

[0023] 3. The present invention significantly improves the accuracy and robustness of hip joint ultrasound image segmentation by combining a dual-input segmentation network with a feature extraction method of Cartesian coordinate system and polar coordinate system. Compared with traditional methods, the present invention processes features in two coordinate systems at the same time, effectively utilizes spatial information in different coordinate systems, and improves the ability to accurately segment the hip joint. The feature alignment module (CFSM) optimizes the alignment of Cartesian and polar coordinate features, removes redundant information, and improves segmentation accuracy. The CCAM module enhances the network's attention to key areas through channel and spatial attention mechanisms, and improves the processing ability of complex hip joint structures. The multi-scale feature fusion module (MSEF) improves the network's multi-scale modeling capability and further enhances segmentation accuracy. The KAN layer improves the nonlinear modeling capability, helps the network focus on key areas, and ensures high-precision segmentation under complex structures. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A flow chart of the infant hip joint ultrasound image segmentation method based on dual coordinate system fusion provided by the present invention;

[0025] Figure 2 A schematic diagram of the dual-input segmentation network structure provided by the present invention;

[0026] Figure 3 This is a visualization result diagram of the test data set provided by the present invention. DETAILED DESCRIPTION

[0027] Any feature disclosed in this specification, unless otherwise stated, can be replaced by other equivalent or alternative features with similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] The process of the infant hip joint ultrasound image segmentation method based on dual coordinate system fusion provided by the present invention is as follows: Figure 1 As shown, the specific steps include:

[0030] (1) Data preparation: Obtain a publicly available dataset of infant hip joint 3D ultrasound images in a Cartesian coordinate system and perform preprocessing on it, including denoising, contrast enhancement, and other operations. Calibrate the hip joints in the images through manual annotation to provide accurate training labels for the model. Divide the images into training and test sets, usually in a ratio of 8:2.

[0031] (2) Obtaining the polar coordinate origin: First, the original hip joint 3D ultrasound image is cut into several 2D cross-sectional slice images. Then, the starting slice and the ending slice containing the area to be segmented are manually selected, and the polar coordinate origin is manually calibrated at the center of the hip joint. Finally, the polar coordinate origin of the intermediate frame is calculated by linear interpolation.

[0032] (3) Polar coordinate transformation: After the polar coordinate origin is calibrated, each point in the Cartesian coordinate system is transformed into the corresponding polar coordinate using the determined polar coordinate origin through linear transformation. Specifically, by calculating the distance (polar diameter ρ) and angle (polar angle θ) from each point to the polar coordinate origin, each pixel coordinate in the image is transformed into the corresponding value in the polar coordinate system. The original Cartesian coordinate ultrasound image and the transformed polar coordinate ultrasound image constitute the final training set and test set.

[0033] (4) Build and train a dual-input segmentation network, which includes the following steps:

[0034] (4-1) Build Figure 2 The dual-input segmentation network shown in Figure 1 is used. The ultrasound image in the Cartesian coordinate system and the ultrasound image in the polar coordinate system are input into the network respectively. The downsampling part of the network includes 3 convolutional layers and 2 KAN layers, which are used to extract multi-scale features from the two coordinate systems. Next, the input feature map is aligned and fused through a 5-layer feature alignment and fusion module to remove redundant information and optimize feature representation. Then, the extracted fused features are input into the MSEF module to further enhance the feature expression capability. Finally, the fused feature map is upsampled by an upsampling module including 2 KAN layers and 3 convolutional layers to obtain the final segmentation result.

[0035] (4-2) Construct the segmentation weighted loss function as shown below.

[0036] Loss=0.25*BCE(y gt ,y pred )+0.5*DICE(y gt ,y pred )+0.5*cldice(y gt ,y pred )

[0037] Among them, y pred is the prediction result of the segmentation network, y gtis the labeled true value. BCE is the binary cross entropy loss function used to measure the difference between the predicted result and the true label. DICE is the Dice loss function, which measures the overlap between the predicted segmentation area and the true segmentation area and optimizes the segmentation accuracy. cldice is a skeleton-based segmentation loss function, which aims to improve the segmentation accuracy of small structures and edge areas.

[0038] (4-3) The network model is trained for 200k iterations with a batch size of 8. The Adam optimizer is used, beta is set to (0.9, 0.999), and the epsilon value is 1e-8. The initial learning rate is set to 1e-4, the gamma value is 0.5, and the learning rate is updated every 50,000 iterations.

[0039] (5) In the application stage, the original Cartesian coordinate image and the linearly transformed polar coordinate image are used as input, and the trained network is used to generate the final segmentation result of the target hip joint. Figure 3 This is the prediction result of this method on the test set and its corresponding annotation map.

[0040] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for infant hip joint ultrasound image segmentation based on dual coordinate system fusion, characterized in that: The method includes: (1) Obtain three-dimensional ultrasound images of infant hip joints in a Cartesian coordinate system and divide the image data into a training set and a test set for model training and verification; (2) cutting the three-dimensional ultrasound image obtained in step (1) into a plurality of two-dimensional cross-sectional slice images, manually selecting a starting slice and an ending slice and polar coordinate origins on the two slices, and using a linear interpolation algorithm to generate the polar coordinate origin of each slice between the starting slice and the ending slice; (3) Generate a two-dimensional ultrasound image in a polar coordinate system using linear transformation based on the polar coordinate origin on each slice. The images in the Cartesian coordinate system and the images in the polar coordinate system constitute the final training set and validation set; (4) Design a dual-input segmentation network, input a two-dimensional ultrasound image in a Cartesian coordinate system and a two-dimensional ultrasound image in a polar coordinate system, and output the hip joint segmentation result in a Cartesian coordinate system; and use a multi-loss function joint optimization strategy to train the network; (5) In the application stage, the original Cartesian coordinate image and the linearly transformed polar coordinate image are taken as input, and the trained network is used to generate the final segmentation result of the target hip joint.

2. The method for infant hip joint ultrasonic image segmentation based on dual coordinate system fusion as claimed in claim 1, characterized in that: In step (2), the interpolation algorithm is to manually select the coordinates (x1, y1) and (x2, y2) of the polar coordinate origins on the starting slice and the ending slice in the Cartesian coordinate system, and then, for any intermediate frame k, calculate the coordinates of the polar coordinate origin of the intermediate frame by interpolation. The interpolation calculation formula is: Where N is the total number of frames, k is the index of the current frame, (x1, y1) is the polar coordinate origin coordinate of the starting slice, and (x2, y2) is the polar coordinate origin coordinate of the ending slice.

3. The method for infant hip joint ultrasonic image segmentation based on dual coordinate system fusion as claimed in claim 1, characterized in that: In the step (3), the linear transformation is to obtain the coordinates of the origin of the polar coordinates (x k ,y k ), the following formula is used to transform the point (x, y) in the Cartesian coordinate system into polar coordinates to obtain the polar diameter ρ and polar angle θ of the point in the polar coordinate system. The specific calculation formula is as follows: Where ρ is the polar radius, which represents the distance from the point (x, y) to the origin of the polar coordinates (x k ,y k ); θ is the polar angle, which represents the distance between the point (x, y) and the origin (x k ,y k ) angle; this transformation obtains the corresponding position in the polar coordinate system by calculating the distance and direction between the position of the point in the Cartesian coordinate system and the polar coordinate origin.

4. The method for infant hip joint ultrasonic image segmentation based on dual coordinate system fusion according to claim 1, characterized in that: In the step (4), the dual-input segmentation network is a U-shaped architecture, including an encoder, i.e., a downsampling part, and a decoder, i.e., an upsampling part; both the encoder and the decoder adopt a five-layer structure, wherein the first three layers are convolutional layers, and the last two layers are KAN layers; The dual coordinate system images are input into the same encoder to extract the multi-scale features Cf in the Cartesian coordinate system and the multi-scale features Pf in the polar coordinate system; In the feature extraction of each layer, the network first extracts the feature maps Cf in Cartesian and polar coordinate systems from the dual coordinate system image. i and Pf i , where i represents the index of the current layer, and the extracted dual coordinate system feature map is superimposed along the channel dimension and input into the feature alignment module; Subsequently, the features processed by the CFSM module will be weighted by the channel attention and spatial attention mechanisms through the CCAM module, and the fused feature map will be combined with the corresponding upsampling layer through jump connections; the MSEF module is added to the bottleneck layer of the network to extract and fuse features at different scales.

Images