Hip joint auxiliary detection method and device based on multi-task learning

Abstract

This invention provides a hip joint assisted detection method and apparatus based on multi-task learning. The method includes: acquiring a standard planar ultrasound image of the hip joint to be analyzed; extracting hip joint features from the standard planar ultrasound image based on a preset shared representation module to obtain a multi-scale feature map corresponding to the standard planar ultrasound image; performing multi-task collaborative processing on the multi-scale feature map corresponding to the standard planar ultrasound image based on a preset multi-task collaboration module to obtain a segmentation structure map, skeletal key points, and Graf lines corresponding to the standard planar ultrasound image; and performing multi-task cross-enhancement processing on the segmentation structure map, skeletal key points, and Graf lines based on a preset cross-enhancement module to obtain the corresponding detection results. The hip joint assisted detection method based on multi-task learning provided by this invention can effectively improve the automation, detection speed, and robustness of assisted detection of hip joint images.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a hip joint-assisted detection method and apparatus based on multi-task learning. Additionally, it relates to an electronic device and a processor-readable storage medium. Background Technology

[0002] Developmental dysplasia of the hip (DDH) is one of the most common and complex diseases in neonatal orthopedics, affecting approximately 3% of children. Most scholars believe that delayed diagnosis and treatment, even with complex surgical interventions, makes it difficult to restore the normal structure, morphology, and function of the hip joint. Therefore, early detection is crucial for the normal development of the hip joint in children. Many challenges exist in the ultrasound examination of DDH. Factors such as ultrasound noise, unclear bone structure, inherent skeletal diversity, and subjective results lead to variability in angle measurements. Consequently, the demand for intelligent, assisted detection methods for DDH is increasing. Therefore, it is necessary to develop intelligent detection methods to assist physicians in detecting DDH and improve detection efficiency.

[0003] However, existing technologies struggle to handle particularly complex situations. Deep learning plays an increasingly important role in medical image analysis, offering significant speed improvements over traditional image processing methods and enabling fully automated dysplasia detection. However, the main limitations of using deep learning algorithms solely for hip joint detection lie in poor interpretability and robustness, while also neglecting the complex relationships between different hip joint tasks. Therefore, designing a more efficient multi-task learning-based hip joint detection scheme is a pressing issue that needs to be addressed. Summary of the Invention

[0004] Therefore, the present invention provides a hip joint assisted detection method and device based on multi-task learning to solve the defects of existing hip joint assisted detection schemes, which have high limitations, resulting in poor hip joint assisted detection effect and robustness.

[0005] In a first aspect, the present invention provides a hip joint assisted detection method based on multi-task learning, comprising:

[0006] Obtain a standard planar ultrasound image of the hip joint to be analyzed;

[0007] Based on the preset shared representation module, hip joint features are extracted from the standard planar image of hip joint ultrasound to obtain a multi-scale feature map corresponding to the standard planar image of hip joint ultrasound.

[0008] Based on a preset multi-task collaborative module, the multi-scale feature map corresponding to the standard planar image of hip joint ultrasound is processed in a multi-task collaborative manner to obtain the segmentation structure map, skeletal key points and Graf lines corresponding to the standard planar image of hip joint ultrasound.

[0009] Based on the preset cross-enhancement module, multi-task cross-enhancement processing is performed on the segmentation structure map, the skeletal key points, and the Graf lines to obtain the corresponding detection results.

[0010] Furthermore, the hip joint feature extraction is performed on the standard planar image of the hip joint ultrasound based on the preset shared representation module to obtain a multi-scale feature map corresponding to the standard planar image of the hip joint ultrasound, specifically including:

[0011] The standard planar image of the hip joint ultrasound is input into four parallel network branches of a preset shared representation module to obtain feature maps at four scales. Based on the feature maps at the four scales, size data is obtained by bilinear interpolation upsampling and then stitched together to obtain a multi-scale feature map corresponding to the standard planar image of the hip joint ultrasound. The four parallel network branches are used to reduce the size of the feature map layer by layer while increasing the number of channels. The shared representation module is based on a deep neural network model used for human pose estimation and image segmentation.

[0012] Furthermore, the preset multi-task collaborative module performs multi-task collaborative processing on the multi-scale feature map corresponding to the standard planar image of the hip joint ultrasound to obtain the segmentation structure map, skeletal key points, and Graf lines corresponding to the standard planar image of the hip joint ultrasound, specifically including:

[0013] Based on a preset multi-task collaborative module, anatomical structure segmentation processing is performed on the multi-scale feature map corresponding to the standard planar ultrasound image of the hip joint to obtain the segmented structure map corresponding to the standard planar ultrasound image of the hip joint; and, using heatmap regression, key point recognition processing is performed on the multi-scale feature map corresponding to the standard planar ultrasound image of the hip joint to obtain the skeletal key points corresponding to the standard planar ultrasound image of the hip joint; and, line detection processing is performed on the multi-scale feature map corresponding to the standard planar ultrasound image of the hip joint to obtain the Graf line corresponding to the standard planar ultrasound image of the hip joint.

[0014] Furthermore, the preset cross-enhancement module performs multi-task cross-enhancement processing on the segmentation structure map, the skeletal key points, and the Graf lines to obtain corresponding detection results, specifically including:

[0015] Based on the preset cross-enhancement module, feature enhancement processing is performed on the spatial dependency relationship between the segmentation structure map and the skeletal key points, and feature enhancement processing is performed on the spatial dependency relationship between the skeletal key points and the Graf line to obtain the corresponding detection results.

[0016] Furthermore, the preset cross-enhancement module performs feature enhancement processing on the spatial dependency relationship between the segmentation structure map and the skeletal keypoints, and performs feature enhancement processing on the spatial dependency relationship between the skeletal keypoints and the Graf lines to obtain corresponding detection results, specifically including:

[0017] Based on the preset cross-enhancement module, feature enhancement processing is performed on the spatial dependency relationship between the segmentation structure map and the skeletal key points, and feature enhancement processing is performed on the spatial dependency relationship between the skeletal key points and the Graf line to obtain the hip joint feature map after feature enhancement processing.

[0018] The corresponding detection results are determined based on the angle formed between the Graf lines in the hip joint feature map.

[0019] Furthermore, the preset cross-enhancement module performs feature enhancement processing on the spatial dependency relationship between the segmented structure map and the skeletal key points, and performs feature enhancement processing on the spatial dependency relationship between the skeletal key points and the Graf line to obtain a hip joint feature map after feature enhancement, specifically including:

[0020] Based on a preset cross-enhancement module, feature enhancement processing is performed on the spatial dependency relationship between the segmentation structure map and the skeletal key points, and feature enhancement processing is performed on the spatial dependency relationship between the skeletal key points and the Graf line to obtain the feature-enhanced segmentation structure map, skeletal key points, and Graf line; the feature-enhanced segmentation structure map, skeletal key points, and Graf line are then superimposed to obtain the feature-enhanced hip joint feature map.

[0021] Furthermore, determining the corresponding detection result based on the angle formed between the Graf lines in the hip joint feature map specifically includes:

[0022] The first and second angles, which represent hip joint features, formed between Graf lines in the hip joint feature map, are compared with preset angle thresholds to obtain angle comparison results. Based on the angle comparison results, the corresponding detection results are determined.

[0023] Secondly, the present invention also provides a hip joint assistive detection device based on multi-task learning, comprising:

[0024] The image acquisition unit is used to acquire standard planar ultrasound images of the hip joint to be analyzed.

[0025] A shared representation unit is used to extract hip joint features from the standard planar image of hip joint ultrasound based on a preset shared representation module, and obtain a multi-scale feature map corresponding to the standard planar image of hip joint ultrasound.

[0026] The multi-task collaborative unit is used to perform multi-task collaborative processing on the multi-scale feature map corresponding to the standard planar image of hip joint ultrasound based on a preset multi-task collaborative module, so as to obtain the segmentation structure map, skeletal key points and Graf lines corresponding to the standard planar image of hip joint ultrasound.

[0027] The cross-enhancement unit is used to perform multi-task cross-enhancement processing on the segmentation structure map, the skeletal key points, and the Graf lines based on a preset cross-enhancement module to obtain the corresponding detection results.

[0028] Furthermore, the shared representation module specifically includes:

[0029] The standard planar image of the hip joint ultrasound is input into four parallel network branches of a preset shared representation module to obtain feature maps at four scales. Based on the feature maps at the four scales, size data is obtained by bilinear interpolation upsampling and then stitched together to obtain a multi-scale feature map corresponding to the standard planar image of the hip joint ultrasound. The four parallel network branches are used to reduce the size of the feature map layer by layer while increasing the number of channels. The shared representation module is based on a deep neural network model used for human pose estimation and image segmentation.

[0030] Furthermore, the multi-task collaboration module specifically includes:

[0031] Based on a preset multi-task collaborative module, anatomical structure segmentation processing is performed on the multi-scale feature map corresponding to the standard planar ultrasound image of the hip joint to obtain the segmented structure map corresponding to the standard planar ultrasound image of the hip joint; and, using heatmap regression, key point recognition processing is performed on the multi-scale feature map corresponding to the standard planar ultrasound image of the hip joint to obtain the skeletal key points corresponding to the standard planar ultrasound image of the hip joint; and, line detection processing is performed on the multi-scale feature map corresponding to the standard planar ultrasound image of the hip joint to obtain the Graf line corresponding to the standard planar ultrasound image of the hip joint.

[0032] Furthermore, the cross-enhancement module specifically includes:

[0033] Based on the preset cross-enhancement module, feature enhancement processing is performed on the spatial dependency relationship between the segmentation structure map and the skeletal key points, and feature enhancement processing is performed on the spatial dependency relationship between the skeletal key points and the Graf line to obtain the corresponding detection results.

[0034] Furthermore, the preset cross-enhancement module performs feature enhancement processing on the spatial dependency relationship between the segmentation structure map and the skeletal keypoints, and performs feature enhancement processing on the spatial dependency relationship between the skeletal keypoints and the Graf lines to obtain corresponding detection results, specifically including:

[0035] Based on the preset cross-enhancement module, feature enhancement processing is performed on the spatial dependency relationship between the segmentation structure map and the skeletal key points, and feature enhancement processing is performed on the spatial dependency relationship between the skeletal key points and the Graf line to obtain the hip joint feature map after feature enhancement processing.

[0036] The corresponding detection results are determined based on the angle formed between the Graf lines in the hip joint feature map.

[0037] Furthermore, the preset cross-enhancement module performs feature enhancement processing on the spatial dependency relationship between the segmented structure map and the skeletal key points, and performs feature enhancement processing on the spatial dependency relationship between the skeletal key points and the Graf line to obtain a hip joint feature map after feature enhancement, specifically including:

[0038] Based on a preset cross-enhancement module, feature enhancement processing is performed on the spatial dependency relationship between the segmentation structure map and the skeletal key points, and feature enhancement processing is performed on the spatial dependency relationship between the skeletal key points and the Graf line to obtain the feature-enhanced segmentation structure map, skeletal key points, and Graf line; the feature-enhanced segmentation structure map, skeletal key points, and Graf line are then superimposed to obtain the feature-enhanced hip joint feature map.

[0039] Furthermore, determining the corresponding detection result based on the angle formed between the Graf lines in the hip joint feature map specifically includes:

[0040] The first and second angles, which represent hip joint features, formed between Graf lines in the hip joint feature map, are compared with preset angle thresholds to obtain angle comparison results. Based on the angle comparison results, the corresponding detection results are determined.

[0041] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the hip joint assisted detection method based on multi-task learning as described above.

[0042] Fourthly, the present invention also provides a processor-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the hip joint assisted detection method based on multi-task learning as described in any of the above claims.

[0043] The hip joint assisted detection method based on multi-task learning provided by this invention acquires a standard planar ultrasound image of the hip joint to be analyzed; extracts hip joint features from the standard planar ultrasound image based on a preset shared representation module to obtain a multi-scale feature map corresponding to the standard planar ultrasound image of the hip joint; performs multi-task collaborative processing on the multi-scale feature map corresponding to the standard planar ultrasound image of the hip joint based on a preset multi-task collaboration module to obtain a segmentation structure map, skeletal key points, and Graf lines corresponding to the standard planar ultrasound image of the hip joint; and performs multi-task cross-enhancement processing on the segmentation structure map, skeletal key points, and Graf lines based on a preset cross-enhancement module to obtain the corresponding detection results. This method can effectively improve the automation, detection speed, and robustness of assisted detection of hip joint images. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart illustrating the hip joint assisted detection method based on multi-task learning provided in an embodiment of the present invention.

[0046] Figure 2 This is the second flowchart of the hip joint assisted detection method based on multi-task learning provided in this embodiment of the invention;

[0047] Figure 3 This is the third flowchart of the hip joint assisted detection method based on multi-task learning provided in this embodiment of the invention;

[0048] Figure 4 This is a schematic diagram of the model structure provided in the embodiment of the present invention;

[0049] Figure 5This is one of the schematic diagrams of hip joint features provided in the embodiments of the present invention;

[0050] Figure 6 This is a schematic diagram of the structure of the hip joint assistive detection device based on multi-task learning provided in an embodiment of the present invention;

[0051] Figure 7 This is a second schematic diagram of the hip joint feature diagram provided in the embodiment of the present invention;

[0052] Figure 8 This is a schematic diagram of the physical structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] This invention proposes an intelligent auxiliary detection technology for hip dysplasia based on ultrasound images. For example... Figure 2 As shown, the input of this invention is a standard planar image of the hip joint ultrasound, and the output is the detection result for hip dysplasia. This invention includes three modules: 1) a shared representation module, which extracts hip joint features using a high-resolution HRNet-based shared representation module; 2) a multi-task collaboration module, which includes network architectures corresponding to anatomical structure segmentation, key point recognition and detection, and Graf line detection tasks; and 3) a cross-enhancement module, which improves detection accuracy by utilizing the spatial dependencies of the segmented structure map, skeletal key points, and Graf lines. The relationships between the modules are as follows: Figure 3 As shown, the input standard planar ultrasound image of the hip joint is processed by the shared representation module to extract multi-scale information of the standard planar ultrasound image of the hip joint, thereby obtaining the multi-scale feature map corresponding to the standard planar ultrasound image of the hip joint. Then, the multi-scale information of the standard planar ultrasound image of the hip joint (i.e., the multi-scale feature map corresponding to the standard planar ultrasound image of the hip joint) is input into the multi-task collaborative module, which simultaneously performs anatomical structure segmentation, skeletal key point recognition, and Graf line detection tasks. The correlation between the three tasks is used to improve the overall performance. The output results of the three tasks are processed by the cross-enhancement module to obtain the final detection result.

[0055] The following section first describes in detail the embodiments of the hip joint assisted detection method based on multi-task learning described in this invention. For example... Figure 1 The diagram shown is a flowchart of the hip joint assisted detection method based on multi-task learning provided in an embodiment of the present invention. The specific implementation process includes the following steps:

[0056] Step 101: Obtain the standard planar ultrasound image of the hip joint to be analyzed.

[0057] Specifically, in this embodiment of the invention, a large number of initial hip joint ultrasound images can first be acquired. By standardizing the hip joint ultrasound images, a standard plane image of the hip joint to be analyzed, namely the DDH Standard Plane, is obtained.

[0058] Step 102: Extract hip joint features from the standard planar image of hip joint ultrasound based on the preset shared representation module to obtain the multi-scale feature map corresponding to the standard planar image of hip joint ultrasound.

[0059] In this embodiment of the invention, the standard planar image of the hip joint ultrasound can be input into a preset shared characterization module (i.e., Figure 3 The shared representation module (in the shared standard module) uses four parallel network branches to obtain feature maps at four scales. Based on these four scale feature maps, bilinear interpolation upsampling is used to obtain size data, which is then stitched together to obtain a multi-scale feature map corresponding to the standard planar image of the hip joint ultrasound. The four parallel network branches are used to progressively reduce the size of the feature maps while increasing the number of channels. The shared representation module is based on a deep neural network model used for human pose estimation and image segmentation. It should be noted that the shared representation module of this invention uses the backbone architecture of HRNet to enhance the resolution feature maps. This shared representation module can be obtained through iterative training using training samples and corresponding labels that are similar to the standard planar image of the hip joint ultrasound. HRNet maintains high-resolution representation throughout the process through parallel multi-resolution convolution and repeated multi-resolution fusion. Figure 4As shown, the Shared Representation Module (HRNet) consists of four parallel branches, where the final output feature map size gradually decreases from the top branch to the bottom branch. The four parallel branches of HRNet progressively reduce the feature map size while increasing the number of channels. Assuming (H, W) represents the size of the original resolution feature map and C is the number of channels, the final output feature maps of the four sizes are (H / 2, W / 2), (H / 4, W / 4), and (H / 8, W / 8), with the number of channels being C, 2C, 4C, and 8C, respectively. This design effectively extracts feature information at different scales in the image. To combine multi-scale feature information, the four scale feature maps output by HRNet are upsampled to the (H, W) size using bilinear interpolation and then concatenated to obtain a multi-scale feature map, which is then passed to the multi-task collaborative module. Feature maps of different resolutions can capture information at different scales in the image. The HRNet network architecture can be used for branches or modules for hip joint feature extraction, enabling the network to process hip joint features and other visual features simultaneously.

[0060] Step 103: Based on the preset multi-task collaborative module, perform multi-task collaborative processing on the multi-scale feature map corresponding to the standard planar ultrasound image of the hip joint to obtain the segmentation structure map, skeletal key points and Graf lines corresponding to the standard planar ultrasound image of the hip joint.

[0061] In this embodiment of the invention, an anatomical structure segmentation process can be performed on the multi-scale feature map corresponding to the standard planar ultrasound image of the hip joint based on a preset multi-task collaborative module to obtain a segmented structure map corresponding to the standard planar ultrasound image of the hip joint; and, key point recognition processing can be performed on the multi-scale feature map corresponding to the standard planar ultrasound image of the hip joint using heatmap regression to obtain skeletal key points corresponding to the standard planar ultrasound image of the hip joint; and, line detection processing can be performed on the multi-scale feature map corresponding to the standard planar ultrasound image of the hip joint to obtain Graf lines corresponding to the standard planar ultrasound image of the hip joint. The multi-task collaborative module is obtained through a multi-task learning method, that is, iterative training using training samples and corresponding labels that are similar to the multi-scale feature map corresponding to the standard planar ultrasound image of the hip joint. Figure 4 As shown, the multi-task collaboration module is called the Multi-task Collaboration Module.

[0062] It should be noted that the three tasks—anatomical structure segmentation, skeletal keypoint recognition, and Graf line detection—are similar. Using Gaussian distribution heatmaps to represent target location and shape, and understanding and extracting the local structure and features of multi-scale feature maps, helps in the accurate localization and detection of the hip joint. Therefore, using multi-task learning methods can uncover similarities and commonalities by learning the parameters of shared model parts. This allows the model to simultaneously learn shared features and knowledge across multiple related tasks, thereby achieving more efficient and accurate multi-task collaborative model training and prediction, resulting in a multi-task collaborative module (i.e., a multi-task collaborative model). Furthermore, multi-task learning can improve data utilization efficiency and reduce the time cost of multi-task collaborative model training and inference. Below is a separate introduction to the three tasks:

[0063] Anatomical segmentation task, i.e., bone segmentation: such as Figure 4 and 5 As shown, four bones S1, S2, S3, and S4 are segmented. Regarding the network architecture, multi-scale feature maps are used as input, passing through two convolutional layers in the network architecture corresponding to the anatomical structure segmentation task in the multi-task collaborative model. The kernel size of each convolution is 1x1. Figure 4 The `conv(1x1)` function in the code achieves comprehensive feature fusion. The output is a function with N... class A segmentation score map (i.e., segmentation structure map) with channels and a size of (H, W) is generated. The first loss function is defined as follows: in Let represent the normalized prediction score of class c at position (i, j) in the segmentation structure map. This represents the normalized predicted actual score of category c at position (i, j) of the segmentation structure map. The network architecture corresponding to the anatomical structure segmentation task in the multi-task collaborative model can be obtained through constrained training using the first loss function and the corresponding training data and labels.

[0064] Skeletal landmark recognition task, i.e., Landmark Localization: such as Figure 4 and 5 As shown, six skeletal keypoints P1, P2, P3, P4, P5, and P6, required for Graf classification, are selected as the recognition targets. Compared to directly predicting keypoint coordinates, heatmap regression is used, which is more robust. Each skeletal keypoint is first used as a heatmap center, and the heatmap values ​​of other points are Gaussian-decreasing according to their distance from the skeletal keypoint. The specific formula is as follows: Where (x) * y 8 H(x, y) represents the coordinates of the key points of the skeleton, and H(x, y) represents the target heatmap H.M The heat value at any point (x, y) is given, where σ is the standard deviation of the Gaussian distribution, and σ = 1.5 is used. The network architecture for skeletal keypoint recognition is similar to that described in the anatomical structure segmentation task. First, the multi-scale feature map is passed through two 1x1 convolutional layers in the network architecture corresponding to the skeletal keypoint recognition task in the multi-task collaborative model. Figure 4 In conv(1x1), we get N landmark Predicted heatmap with channels and dimensions (H, W) Based on the heat values ​​in the predicted heatmap, the points with the highest heat concentration are identified as skeletal keypoints. Specifically, there are 6 skeletal keypoints that need to be regressed, therefore N landmark It is set to 6. Each channel in the predicted heatmap corresponds to the prediction of a specific marker point, whose coordinates (i.e., skeletal keypoints) can be obtained by selecting the coordinates of the maximum value in the heatmap. The second loss function of the heatmap regression involves calculating the mean squared error loss between the predicted and target heatmaps, as follows: The network architecture corresponding to the skeletal key point recognition task in the multi-task collaborative model can be obtained by constrained training based on the second loss function and the corresponding training data and labels.

[0065] Graf line detection task, or Line Delection: In the Graf fractal method, angle evaluation requires determining the angle between three lines, as shown in the reference... Figure 5 Therefore, a detection task is performed on the three lines involved in the Graf classification method. Before line detection, the lines are first transformed to facilitate prediction. There are multiple ways to represent a line in a Cartesian coordinate system. A line can be represented by the coordinates of two points, or by the intercept of the line and a point on the line. Both of these methods involve the coordinates of specific points, but since there are countless points on a line, it is difficult to select a representative point for prediction. Therefore, this invention adopts a polar coordinate system, using polar angles and polar radii to represent the line. In order to represent the line using polar angles and polar radii, the line needs to be transformed. Figure 7This paper uses line L2 as an example to illustrate how to transform a straight line to aid in neural network prediction. First, a coordinate system is established with the bottom left corner of the image as the origin O. Then, starting from the origin, a perpendicular line OK2 is drawn from line L2 through the origin, where K2 represents the intersection of the perpendicular line and line L2. Since a line has only one perpendicular line through the origin, line OK2 can be used to represent line L2. Finally, the polar angle θ2 of the perpendicular line OK2 is calculated by rotating counterclockwise around the x-axis. Using the length of K2 at the origin O as the polar radius ρ2, the polar coordinate expression (ρ2, θ2) of line L2 is obtained. Similarly, the polar coordinates (ρ1, θ1) and (ρ3, θ3) of two Graf lines L1 and L3 can be obtained. Ultimately, the prediction target can be represented as a 6-dimensional vector y. l =(ρ1, ρ2, ρ3, θ1, θ2, θ3).

[0066] In terms of network architecture, multi-scale features are passed through two convolutional layers with a kernel size of 3 and a stride of 3 in the network architecture corresponding to the skeleton Graf line detection task in the multi-task collaborative model. Figure 4 The conv(3x3) is then passed through a linear layer to obtain a prediction vector of length 6. Based on the prediction vector The Graf line corresponding to the standard planar ultrasound image of the hip joint is determined. The third loss function uses the mean squared error (MSE) error, as shown below: The network architecture for the skeleton Graf line detection task in the multi-task collaborative model can be obtained by constrained training using the third loss function and the corresponding training data and labels.

[0067] Step 104: Perform multi-task cross-enhancement processing on the segmentation structure map, the skeletal key points, and the Graf lines based on the preset cross-enhancement module to obtain the corresponding detection results.

[0068] In the preceding multi-task collaboration module, the correlation between the three tasks was implicitly utilized. This section's multi-task cross-enhancement module, however, explicitly utilizes the spatial correlation between different tasks to further improve prediction accuracy. This multi-task cross-enhancement module consists of two phases: training and inference. During training, auxiliary tasks and auxiliary loss functions are constructed based on the spatial dependencies between skeletal segmentation structures, skeletal keypoints, and Graf lines to improve prediction accuracy. During inference, the results of structural segmentation, skeletal keypoint inspection, and Graf line regression are comprehensively utilized to obtain more precise α and β angles, and to more accurately diagnose hip joint diseases. The multi-task cross-enhancement module will be described in detail below, focusing on both the training and inference phases.

[0069] Two additional auxiliary loss functions were used during the training phase. The multi-task cross-enhancement module during training is divided into two parts: structure-point enhancement and point-line enhancement. These utilize the spatial association between skeletal structure and skeletal keypoints, and the spatial association between skeletal keypoints and Graf lines, respectively, to construct auxiliary loss functions for enhancement. Structure-point enhancement: (e.g., ...) Figure 5 As shown, enhancement is performed based on the association between the three segmentation structure diagrams and their corresponding skeletal keypoints. Specifically, firstly, the skeletal keypoint P4 is obtained by utilizing the lowermost point of the anatomical structure S3, the lowermost point of S1, and the geometric center of S4 in the segmentation structure diagrams. ′ P5 ′ P6 ′ Next, based on the skeletal key points extracted from the anatomical structure, a heatmap H is constructed, modeled after the skeletal key point recognition task in the multi-task collaborative module. S Finally, the predicted heatmap is used. Thermograph H of the structure S The regression loss is calculated using the fourth loss function as follows: Point-line enhancement: such as Figure 5 As shown, enhancement is achieved by leveraging the relationship between skeletal keypoints and Graf lines. First, for a predicted skeletal keypoint P, its projection P′ onto the predicted Graf line is calculated. Then, following the method mentioned in the skeletal keypoint recognition task of the multi-task collaborative module, a heatmap H of P′ is constructed. L Then use the predicted heatmap Thermograph H of the structure L The regression loss is calculated using the fifth loss function as follows: The fourth and fifth loss functions obtained from the multi-task cross-enhancement module, along with the first, second, and third loss functions obtained from the previous multi-task collaboration module, are added together to obtain the total loss function for the training phase, as follows: L total =L s +L M +L L +L S-M +L M-L The neural network can be trained using this loss function, along with the corresponding training data and labels.

[0070] During the inference phase, the multi-task cross-enhancement module is responsible for integrating the results of three tasks: structural segmentation, keypoint inspection, and linear regression, calculating the α and β angles, and determining hip joint disease. The specific process is as follows: First, based on the structure-point correlation mentioned in the cross-enhancement module, keypoint information can be extracted from the structural segmentation results, and then... In this diagram, the subscript i represents the i-th keypoint, and there are a total of 6 skeletal keypoints. The superscript S indicates that these are skeletal keypoints extracted from the structural segmentation results. Similarly, based on the point-line association mentioned in the cross-enhancement module, keypoint information can be extracted from the Graf line prediction results. The subscript 'i' has the same meaning as above, and the superscript 'L' indicates that this is the skeletal keypoint information extracted from the Graf line; in addition, the skeletal keypoint detection task itself also predicts the coordinates of the keypoints. The superscript M indicates that this is the skeletal keypoint information output by the skeletal keypoint detection task. The final keypoint prediction is obtained by averaging and summing the coordinates of these three different sources of skeletal keypoints. like Figure 4 Based on the predicted coordinates of the six key points, the predicted angles of lines L1, L2, and L3 can be calculated. The first angle used to assess hip joint disease Second angle Predict and By comparing the angle with the medical standard threshold, a diagnosis of hip dysplasia can be obtained.

[0071] The training data preprocessing includes training data collection, training data processing, and training label processing. Training data collection involves collecting 1211 standard planar ultrasound images of the hip joint from a hospital. Each image is labeled with 6 skeletal key points and 3 anatomical structures using the Graf classification method by experienced personnel, and the α and β angles are calculated. Finally, hip dysplasia is detected for each standard planar ultrasound image. The data is divided into training, validation, and test sets in a 7:1:2 ratio. Training data processing includes standardizing the values ​​of each standard planar ultrasound image to between 0 and 1. The size of the standard planar ultrasound images is adjusted to 512*512 using bilinear interpolation. The standard planar ultrasound images in the training set are randomly cropped and noise-enhanced. Training label processing includes: a pixel-by-pixel 4-class classification task, including background and 3 anatomical structure categories; and a skeletal key point recognition task with 6 key points, constructed using a Gaussian distribution with σ=1.5, and a heatmap is created as a label for the skeletal key points. The Graf line detection task calculates the polar coordinates of a line using key points, which serve as the label.

[0072] The evaluation metrics for the implementation results are described below. For the detection of hip dysplasia, angular error, Graf classification accuracy, and disease detection accuracy are used for evaluation. In the anatomical segmentation task, the intersection-over-union (IoU) metric is used, which is expressed as follows: In keypoint recognition tasks, point error (PE) and success rate (SDR) are used as metrics. The point error of the nth point is... Where P n This represents the predicted coordinates of the nth point. Table i represents the true coordinates of the nth point, PE n This represents the Euclidean distance between the predicted and actual coordinates. Success rate. Where D represents the distance threshold, the detection is considered successful when the distance between the predicted point and the ground truth point is less than the threshold. The threshold D is related to the image resolution; in the experiments of this invention, D is taken as 5 pixels. These two metrics ultimately need to be converted from pixel units to actual physical distance. In the experimental images of this invention, the actual distance per pixel unit is 0.1 millimeters.

[0073] Results: To evaluate the performance of the method for detecting hip dysplasia, it was compared with the best baseline method in terms of accuracy of α and β angles and accuracy of disease classification. The results are shown in Table 1 below.

[0074] Table 1. Disease Classification Performance Table

[0075]

[0076] Compared to the classic UNet in the ultrasound field, this method significantly improves across all metrics. Mask R-CNN can perform multi-task training for segmentation and keypoint detection, and this method achieves better performance than Mask R-CNN through cross-enhancement between tasks. HR-seg, HR-landmark, and HR-line represent single-task tasks of segmentation, keypoint recognition, and line detection using the HRNet structure, respectively. This method also shows consistent improvement over single-task tasks through multi-task collaboration and cross-enhancement. As shown in Table 2, this invention outperforms HRNet and Mask R-CNN in terms of PE and SDR. This is because this invention utilizes the spatial dependencies between keypoints and skeletons and lines to enhance the position of keypoints. This further validates the effectiveness of the cross-enhancement module.

[0077] Table 2 Performance of Skeletal Key Point Recognition

[0078]

[0079] The hip joint assisted detection method based on multi-task learning described in this invention acquires a standard planar ultrasound image of the hip joint to be analyzed; extracts hip joint features from the standard planar ultrasound image based on a preset shared representation module to obtain a multi-scale feature map corresponding to the standard planar ultrasound image; performs multi-task collaborative processing on the multi-scale feature map corresponding to the standard planar ultrasound image based on a preset multi-task collaboration module to obtain a segmentation structure map, skeletal key points, and Graf lines corresponding to the standard planar ultrasound image; and performs multi-task cross-enhancement processing on the segmentation structure map, skeletal key points, and Graf lines based on a preset cross-enhancement module to obtain the corresponding detection results. This method can effectively improve the automation, detection speed, and robustness of assisted detection of hip joint images.

[0080] Corresponding to the multi-task learning-based hip joint assistive detection method provided above, this invention also provides a multi-task learning-based hip joint assistive detection device. Since the embodiments of this device are similar to the above method embodiments, the description is relatively simple. For relevant details, please refer to the description in the above method embodiment section. The embodiments of the multi-task learning-based hip joint assistive detection device described below are merely illustrative. Please refer to... Figure 6 The diagram shown is a structural schematic of a hip joint assistive detection device based on multi-task learning provided in an embodiment of the present invention. The hip joint assistive detection device based on multi-task learning according to the present invention includes the following parts:

[0081] Image acquisition unit 601 is used to acquire standard planar ultrasound images of the hip joint to be analyzed;

[0082] The shared representation module 602 is used to extract hip joint features from the standard planar image of the hip joint ultrasound based on a preset shared representation module, and obtain a multi-scale feature map corresponding to the standard planar image of the hip joint ultrasound.

[0083] The multi-task collaboration module 603 is used to perform multi-task collaborative processing on the multi-scale feature map corresponding to the standard planar image of hip joint ultrasound based on a preset multi-task collaboration module, so as to obtain the segmentation structure map, skeletal key points and Graf lines corresponding to the standard planar image of hip joint ultrasound.

[0084] The cross-enhancement module 604 is used to perform multi-task cross-enhancement processing on the segmentation structure map, the skeletal key points and the Graf lines based on a preset cross-enhancement module to obtain the corresponding detection results.

[0085] Furthermore, the shared representation module specifically includes:

[0086] The standard planar image of the hip joint ultrasound is input into four parallel network branches of a preset shared representation module to obtain feature maps at four scales. Based on the feature maps at the four scales, size data is obtained by bilinear interpolation upsampling and then stitched together to obtain a multi-scale feature map corresponding to the standard planar image of the hip joint ultrasound. The four parallel network branches are used to reduce the size of the feature map layer by layer while increasing the number of channels. The shared representation module is based on a deep neural network model used for human pose estimation and image segmentation.

[0087] Furthermore, the multi-task collaboration module specifically includes:

[0088] Based on a preset multi-task collaborative module, anatomical structure segmentation processing is performed on the multi-scale feature map corresponding to the standard planar ultrasound image of the hip joint to obtain the segmented structure map corresponding to the standard planar ultrasound image of the hip joint; and, using heatmap regression, key point recognition processing is performed on the multi-scale feature map corresponding to the standard planar ultrasound image of the hip joint to obtain the skeletal key points corresponding to the standard planar ultrasound image of the hip joint; and, line detection processing is performed on the multi-scale feature map corresponding to the standard planar ultrasound image of the hip joint to obtain the Graf line corresponding to the standard planar ultrasound image of the hip joint.

[0089] Furthermore, the cross-enhancement module specifically includes:

[0090] Based on the preset cross-enhancement module, feature enhancement processing is performed on the spatial dependency relationship between the segmentation structure map and the skeletal key points, and feature enhancement processing is performed on the spatial dependency relationship between the skeletal key points and the Graf line to obtain the corresponding detection results.

[0091] Furthermore, the preset cross-enhancement module performs feature enhancement processing on the spatial dependency relationship between the segmentation structure map and the skeletal keypoints, and performs feature enhancement processing on the spatial dependency relationship between the skeletal keypoints and the Graf lines to obtain corresponding detection results, specifically including:

[0092] Based on the preset cross-enhancement module, feature enhancement processing is performed on the spatial dependency relationship between the segmentation structure map and the skeletal key points, and feature enhancement processing is performed on the spatial dependency relationship between the skeletal key points and the Graf line to obtain the hip joint feature map after feature enhancement processing.

[0093] The corresponding detection results are determined based on the angle formed between the Graf lines in the hip joint feature map.

[0094] Furthermore, the preset cross-enhancement module performs feature enhancement processing on the spatial dependency relationship between the segmented structure map and the skeletal key points, and performs feature enhancement processing on the spatial dependency relationship between the skeletal key points and the Graf line to obtain a hip joint feature map after feature enhancement, specifically including:

[0095] Based on a preset cross-enhancement module, feature enhancement processing is performed on the spatial dependency relationship between the segmentation structure map and the skeletal key points, and feature enhancement processing is performed on the spatial dependency relationship between the skeletal key points and the Graf line to obtain the feature-enhanced segmentation structure map, skeletal key points, and Graf line; the feature-enhanced segmentation structure map, skeletal key points, and Graf line are then superimposed to obtain the feature-enhanced hip joint feature map.

[0096] Furthermore, determining the corresponding detection result based on the angle formed between the Graf lines in the hip joint feature map specifically includes:

[0097] The first and second angles, which represent hip joint features, formed between Graf lines in the hip joint feature map, are compared with preset angle thresholds to obtain angle comparison results. Based on the angle comparison results, the corresponding detection results are determined.

[0098] The hip joint assisted detection device based on multi-task learning described in this invention acquires a standard planar ultrasound image of the hip joint to be analyzed; extracts hip joint features from the standard planar ultrasound image based on a preset shared representation module to obtain a multi-scale feature map corresponding to the standard planar ultrasound image; performs multi-task collaborative processing on the multi-scale feature map corresponding to the standard planar ultrasound image based on a preset multi-task collaboration module to obtain a segmentation structure map, skeletal key points, and Graf lines corresponding to the standard planar ultrasound image; and performs multi-task cross-enhancement processing on the segmentation structure map, skeletal key points, and Graf lines based on a preset cross-enhancement module to obtain the corresponding detection results. This device can effectively improve the automation, detection speed, and robustness of assisted detection of hip joint images.

[0099] Corresponding to the multi-task learning-based hip joint assisted detection method provided above, this invention also provides an electronic device. Since the embodiment of this electronic device is similar to the method embodiment described above, it is described simply. For relevant details, please refer to the description in the method embodiment section above. The electronic device described below is merely illustrative. Figure 8The diagram shows a physical structure of an electronic device disclosed in an embodiment of the present invention. The electronic device may include a processor 801, a memory 802, and a communication bus 803. The processor 801 and the memory 802 communicate with each other via the communication bus 803 and communicate with external systems via a communication interface 804. The processor 801 can call logical instructions in the memory 802 to execute a hip joint assisted detection method based on multi-task learning. This method includes: acquiring a standard planar ultrasound image of the hip joint to be analyzed; extracting hip joint features from the standard planar ultrasound image based on a preset shared representation module to obtain a multi-scale feature map corresponding to the standard planar ultrasound image; performing multi-task collaborative processing on the multi-scale feature map corresponding to the standard planar ultrasound image based on a preset multi-task collaborative module to obtain a segmentation structure map, skeletal key points, and Graf lines corresponding to the standard planar ultrasound image; and performing multi-task cross-enhancement processing on the segmentation structure map, the skeletal key points, and the Graf lines based on a preset cross-enhancement module to obtain corresponding detection results.

[0100] Furthermore, the logical instructions in the aforementioned memory 802 can be implemented as software functional modules and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as memory chips, USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0101] On the other hand, embodiments of the present invention also provide a computer program product, the computer program product including a computer program stored on a processor-readable storage medium, the computer program including program instructions, and when the program instructions are executed by a computer, the computer is able to execute the hip joint assisted detection method based on multi-task learning provided in the above-described method embodiments. The method includes: acquiring a standard planar ultrasound image of the hip joint to be analyzed; extracting hip joint features from the standard planar ultrasound image of the hip joint based on a preset shared representation module to obtain a multi-scale feature map corresponding to the standard planar ultrasound image of the hip joint; performing multi-task collaborative processing on the multi-scale feature map corresponding to the standard planar ultrasound image of the hip joint based on a preset multi-task collaborative module to obtain a segmentation structure map, skeletal key points, and Graf lines corresponding to the standard planar ultrasound image of the hip joint; and performing multi-task cross-enhancement processing on the segmentation structure map, the skeletal key points, and the Graf lines based on a preset cross-enhancement module to obtain corresponding detection results.

[0102] In another aspect, embodiments of the present invention also provide a processor-readable storage medium storing a computer program, which, when executed by a processor, implements the hip joint assisted detection method based on multi-task learning provided in the above embodiments. The method includes: acquiring a standard planar ultrasound image of the hip joint to be analyzed; extracting hip joint features from the standard planar ultrasound image based on a preset shared representation module to obtain a multi-scale feature map corresponding to the standard planar ultrasound image; performing multi-task collaborative processing on the multi-scale feature map corresponding to the standard planar ultrasound image based on a preset multi-task collaborative module to obtain a segmentation structure map, skeletal key points, and Graf lines corresponding to the standard planar ultrasound image; and performing multi-task cross-enhancement processing on the segmentation structure map, the skeletal key points, and the Graf lines based on a preset cross-enhancement module to obtain corresponding detection results.

[0103] The processor-readable storage medium can be any available medium or data storage device that the processor can access, including but not limited to magnetic memory (e.g., floppy disk, hard disk, magnetic tape, magneto-optical disk (MO)), optical memory (e.g., CD, DVD, BD, HVD), and semiconductor memory (e.g., ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)).

[0104] The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0105] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A hip joint assisted detection method based on multi-task learning, characterized in that, include: Obtain a standard planar ultrasound image of the hip joint to be analyzed; Based on the preset shared representation module, hip joint features are extracted from the standard planar image of hip joint ultrasound to obtain a multi-scale feature map corresponding to the standard planar image of hip joint ultrasound. Based on a preset multi-task collaborative module, the multi-scale feature map corresponding to the standard planar image of hip joint ultrasound is processed in a multi-task collaborative manner to obtain the segmentation structure map, skeletal key points and Graf lines corresponding to the standard planar image of hip joint ultrasound. The segmentation structure map, skeletal key points, and Graf lines are subjected to multi-task cross-enhancement processing based on a preset cross-enhancement module to obtain corresponding detection results; the multi-task collaborative processing of the multi-scale feature map corresponding to the standard planar image of the hip joint ultrasound is performed based on a preset multi-task collaborative module to obtain the segmentation structure map, skeletal key points, and Graf lines corresponding to the standard planar image of the hip joint ultrasound, specifically including: The hip joint ultrasound standard planar image is processed by performing anatomical structure segmentation on the multi-scale feature map based on a preset multi-task collaborative module to obtain the segmented structure map corresponding to the hip joint ultrasound standard planar image; and, key point recognition processing is performed on the multi-scale feature map corresponding to the hip joint ultrasound standard planar image using heatmap regression to obtain the skeletal key points corresponding to the hip joint ultrasound standard planar image; and, line detection processing is performed on the multi-scale feature map corresponding to the hip joint ultrasound standard planar image to obtain the Graf line corresponding to the hip joint ultrasound standard planar image; the hip joint feature extraction based on a preset shared representation module to obtain the multi-scale feature map corresponding to the hip joint ultrasound standard planar image specifically includes: The standard planar image of the hip joint ultrasound is input into four parallel network branches of a preset shared representation module to obtain feature maps at four scales. Based on these four scale feature maps, size data is obtained through bilinear interpolation upsampling and then stitched together to obtain a multi-scale feature map corresponding to the standard planar image of the hip joint ultrasound. The four parallel network branches are used to progressively reduce the size of the feature maps while increasing the number of channels. The shared representation module is based on a deep neural network model used for human pose estimation and image segmentation. A preset cross-enhancement module performs multi-task cross-enhancement processing on the segmentation structure map, the skeletal key points, and the Graf lines to obtain corresponding detection results, specifically including: Based on the preset cross-enhancement module, feature enhancement processing is performed on the spatial dependency relationship between the segmentation structure map and the skeletal key points, and feature enhancement processing is performed on the spatial dependency relationship between the skeletal key points and the Graf line to obtain the corresponding detection results.

2. The hip joint assisted detection method based on multi-task learning according to claim 1, characterized in that, The preset cross-enhancement module performs feature enhancement processing on the spatial dependency relationship between the segmentation structure map and the skeletal keypoints, and performs feature enhancement processing on the spatial dependency relationship between the skeletal keypoints and the Graf lines to obtain corresponding detection results, specifically including: Based on the preset cross-enhancement module, feature enhancement processing is performed on the spatial dependency relationship between the segmentation structure map and the skeletal key points, and feature enhancement processing is performed on the spatial dependency relationship between the skeletal key points and the Graf line to obtain the hip joint feature map after feature enhancement processing. The corresponding detection results are determined based on the angle formed between the Graf lines in the hip joint feature map.

3. The hip joint assisted detection method based on multi-task learning according to claim 2, characterized in that, The preset cross-enhancement module performs feature enhancement processing on the spatial dependency relationship between the segmented structure map and the skeletal key points, and performs feature enhancement processing on the spatial dependency relationship between the skeletal key points and the Graf lines to obtain a hip joint feature map after feature enhancement, specifically including: Based on a preset cross-enhancement module, feature enhancement processing is performed on the spatial dependency relationship between the segmentation structure map and the skeletal key points, and feature enhancement processing is performed on the spatial dependency relationship between the skeletal key points and the Graf line to obtain the feature-enhanced segmentation structure map, skeletal key points, and Graf line; the feature-enhanced segmentation structure map, skeletal key points, and Graf line are then superimposed to obtain the feature-enhanced hip joint feature map.

4. The hip joint assisted detection method based on multi-task learning according to claim 2, characterized in that, The determination of the corresponding detection result based on the angle formed between the Graf lines in the hip joint feature map specifically includes: The first and second angles, which represent hip joint features, formed between Graf lines in the hip joint feature map, are compared with preset angle thresholds to obtain angle comparison results. Based on the angle comparison results, the corresponding detection results are determined.

5. A hip joint assistive detection device based on multi-task learning, characterized in that, include: The image acquisition unit is used to acquire standard planar ultrasound images of the hip joint to be analyzed. A shared representation unit is used to extract hip joint features from the standard planar image of hip joint ultrasound based on a preset shared representation module, and obtain a multi-scale feature map corresponding to the standard planar image of hip joint ultrasound. The multi-task collaborative unit is used to perform multi-task collaborative processing on the multi-scale feature map corresponding to the standard planar image of hip joint ultrasound based on a preset multi-task collaborative module, so as to obtain the segmentation structure map, skeletal key points and Graf lines corresponding to the standard planar image of hip joint ultrasound. The cross-enhancement unit is used to perform multi-task cross-enhancement processing on the segmentation structure map, the skeletal key points and the Graf lines based on a preset cross-enhancement module to obtain the corresponding detection results; The multi-task collaborative unit specifically includes: Based on a preset multi-task collaborative module, anatomical structure segmentation processing is performed on the multi-scale feature map corresponding to the standard planar ultrasound image of the hip joint to obtain the segmented structure map corresponding to the standard planar ultrasound image of the hip joint; and, using heatmap regression, key point recognition processing is performed on the multi-scale feature map corresponding to the standard planar ultrasound image of the hip joint to obtain the skeletal key points corresponding to the standard planar ultrasound image of the hip joint; and, line detection processing is performed on the multi-scale feature map corresponding to the standard planar ultrasound image of the hip joint to obtain the Graf line corresponding to the standard planar ultrasound image of the hip joint. The shared representation unit specifically includes: The standard planar image of the hip joint ultrasound is input into four parallel network branches of a preset shared representation module to obtain feature maps at four scales. Based on these four scale feature maps, size data is obtained through bilinear interpolation upsampling and then stitched together to obtain a multi-scale feature map corresponding to the standard planar image of the hip joint ultrasound. The four parallel network branches are used to progressively reduce the size of the feature maps while increasing the number of channels. The shared representation module is based on a deep neural network model used for human pose estimation and image segmentation. The cross-enhancement unit specifically includes: Based on the preset cross-enhancement module, feature enhancement processing is performed on the spatial dependency relationship between the segmentation structure map and the skeletal key points, and feature enhancement processing is performed on the spatial dependency relationship between the skeletal key points and the Graf line to obtain the corresponding detection results.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the hip joint assisted detection method based on multi-task learning as described in any one of claims 1 to 4.

7. A processor-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the hip joint assisted detection method based on multi-task learning as described in any one of claims 1 to 4.

Images