Hip joint body surface recognition method and system based on double-model fusion

Abstract

The invention provides a hip joint body surface recognition method based on double-model fusion, and the method specifically comprises the following steps: S1, constructing a diversified image data set containing the key features of the body surface of a newborn for a hip joint ultrasonic examination scene; s2, a network structure of a YOLOv8-Tiny target detection model and a network structure of an OpenPose human body posture estimation model are improved; s3, respectively training the improved YOLOv8-Tiny target detection model and the improved OpenPose human body posture estimation model by utilizing the marked training set, and obtaining an optimal training model parameter; and through double-model joint reasoning, feature extraction and position prediction are performed on an input image, output results of the two models are fused in combination with geometrical relationship constraint and a confidence coefficient weighting strategy, and finally, high-precision recognition coordinates of the body surface of the hip joint of the newborn are output. According to the method, a more accurate and reliable identification result can be provided in real time under the condition of relatively low computing resources, and the detection efficiency and precision are remarkably improved.

Description

Technical Field

[0001] The present invention relates to computer image processing, which is applied to ultrasound scanning-assisted positioning of developmental dysplasia of the hip (DDH) in newborns, and in particular to a hip joint surface recognition method and system combining deep learning with human feature point recognition. Background Art

[0002] Developmental dysplasia of the hip (DDH) is one of the most common skeletal disorders in children, and ultrasound examinations have been shown to effectively reduce its incidence. Currently, ultrasound examinations of the hip joint rely heavily on the physician's experience, and position identification often relies on manual labor. This is not only easily limited by the physician's experience, but also has low detection efficiency. With the continuous advancement of artificial intelligence technology, deep learning-based image recognition technology has been widely used in the medical field, performing particularly well in image recognition.

[0003] Existing hip joint recognition methods can generally be categorized into the following categories based on the different recognition features: 1) Image feature-based recognition methods. These algorithms perform well in standard newborn body positions, accurately identifying target areas and are particularly suitable for facial or common posture recognition. However, as newborn body positions change, especially in complex postures, the accuracy of traditional skin feature recognition methods decreases significantly, errors increase, and recognition performance is significantly reduced. 2) Human skeleton recognition and pose estimation algorithms (such as OpenPose). These algorithms can accurately identify human skeletal features and perform overall pose estimation, achieving excellent results in adult pose recognition. However, because newborn skeletons are not fully developed, and posture and skeletal features are relatively soft, flexible, and highly variable, existing pose estimation algorithms still face significant challenges in newborn pose recognition. Recognition accuracy in different body positions or complex postures, in particular, needs to be improved urgently. 3) Recognition methods based on multi-model fusion. Some studies have proposed multi-model fusion image recognition methods, such as the patent "A Dual-Model-Based Infant Sleeping Position Recognition Method" (CN119580340). This method combines a modified version of the YOLOv5 and GoogleNet network models for infant sleeping position recognition. Although this method improves accuracy, the simultaneous execution of two deep learning models results in a high computational load and low frame rate, making real-time recognition difficult. Summary of the Invention

[0004] To solve the above problems, the present invention discloses a hip joint surface recognition method and system that combines deep learning with human feature point recognition. By identifying predefined feature points, it can provide more accurate and reliable recognition results in real time under conditions of lower computing resources, significantly improving detection efficiency and accuracy.

[0005] The present invention proposes a hip joint surface recognition method that combines deep learning with human feature point recognition, which specifically includes the following steps:

[0006] S1: For hip ultrasound examination scenarios, a diverse image dataset containing key surface features of newborns was constructed. After data augmentation and preprocessing, the diverse image dataset was further divided into two groups and annotated separately: for the dataset used by the YOLOv8-Tiny object detection model, the detection boxes of four key areas were annotated, namely the ears, heels, front thigh contours, and back thigh contours; for the dataset used by the OpenPose human pose estimation model, the key points of the left and right hips were annotated;

[0007] S2: Improve the network structure of the YOLOv8-Tiny object detection model and the OpenPose human pose estimation model. Specifically, the Spatial Attention Module (SAM) is inserted between the Backbone and Neck layers of the YOLOv8-Tiny object detection model to enhance the model's ability to distinguish skin areas from complex backgrounds. The CIoU Loss function is used instead of the original loss function to optimize the bounding box regression accuracy for small object detection tasks. The original ResNet backbone network of the OpenPose human pose estimation model is replaced with a lightweight MobileNetV3-Smal l network, and the Coordinate Attention module is introduced in its convolutional layer to improve the spatial perception ability of key point positioning and reduce the model's computational complexity.

[0008] S3: Use the labeled training set to train the improved YOLOv8-Tiny object detection model and the OpenPose human pose estimation model to obtain the optimal training model parameters. Through dual-model joint reasoning, feature extraction and position prediction are performed on the input image. The output results of the two models are fused by combining geometric relationship constraints and confidence weighting strategies to finally output high-precision identification coordinates of the neonatal hip joint surface.

[0009] Furthermore, the dataset annotation method of S1 specifically includes:

[0010] S11: Copy the newborn image dataset that has undergone data augmentation and preprocessing in S2 into two copies, one for labeling the YoloV8-Tiny object detection model and the other for labeling the OpenPose human pose estimation model;

[0011] S12: Label specific detection boxes in the dataset used for training the YoloV8-Tiny object detection model. Specifically, label four detection boxes: the ear, heel, front thigh outline, and back thigh outline of the newborn.

[0012] S13: In the dataset used for the OpenPose human pose estimation model, the focus of annotation is to clearly mark the key points of the left and right hips of newborns;

[0013] S14: Divide the labeled dataset and its labels into training and test sets according to a predetermined ratio for subsequent model training and evaluation.

[0014] Furthermore, in step S2, the network structures of the YoloV8-Tiny target detection model and the OpenPose human pose estimation model are improved, specifically including:

[0015] S21: Add a spatial attention module (SAM) between the C2f module in the Backbone layer and the Concat module in the Neck layer of the YoloV8-Tiny object detection model to enhance the model's ability to distinguish between skin areas and background, thereby improving object detection accuracy.

[0016] S22: The Clou Loss loss function is used to replace the original bounding box regression loss function of the YoloV8-Tiny object detection model to optimize the bounding box positioning performance in small object detection tasks and improve the recognition accuracy of key parts;

[0017] S23: Replace the original ResNet backbone network of the OpenPose human pose estimation model with the MobileNetV3-Small network, significantly reducing the model's computational complexity and parameter size while ensuring pose estimation performance, meeting the system's real-time processing requirements.

[0018] S24: Introducing the CoordinateAttention module into the pose estimation convolutional layer of the OpenPose human pose estimation model to replace the traditional convolutional layer, further improving the spatial positioning accuracy and robustness of human key points.

[0019] Furthermore, the S3 method for identifying the surface of the neonatal hip joint based on the fusion of dual-model recognition results specifically includes:

[0020] S31: Use the YOLOv8-Tiny object detection model to identify four key parts in the newborn image, namely the ear (a), heel (b), front thigh (c), and back thigh (d), and output the corresponding confidence value C a ,C b ,C c ,C d; Connect the center points of the ear and heel to form a line segment ab, and connect the midpoints of the front and back thigh contours to form a line segment cd; calculate the intersection point e(u e ,v e ), as the initial positioning point on the skin surface of the hip joint;

[0021] The geometric mean method is further used to calculate the confidence of line segments ab and cd:

[0022]

[0023] Then calculate the comprehensive confidence C of the intersection e e

[0024] C e =1-(1-C ab )(1-C cd )

[0025] S32: The OpenPose human pose estimation model identifies the newborn's left hip joint (f) at S301 and the right hip joint (g) at S302. The Euclidean distance between the two key points and the intersection point e is calculated:

[0026]

[0027] S34: Use the exponentially weighted moving average method to smooth the final recognition results of the three most recent consecutive frames, the current frame t, the previous frame t-1, and the previous two frames t-2:

[0028] p smooth =0.6·p final (t) +0.3·p final (t-1) +0.1·p final (t-2)

[0029] Among them, p final (t) Indicates the final recognition coordinates of the current frame, p final (t-1) and p final (t-2) Represent the final recognition coordinates of the previous frame and the previous two frames respectively.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] 1. This paper combines the YOLOv8-Tiny object detection model with the OpenPose human pose estimation model, performing a weighted fusion of their outputs to achieve high-precision positioning of the hip joint scanning starting point. Unlike existing approaches that rely solely on object detection or pose estimation, this dual-model fusion approach simultaneously accounts for both skin region recognition and skeletal feature estimation, significantly improving recognition accuracy in complex body positions.

[0032] 2. Considering the limitations of computing resources, the network structure design ensures both high accuracy and real-time performance. In particular, the adoption of the MobileNetV3-Small network and the Spatial Attention Module (SAM) enables efficient inference with low computing resources, making it suitable for real-time ultrasound scanning in actual clinical environments.

[0033] 3. This paper uses a confidence-weighted fusion method to integrate the recognition results of the YOLOv8-Tiny object detection model and the OpenPose human pose estimation model to further improve the accuracy of the final recognition point. Combining the confidence values ​​of both ensures the high reliability of the hip joint scanning starting point. The final recognition result is smoothed using the exponentially weighted moving average (EWMA) method on the coordinates of the last six frames, effectively filtering out short-term noise and improving system stability.

[0034] 4. This invention can achieve high-precision hip joint origin recognition in complex neonatal postures. Traditional image recognition algorithms typically perform well only in standard postures, with recognition accuracy significantly declining when faced with posture changes or occlusions. Through dual-model fusion and optimized network design, this invention effectively overcomes this problem, making it particularly suitable for real-time dynamic scanning and various postures in clinical scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 , a flowchart of the hip recognition algorithm of the present invention;

[0036] Figure 2 , Yolov8-Tiny target detection model data annotation diagram;

[0037] Figure 3 , OpenPose human pose estimation model data annotation diagram;

[0038] Figure 4 , improved YoloV8-Tiny target detection model network structure diagram;

[0039] Figure 5 , schematic diagram of hip joint surface recognition results after fusion of dual model recognition results;

[0040] Figure 6, Schematic diagram of hip joint surface recognition system based on dual model fusion.

[0041] Figure 7 , comprehensive performance comparison chart. DETAILED DESCRIPTION

[0042] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are intended only to illustrate the present invention and are not intended to limit the scope of the present invention. It should be noted that the terms "front," "rear," "left," "right," "up," and "down" used in the following description refer to directions in the accompanying drawings, and the terms "inward" and "outward" refer to directions toward or away from the geometric center of a particular component, respectively.

[0043] The present invention proposes a hip joint surface recognition method that combines deep learning with human feature point recognition. Figure 1 As shown, including:

[0044] S1: For hip ultrasound examination scenarios, a diverse image dataset containing key surface features of newborns was constructed. After data augmentation and preprocessing, the diverse image dataset was further divided into two groups and annotated separately: for the dataset used by the YOLOv8-Tiny object detection model, the detection boxes of four key areas were annotated, namely the ears, heels, front thigh contours, and back thigh contours; for the dataset used by the OpenPose human pose estimation model, the key points of the left and right hips were annotated;

[0045] S2: Improve the network structure of the YOLOv8-Tiny object detection model and the OpenPose human pose estimation model. Specifically, the Spatial Attention Module (SAM) is inserted between the Backbone and Neck layers of the YOLOv8-Tiny object detection model to enhance the model's ability to distinguish skin areas from complex backgrounds. The CIoU Loss function is used instead of the original loss function to optimize the bounding box regression accuracy for small object detection tasks. The original ResNet backbone network of the OpenPose human pose estimation model is replaced with a lightweight MobileNetV3-Smal l network, and the Coordinate Attention module is introduced in its convolutional layer to improve the spatial perception ability of key point positioning and reduce the model's computational complexity.

[0046] S3: Use the labeled training set to train the improved YOLOv8-Tiny object detection model and the OpenPose human pose estimation model to obtain the optimal training model parameters. Through dual-model joint reasoning, feature extraction and position prediction are performed on the input image. The output results of the two models are fused by combining geometric relationship constraints and confidence weighting strategies to finally output high-precision identification coordinates of the neonatal hip joint surface.

[0047] The specific contents of this embodiment include the following:

[0048] Step 1: Obtain a diverse dataset of body surface images in a neonatal hip ultrasound scenario

[0049] We collected image data from neonates undergoing hip ultrasound examinations and used this data to construct a diverse image dataset featuring the surface characteristics of the neonatal hip joint. This diverse image dataset should include full-body lateral images of the neonate, clearly showing the ears, heels, and front and back of the thighs. Furthermore, the diverse image dataset should include images of the neonate in different postures and from different viewing angles during ultrasound examinations to ensure sufficient diversity and generalization for model training.

[0050] Step 2: Image data enhancement and preprocessing

[0051] The data enhancement methods include:

[0052] (1) Rotate and flip the image horizontally / vertically to simulate the image changes of newborns in different body positions. These transformations help enhance the model's adaptability to body position changes.

[0053] (2) Adjust the brightness, contrast, and saturation of the image to simulate the imaging effects of the newborn at different distances from the camera and under different lighting conditions. These color changes ensure that the model can adapt to the image characteristics of different environments.

[0054] (3) Rectangular occlusion blocks are introduced into some diverse image datasets to simulate occlusion scenarios that may occur in clinical practice (for example, diapers, clothing, and other obstructions). These occlusion blocks simulate situations that may be encountered in actual operations and enhance the model's recognition ability for partially occluded images.

[0055] (4) Introducing Gaussian noise into the image to simulate the blur and noise that may appear in the image. This operation can effectively enhance the model's adaptability when facing blurry or low-quality images.

[0056] Preprocessing specifically includes:

[0057] (1) All images are resized to a uniform size of 640 × 480 pixels. This standardization process minimizes the impact of different image sizes on model training and also helps speed up the training process.

[0058] (2) Scale the pixel values ​​of the image to the range of [0, 1] to ensure that all input data are processed at the same scale, which helps improve the training efficiency and convergence speed of the model.

[0059] (3) Use the Gaussian filter algorithm to perform denoising on the image to remove high-frequency noise in the image. Gaussian filtering smoothes the image, retains important image features, and reduces the interference of noise on subsequent processing and recognition, thereby improving the overall quality of the image.

[0060] Step 3: Dual-model dataset annotation

[0061] (1) YOLO model dataset annotation:

[0062] In the dataset, four key detection areas of newborns need to be labeled, such as Figure 2 As shown, they are: S201 ear, S202 heel, S203 front thigh outline, and S204 back thigh outline. Detection targets are determined by manually annotating bounding boxes for these areas. Accurate annotation is crucial to the training of the YOLO model. The intersection of the ear and heel lines, as well as the midpoints of the thigh outlines, is determined. This intersection is the surface location of the newborn's hip and serves as one of the key detection points for the YOLO model.

[0063] (2) OpenPose human pose estimation model dataset annotation:

[0064] In the dataset annotation of the OpenPose human pose estimation model, the focus is on clearly marking the key points of the newborn's S301 left hip and S302 right hip, such as Figure 3 These joints are crucial for accurately estimating the newborn's body position and hip location. For each image, the key points of the left and right hips were annotated using annotation tools to ensure consistency and accuracy.

[0065] (3) Dataset integration and division

[0066] After annotating the YoloV8-Tiny object detection model and OpenPose human pose estimation model datasets, the two datasets should be merged. This merged dataset will contain object detection regions (ears, heels, front and back thigh contours, etc.) as well as pose estimation keypoints (left and right hips). This merged dataset will be split into training and validation sets according to a predetermined ratio (8:2). Specifically, 80% of the data will be used for training and 20% for validating model performance.

[0067] Step 4: Modify the structure of YoloV8-Tiny target detection model and OpenPose human pose estimation model

[0068] (1) Inserting the spatial attention module into the Neck layer of the YoloV8-Tiny target detection model

[0069] In order to enhance the network's ability to distinguish between skin areas and backgrounds, the present invention inserts the S401 spatial attention module (SAM) into the Neck layer of the YOLOv8-Tiny target detection model. Figure 4 This module dynamically allocates more attention to important areas (such as skin) in the spatial dimension, thereby improving target detection accuracy. This is particularly true against complex backgrounds, enhancing the ability to locate surface features of neonatal hip joints. Through this adaptive attention mechanism, the network can better focus on key areas in and around the joints, improving detection accuracy.

[0070] (2) Using Clou Loss function

[0071] To further improve the accuracy of bounding box regression, this paper uses the Clou Loss loss function in the YOLOv8-Tiny object detection model, replacing the original loss function. Clou Loss is particularly suitable for small object detection, enabling more accurate bounding box regression and reducing the positioning error of small objects (such as the hip joint and surrounding features of a newborn). By strengthening the focus on bounding box accuracy, this loss function optimizes the positioning accuracy in the regression task, especially improving the detection ability of small objects.

[0072] (3) The backbone network of the OpenPose human pose estimation model is replaced with the MobileNetV3-Small network

[0073] To reduce computational complexity and meet real-time requirements, this paper replaces the backbone network of the OpenPose human pose estimation model with the MobileNetV3-Small network. Compared with traditional ResNet backbone networks or other heavier convolutional neural networks, the MobileNetV3-Small network has significantly lower computational complexity and smaller model size. This allows the model to significantly reduce computational overhead while maintaining high performance, thereby achieving real-time pose estimation, which is particularly suitable for real-time applications such as neonatal hip ultrasound scanning.

[0074] (4) Introducing the Coordinate Attention module

[0075] To improve pose estimation accuracy, especially when dealing with complex backgrounds and small features, this paper introduces a Coordinate Attention module into the convolutional layers of the OpenPose human pose estimation model. Traditional convolutional layers fail to effectively capture spatial relationships when extracting features. By explicitly modeling spatial positions, the Coordinate Attention module enhances the network's focus on local features, particularly for accurately locating key points in the hip joint of newborns. The introduction of this module significantly improves pose estimation performance, particularly in low-resolution and blurred images.

[0076] Step 5: YoloV8-Tiny target detection model and OpenPose human pose estimation model network training

[0077] The labeled training sets of the YoloV8-Tiny target detection model and the OpenPose human pose estimation model are input into the corresponding networks for training.

[0078] During the training process, back propagation is used to adjust the network parameters according to the calculation results of the loss function. For the YoloV8-Tiny target detection model, the main adjustments are to the parameters such as the weights and bias of the convolutional layer to ensure the accuracy of the detected target; for the OpenPose human posture estimation model, the weights in the posture estimation network are optimized to improve the recognition ability of the key points of the newborn hip joint. The training is carried out through multiple rounds of iterations. After each round of training, the network parameters will be adjusted according to the feedback of the loss function to minimize the prediction error and gradually improve the recognition accuracy. The training results of the YoloV8-Tiny target detection model are as follows: Figure 4As shown in the figure, the optimized network architecture and loss function can provide high accuracy in the neonatal hip joint object detection task. During training, the model was continuously optimized, ultimately achieving good recognition performance on both the training and validation sets. The YOLO model demonstrated strong robustness in small object detection and complex backgrounds. By introducing the MobileNetV3-Small network and the Coordinate Attention module, the OpenPose human pose estimation model can effectively improve the accuracy of pose estimation of key points in the neonatal hip joint task, especially in low-resolution and blurred images.

[0079] Step 6: Hip joint surface recognition method based on dual model fusion

[0080] During the recognition process, we first select the optimal weight file obtained during training. Then, based on the recognition results of the YOLOv8-Tiny object detection model and the OpenPose human pose estimation model, we perform multi-step calculation and weighted fusion to ultimately obtain the precise position of the hip joint. This position is then smoothed to ensure good stability and accuracy in real-time applications.

[0081] YOLOv8-Tiny target detection model recognition result processing

[0082] Through the YOLOv8-Tiny target detection model, such as Figure 5 , four feature points in the newborn image are identified, namely S501 ear (a), S502 heel (b), S503 front thigh (c) and S504 back thigh (d). These four feature points correspond to confidence values ​​c a ,c b ,c c ,c d Connect the center points of the ear and heel, record it as line segment ab. Connect the midpoints of the outline of the thigh on both sides, record it as line segment cd.

[0083] The intersection of lines ab and cd at S505 is e(u e ,v e ) is the skin positioning point of the hip joint.

[0084] The confidence level of the ab line and the cd line is calculated using the geometric mean:

[0085] Calculate the intersection point e(u e ,v e ), as the initial positioning point on the skin surface of the hip joint;

[0086] The geometric mean method is further used to calculate the confidence of line segments ab and cd:

[0087]

[0088] Then calculate the comprehensive confidence C of the intersection e e

[0089] C e =1-(1-C ab )(1-C cd )

[0090] S32: The OpenPose human pose estimation model identifies the newborn's left hip joint (f) at S506 and the right hip joint (g) at S507; the Euclidean distance between these two key points and the intersection point e is calculated:

[0091]

[0092] S34: Use the exponentially weighted moving average method to smooth the final recognition results of the three most recent consecutive frames, the current frame t, the previous frame t-1, and the previous two frames t-2:

[0093] p smooth =0.6·p final (t) +0.3·p final (t-1) +0.1·p final (t-2)

[0094] Among them, p final (t) Indicates the final recognition coordinates of the current frame, p final (t-1) and p final (t-2) Represent the final recognition coordinates of the previous frame and the previous two frames respectively.

[0095] This embodiment provides a hip joint surface recognition system based on dual model fusion. Figure 6As shown in the figure, the system includes an S602 camera image acquisition module for acquiring image data in the neonatal hip joint ultrasound examination scenario; an S604 image data enhancement and preprocessing module for performing enhancement processing such as rotation, translation, color change, etc. on the acquired image data, and performing standardization and noise reduction; an S606 image data set annotation and division module for annotating the image data and dividing it into a training set and a validation set according to a certain ratio; an S608 YoloV8-Tiny target detection model and OpenPose human pose estimation model improvement module for structurally improving the YoloV8-Tiny target detection model and the OpenPose human pose estimation model to improve recognition accuracy and computational efficiency; an S610 YoloV8-Tiny target detection model and OpenPose human pose estimation model training module for respectively training the YoloV8-Tiny target detection model and the OpenPose human pose estimation model to obtain the optimal model weight; and an S612 YoloV8-Tiny target detection model and OpenPose human pose estimation model recognition result fusion module for performing weighted fusion based on the recognition results of the two models to ultimately obtain accurate positioning of the hip joint.

[0096] like Figure 7 The following is a comprehensive performance comparison chart: mAP@0.5 means COCO The average precision of the dataset, with a threshold of 0.5 (applicable to object detection models). FPS is the frame rate, indicating the number of frames processed per second. The CPU in parentheses indicates the hardware platform. The number of parameters indicates the size of the model, with M as the unit. The inference time is the time it takes the model to infer one frame of data on the CPU platform, in milliseconds. "-" indicates that this metric is not available.

[0097] The fusion method proposed in this paper significantly surpasses the YOLOv8-Tiny object detection model in mean average precision (mAP). On the CPU platform, its frame rate (FPS) is higher than that of the OpenPose human pose estimation model. Although lower than that of the YOLOv8-Tiny object detection model, the method achieves efficient inference performance with an inference time shorter than that of the OpenPose human pose estimation model and close to that of the YOLOv8-Tiny object detection model, thanks to its balanced parameter size between the two (higher than that of the YOLOv8-Tiny object detection model and lower than that of the OpenPose human pose estimation model).

Claims

1. A hip joint surface recognition method based on dual-model fusion, characterized by: The specific steps include: S1: For the hip joint ultrasound examination scenario, a diverse image dataset containing key features of the neonatal body surface is constructed; after data enhancement and preprocessing, the diverse image dataset is divided into two groups and annotated separately: S2: Improve the network structure of the YOLOv8-Tiny object detection model and the OpenPose human pose estimation model; insert a spatial attention module between the Backbone layer and the Neck layer of the YOLOv8-Tiny object detection model; replace the original loss function with the CIoULoss loss function to optimize the bounding box regression accuracy for small object detection tasks; replace the original ResNet backbone network of the OpenPose human pose estimation model with a lightweight MobileNetV3-Small network, and introduce the Coordinate Attention module in its convolutional layers; S3: Using the labeled training set, the improved YOLOv8-Tiny object detection model and the OpenPose human pose estimation model are trained to obtain the optimal model parameters. A newborn hip joint surface recognition method based on the fusion of the dual-model recognition results is finally output with high-precision recognition coordinates of the newborn hip joint surface.

2. The hip joint surface recognition method based on dual-model fusion according to claim 1, characterized in that: The S1 dataset annotation method specifically includes: S11: Copy the diversified image dataset that has undergone data augmentation and preprocessing in S1 into two copies, one for labeling the YoloV8-Tiny object detection model and the other for labeling the OpenPose human pose estimation model; S12: In a diverse image dataset used for training the YoloV8-Tiny object detection model, label the detection boxes; specifically, label the four detection boxes of the newborn's ear, heel, front thigh outline, and back thigh outline; S13: In the diverse image dataset used for the OpenPose human pose estimation model, the focus of annotation is to clearly mark the key points of the left and right hips of newborns; S14: Divide the annotated diverse image dataset and its labels into training and test sets according to a predetermined ratio for subsequent model training and evaluation.

3. The hip joint surface recognition method based on dual-model fusion according to claim 1, characterized in that: The S2 specifically includes: S21: Add a spatial attention module between the C2f module in the Backbone layer and the Concat module in the Neck layer of the YoloV8-Tiny object detection model to enhance the model's ability to distinguish between skin areas and background; S22: Use the Clou Loss loss function to replace the original bounding box regression loss function of the YoloV8-Tiny target detection model; S23: Replace the original ResNet backbone network of the OpenPose human pose estimation model with the MobileNetV3-Small network to meet the application requirements of the system's real-time processing; S24: Introduce the Coordinate Attention module into the pose estimation convolutional layer of the OpenPose human pose estimation model.

4. The hip joint surface recognition method based on dual-model fusion according to claim 1, characterized in that: The S3 neonatal hip joint surface recognition method based on the fusion of dual-model recognition results specifically includes: S31: Use the YOLOv8-Tiny object detection model to identify four key parts in the newborn image, namely the ear (a), heel (b), front thigh (c), and back thigh (d), and output the corresponding confidence value C a ,C b ,C c ,C d ; Connect the center points of the ear and heel to form a line segment ab, and connect the midpoints of the front and back thigh contours to form a line segment cd; calculate the intersection point e(u e ,v e ), as the initial positioning point on the skin surface of the hip joint; The geometric mean method is further used to calculate the confidence of line segments ab and cd: Then calculate the comprehensive confidence C of the intersection e e C e =1-(1-C ab )(1-C ab ) S32: The OpenPose human pose estimation model identifies the newborn's left hip joint (f) at S301 and the right hip joint (g) at S302. The Euclidean distance between the two key points and the intersection point e is calculated: S34: Use the exponentially weighted moving average method to smooth the final recognition results of the three most recent consecutive frames, the current frame t, the previous frame t-1, and the previous two frames t-2: p smooth =0.6·p final (t) +0.3·p final (t-1) +0.1·p final (t-2) Among them, p final (t) Indicates the final recognition coordinates of the current frame, p final (t-1) and p final (t-2) Represent the final recognition coordinates of the previous frame and the previous two frames respectively.

5. A hip joint surface recognition system based on dual-model fusion, characterized by: It includes a camera image acquisition module for acquiring image data in the neonatal hip joint ultrasound examination scenario; an image data enhancement and preprocessing module for performing rotation, translation, color change enhancement processing on the acquired image data, and performing standardization and noise reduction; The diverse image dataset annotation and partitioning module is responsible for annotating diverse image data and partitioning it into training and validation sets according to a certain ratio. Improvement module for the YoloV8-Tiny object detection model and OpenPose human pose estimation model, which improves the structure of the YoloV8-Tiny object detection model and the OpenPose human pose estimation model to improve recognition accuracy and computational efficiency; YoloV8-Tiny object detection model and OpenPose human pose estimation model training modules, which train the YoloV8-Tiny object detection model and OpenPose human pose estimation model respectively to obtain the optimal model weights; The YoloV8-Tiny target detection model and the OpenPose human pose estimation model recognition result fusion module perform weighted fusion based on the recognition results of the two models, and ultimately obtain accurate positioning of the hip joint.

Images