Hemophilia arthropathy detection method based on deep learning

Abstract

The invention provides a hemophilia arthropathy detection method based on deep learning, and the method comprises the steps: carrying out data collection and standardization processing through employing an ultrasonic image database, and constructing a standardized ultrasonic image database; for the constructed database, expanding a small sample data set by using a data enhancement technology; performing multi-task classification and detection on the ultrasonic image data by using a deep learning algorithm; performing pre-classification and pre-labeling on the unlabeled ultrasonic images through the trained classification model; and combining model weights of different stages in the training process to generate a final hybrid model. The hemophilia arthropathy detection method based on deep learning has the beneficial effects that the ultrasonic image is intelligently analyzed through deep learning, real-time diagnosis and auxiliary marking are realized, and a solution is provided for the problems of data scarcity, model over-fitting and adaptability of an algorithm and equipment.

Description

Technical Field

[0001] The present invention belongs to the technical field of image detection, and in particular relates to a method for detecting hemophilia joint lesions based on deep learning. Background Art

[0002] Hemophilic arthropathy is the most common complication and the main cause of disability in hemophilia patients. Its early detection and treatment are of great significance in reducing the risk of bleeding events and reducing joint damage.

[0003] As a real-time, non-invasive assessment method, ultrasound imaging can clearly display joint lesions (such as joint effusion, synovial hyperplasia, synovial blood flow signal hyperplasia, cartilage destruction, etc.), providing a basis for the diagnosis and prevention of hemophilic arthropathy.

[0004] However, existing image assessment relies on manual interpretation, which is inefficient and susceptible to subjective factors. Summary of the invention

[0005] In view of this, the present invention aims to propose a deep learning-based method for detecting hemophilia arthropathy, which integrates a portable ultrasound device with a deep learning algorithm, provides a real-time, intelligent joint lesion risk assessment solution for hemophilia patients, and realizes early warning and classification of hemophilia arthropathy.

[0006] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0007] A method for detecting hemophilia joint lesions based on deep learning, comprising:

[0008] Use the ultrasound image database to collect and standardize data and build a standardized ultrasound image database;

[0009] For the constructed database, data enhancement technology is used to expand the small sample data set to expand the training sample size;

[0010] Use deep learning algorithms to perform multi-task classification and detection on ultrasound image data to identify joint lesion characteristics;

[0011] Pre-classify and pre-label the unlabeled ultrasound images through the trained classification model;

[0012] The model weights from different stages of the training process are combined to generate the final hybrid model.

[0013] Furthermore, the use of the ultrasound image database for data collection and standardization processing to construct a standardized ultrasound image database includes:

[0014] The ultrasound image data of the knee joint of the hemophilia patient are collected according to the standardized HC scanning method, wherein the ultrasound image data include K1 midsagittal section, K2a parapatellar lateral cross section, K2b parapatellar medial cross section, K3 femoral trochlea cross section, and K4 femorotibial joint coronal section;

[0015] Feature annotation is performed on the collected ultrasound image data, wherein the features include patient code, picture number, patient age, left knee or right knee, section number, section result, and corresponding HC item score.

[0016] Furthermore, the constructed database is used to expand the small sample data set using data enhancement technology, including:

[0017] According to the characteristics of different joints, an enhancement method combining symmetric flipping, rotation transformation and cutting is used to expand the small sample data set.

[0018] Furthermore, the method uses a deep learning algorithm to perform multi-task classification and detection on ultrasound image data, including:

[0019] By constructing a targeted training set, the model is iteratively trained to optimize the weight parameters.

[0020] Furthermore, the model weights at different stages of the training process are combined to generate a final hybrid model, including:

[0021] Select model weights at different stages of the training process and combine multiple model checkpoints to generate a final model with stronger generalization capabilities.

[0022] Furthermore, multiple model checkpoints in the training process are sampled, and the last model checkpoint is weighted mixed with multiple previous checkpoints to obtain a final model after weight mixing.

[0023] Furthermore, the present solution discloses an electronic device, comprising a processor and a memory that is communicatively connected to the processor and is used to store executable instructions of the processor, wherein the processor is used to execute a method for detecting hemophilia joint lesions based on deep learning.

[0024] Furthermore, the present solution discloses a server comprising at least one processor and a memory communicatively connected to the processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor so that the at least one processor executes a method for detecting hemophilia joint lesions based on deep learning.

[0025] Furthermore, the present solution discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, implements a method for detecting hemophilia joint lesions based on deep learning.

[0026] Compared with the prior art, the method for detecting hemophilia joint lesions based on deep learning described in the present invention has the following beneficial effects:

[0027] (1) The present invention discloses a method for detecting hemophilia joint lesions based on deep learning, which uses deep learning to intelligently analyze ultrasound images, realizes real-time diagnosis and auxiliary annotation, and proposes solutions to the problems of data scarcity, model overfitting, and algorithm and device adaptability.

[0028] (2) The present invention discloses a method for detecting hemophilia joint lesions based on deep learning, which designs a complete data framework from hemophilia joint disease ultrasound image data to deep learning training, thereby achieving the purpose of real-time monitoring of hemophilia joint disease with high accuracy and high generalization through ultrasound images. At the same time, it can reduce the cost of manual labeling, making it possible to rapidly expand the hemophilia joint ultrasound image database. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0030] Figure 1 This is a schematic diagram of a method for detecting hemophilia joint lesions based on deep learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0032] The present invention will be described in detail below with reference to examples.

[0033] By training deep neural networks, the present invention realizes a high-accuracy and high-generalization hemophilia risk identification algorithm that can be deployed in portable devices. The integration of artificial intelligence and portable ultrasound equipment can provide joint screening and auxiliary diagnosis for hemophilia patients more quickly, conveniently and objectively, realize early detection and warning of hemophilic arthropathy and risk grading, and provide real-time monitoring and evaluation of patients' joint health status; improve the efficiency of homogeneous ultrasound evaluation, thereby improving the multidisciplinary comprehensive management level of domestic hemophilia management centers; benefit more hemophilia patients, apply it to home care and remote areas, and realize remote consultation and individualized management.

[0034] The specific implementation process mainly includes the following steps:

[0035] (1) Hemophilia joint ultrasound image database

[0036] The present invention selects outpatient and inpatient data from September 2015 to June 2024, and the patients diagnosed with hemophilia A or hemophilia B are aged 1-64 years (median age 32.5 years old). All patients are male. Ultrasound images of the knee joints of all hemophilia patients are collected, a total of 900 cases, and a total of 5500 ultrasound images;

[0037] Ultrasound images were strictly recorded in accordance with the HC scanning method and standard sections, including K1 midsagittal section; K2a parapatellar lateral cross section; K2b parapatellar medial cross section; K3 femoral trochlea cross section; K4 femorotibial joint coronal section;

[0038] All knee joint images were numbered and then manually scored, including patient code, image number, patient age, joint (left knee or right knee), section number, section result (0 for negative and 1 for positive), and the corresponding HC item score;

[0039] After completion, it will be submitted to the fixed chief reviewer for review and summary.

[0040] (2) Data enhancement for joint ultrasound images

[0041] Joint ultrasound images have high acquisition and annotation costs and are typical small sample data. At the same time, joint ultrasound image data has joint symmetry, and the characteristics of the same detection item are relatively fixed, which can easily lead to overfitting in training. In actual application scenarios, due to the patient's limited professional knowledge or mobility difficulties, the acquired images may be quite different from the standard data set, posing a challenge to the generalization of the model. In view of the above characteristics, the present invention designs a data enhancement method for joint ultrasound images;

[0042] According to the characteristics of different joints, targeted methods such as symmetry, rotation, and special feature cutting combinations are used to efficiently and high-quality expand the existing data set. The expanded joint ultrasound image data set contains more diverse situations and can better cover the situations that may occur in real scenes, which is helpful for the generalization ability of the model and the actual application performance.

[0043] (3) Image classification algorithm

[0044] The present invention uses the mainstream deep learning target detection image classification method, and is designed for the diagnosis of hemophilia arthritis ultrasound images. Based on an in-depth analysis of the characteristics of existing data sets and actual clinical application needs, the present invention selects two deep learning models with smaller parameters, namely the 37M model and the 64M model. These two models effectively reduce the demand for hardware resources and reduce computing delays in practical applications while ensuring the accuracy of image classification. The specific task is to perform binary classification on the given ultrasonic joint image in subtasks such as joint effusion, synovial hyperplasia, synovial blood flow signal hyperplasia, cartilage destruction and subchondral bone changes, and obtain a positive or negative conclusion for the corresponding subtask;

[0045] In terms of model design, the present invention adopts a common network structure for target detection and image recognition. The network structure consists of three parts: Backbone, Neck and Head. Backbone is responsible for extracting features from the input image. It consists of multiple convolutional layers and deconvolutional layers, and incorporates residual connections and bottleneck structures. Such a design can enhance its expressiveness while compressing the network size. At the same time, in order to further enhance the expressiveness, technologies such as deep separable convolution and dilated convolution are also used in the network design. The function of the Neck part is to integrate multi-scale features. It merges the feature maps output by different levels of Backbone to enhance the representation ability of features. Neck adopts the PAN-FAN architecture, which includes two PAN modules and one SPPF module. The PAN module enhances the description ability of the feature map through information transmission from bottom to top and from top to bottom. The SPPF module performs multi-scale pooling operations to connect feature maps of different scales, thereby enhancing the model's detection performance for targets of various sizes. The Head part is responsible for completing the final task of target detection and classification. It consists of a detection head and a classification head. The detection head contains a series of convolutional and deconvolutional layers, and its main function is to generate detection results, including the bounding box, category label, and confidence of the predicted target. The classification head classifies each feature map through global average pooling technology and outputs the probability distribution of the corresponding category;

[0046] After training, this method achieved a binary classification accuracy of more than 90% in subtasks such as joint effusion, synovial hyperplasia, synovial blood flow signal hyperplasia, cartilage destruction and subchondral bone changes in ultrasound image diagnosis of hemophilic arthritis.

[0047] (4) Auxiliary marking

[0048] The high-accuracy ultrasound image classification model obtained through training can be directly used to assist manual labeling. In the actual manual labeling task, the ultrasound image classification model can efficiently and massively complete the pre-classification and pre-labeling of the original data, thereby providing certain prior information during manual labeling, effectively reducing the difficulty and workload of manual labeling. With the improvement of labeling efficiency, it is possible to further expand the scale of the hemophilia joint ultrasound image database, and then the training method of the present invention can iterate the training-labeling cycle, and gradually obtain an ultrasound image classification model with better classification performance.

[0049] (5) Model weight mixing

[0050] At present, the total amount of hemophilia joint ultrasound image data sets is small, and overfitting is prone to occur during training, resulting in reduced performance of the classification model in actual application scenarios. To address this problem, the present invention adopts a model weight mixing method. When the model training is completed, multiple model checkpoints in the training process are sampled and the last model checkpoint is weighted mixed with multiple previous checkpoints to obtain the final model after weight mixing;

[0051] The final model after weight mixing retains the knowledge gained from the training of the hemophilia joint ultrasound image dataset, while also having a certain degree of extensive target detection capabilities in the pre-trained model. This method can improve the generalization ability of the model and reduce its overfitting problem in specific scenarios. Through experimental verification, this method effectively improves the generalization ability of the model while ensuring that the detection performance remains basically unchanged.

[0052] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0053] In the several embodiments provided in the present application, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the division of the units described above is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The above-mentioned units may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0054] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

[0055] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting hemophilia joint lesions based on deep learning, characterized in that: include: Use the ultrasound image database to collect and standardize data and build a standardized ultrasound image database; For the constructed database, use data enhancement technology to expand the small sample data set; Use deep learning algorithms to perform multi-task classification and detection of ultrasound image data; Pre-classify and pre-label the unlabeled ultrasound images through the trained classification model; The model weights from different stages of the training process are combined to generate the final hybrid model.

2. A method for detecting hemophilia joint lesions based on deep learning according to claim 1, characterized in that: The method of using the ultrasound image database to collect data and perform standardization processing to construct a standardized ultrasound image database includes: The ultrasound image data of the knee joint of the hemophilia patient are collected according to the standardized HC scanning method, wherein the ultrasound image data include K1 midsagittal section, K2a parapatellar lateral cross section, K2b parapatellar medial cross section, K3 femoral trochlea cross section, and K4 femorotibial joint coronal section; Feature annotation is performed on the collected ultrasound image data, wherein the features include patient code, picture number, patient age, left knee or right knee, section number, section result, and corresponding HC item score.

3. A method for detecting hemophilia joint lesions based on deep learning according to claim 1, characterized in that: The constructed database uses data enhancement technology to expand the small sample data set, including: According to the characteristics of different joints, an enhancement method combining symmetric flipping, rotation transformation and cutting is used to expand the small sample data set.

4. The method for detecting hemophilia joint lesions based on deep learning according to claim 1, characterized in that: The method uses a deep learning algorithm to perform multi-task classification and detection on ultrasound image data, including: By constructing a targeted training set, the model is iteratively trained to optimize the weight parameters.

5. The method for detecting hemophilia joint lesions based on deep learning according to claim 1, characterized in that: The method combines the model weights at different stages of the training process to generate a final hybrid model, including: Select model weights at different stages of the training process and combine multiple model checkpoints to generate a final model with stronger generalization capabilities.

6. A method for detecting hemophilia joint lesions based on deep learning according to claim 5, characterized in that: Multiple model checkpoints in the training process are sampled, and the last model checkpoint is weighted mixed with multiple previous checkpoints to obtain the final model after weight mixing.

7. An electronic device, comprising a processor and a memory connected to the processor for storing instructions executable by the processor, characterized in that: The processor is used to execute a method for detecting hemophilia joint lesions based on deep learning as described in any one of claims 1 to 6.

8. A server, characterized in that: It includes at least one processor and a memory communicatively connected to the processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor so that the at least one processor executes a deep learning-based method for detecting hemophilia joint lesions as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements a method for detecting hemophilia joint lesions based on deep learning as described in any one of claims 1 to 6.

Images