Lumbar multi-disease combined intelligent identification model training method and system based on weak attention supervision and positioning

Abstract

The invention provides a training method and system of a lumbar vertebra multi-disease combined intelligent identification model based on attention weakness supervision and positioning. The training method comprises the following steps: training a preset multi-task classification network by using a lumbar magnetic resonance image marked with various lumbar disease grade categories, so that the multi-task classification network can predict the grade categories of various lumbar diseases at the same time; for each predicted lumbar disease type, calculating a class activation diagram corresponding to the target grade class, positioning an area of interest in the lumbar magnetic resonance image according to the class activation diagram, and training a corresponding preset branch classification network by taking the area of interest as a training set, the branch classification network can predict the grade category of the lumbar disease type; and taking the sum of the loss of the multi-task classification network and the loss of each branch classification network as the total loss, optimizing the parameters of the multi-task classification network and each branch classification network by minimizing the total loss, and finally obtaining the lumbar vertebra multi-disease combined intelligent identification model.

Description

Technical Field

[0001] The present invention relates to the technical fields of deep learning, attention mechanism and image multi-task classification and recognition, and in particular to a training method and system for a joint intelligent recognition model for multiple lumbar diseases based on weakly supervised attention positioning. Background Art

[0002] Lumbar spine disease is a very common clinical condition with a high incidence rate. It is primarily caused by soft tissue damage around the spine, bone hyperplasia, and organic lesions in the lumbar intervertebral disc. Long-term improper posture and excessive activity can lead to lumbar degenerative disease. As a chronic condition, lumbar spine disease can go unnoticed, potentially missing the optimal treatment opportunity, resulting in serious harm and significant complications during subsequent treatment, increasing healthcare costs and financial burdens for patients. Lumbar spine disease affects a wide age range, ranging from young people in their 20s and 30s to the elderly. Magnetic resonance imaging (MRI) is a clinically used diagnostic tool for lumbar spine disease, primarily focusing on lumbar disc degeneration, disc herniation severity, intraspinal nerve compression, and tumors within the spinal canal / vertebral column. It is highly valuable for diagnosing lumbar spine disease, especially subtle changes such as vertebral compression fractures and bone contusions.

[0003] In actual clinical image diagnosis, lumbar spine images show that a patient may suffer from multiple common lumbar diseases. These diseases often occur in the pyramidal and intervertebral disc regions. The intervertebral disc region mainly includes the central spinal canal, paracentral region, intervertebral foramina, and extreme lateral region. Different diseases may occur in different areas. Localizing the lesion area is of great significance for the subsequent accurate identification of the disease grade. Predicting the disease grade by manually annotating the lesion area is a common method, but manual labeling is time-consuming and labor-intensive, making medical data collection more difficult and reducing the data size.

[0004] However, the network models trained by the training methods used in the intelligent models in the intelligent detection and identification systems for lumbar diseases based on deep learning are mainly focused on the diagnosis of a single disease. There is still a lack of training methods and intelligent detection and identification systems for the joint identification and classification models of multiple lumbar diseases. Summary of the Invention

[0005] In view of the problem that current research on intelligent diagnosis of lumbar diseases mainly focuses on the study of a single disease and lacks training methods and intelligent detection and recognition systems for the joint recognition and classification models of multiple lumbar diseases, the present invention proposes a training method and system for the joint intelligent recognition model of multiple lumbar diseases based on weakly supervised positioning of attention.

[0006] In a first aspect, the present invention provides a training method for a lumbar vertebra multi-disease joint intelligent recognition model based on weakly supervised attention positioning, comprising: Using lumbar magnetic resonance images labeled with multiple lumbar disease grade categories to train a preset multi-task classification network, so that the multi-task classification network can simultaneously predict the grade categories of multiple lumbar disease; For each predicted lumbar disease type, a class activation map corresponding to a target grade category is calculated, and a region of interest corresponding to the lumbar disease type is located in the lumbar magnetic resonance image based on the class activation map. The region of interest is used as a training set, and a preset branch classification network corresponding to the lumbar disease type is trained using the training set, so that the branch classification network can predict the grade category of the lumbar disease type; The sum of the loss of the multi-task classification network and the loss of each branch classification network is taken as the total loss. The parameters of the multi-task classification network and each branch classification network are optimized by minimizing the total loss, and finally a joint intelligent recognition model for multiple lumbar diseases is obtained.

[0007] Furthermore, the multi-task classification network includes a feature extraction network and a plurality of parallel classification heads respectively connected to the feature extraction network, and one classification head corresponds to a hierarchical category classification task of a type of lumbar disease.

[0008] Furthermore, each classification head adopts a fully connected layer, and the number of output neurons in each fully connected layer is the same as the number of level categories of the corresponding lumbar disease type.

[0009] Furthermore, the class activation map is calculated using a class activation mapping method or a gradient weighted class activation mapping method.

[0010] Furthermore, locating a region of interest corresponding to the type of lumbar disease in the lumbar magnetic resonance image according to the class activation map specifically includes: A threshold is set, and an image region located in the lumbar vertebrae magnetic resonance image and corresponding to a region in the class activation map having a pixel value greater than the threshold is used as a region of interest corresponding to the lumbar vertebrae disease type.

[0011] In a second aspect, the present invention provides a lumbar vertebra multi-disease joint intelligent recognition system based on weak attention supervision positioning, comprising: A training module, configured to train a lumbar vertebrae multi-disease joint intelligent recognition model using the training method for the lumbar vertebrae multi-disease joint intelligent recognition model based on weakly supervised attention positioning as described in the first aspect; The identification module is used to input the lumbar vertebra magnetic resonance image to be identified into the lumbar vertebra multi-disease joint intelligent identification model to predict the lumbar vertebra disease type and the disease level under each disease type.

[0012] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0013] In a fourth aspect, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in the first aspect.

[0014] The beneficial effects of the present invention are: The present invention provides a training method for a lumbar spine multi-disease joint intelligent recognition model based on weakly supervised localization of attention and a corresponding lumbar spine multi-disease joint intelligent recognition system. First, the multi-task classification network is trained using collected lumbar spine magnetic resonance images. Then, class activation maps of different classification tasks are calculated based on the multi-task classification network. The calculated class activation maps are mapped to the original image to obtain the image attention area corresponding to each classification task and extract the located image key area (i.e., region of interest). The branch classification network of each classification task is trained with the key image area as a training set to extract the local features of the image attention area. The global and local multi-task losses are jointly optimized, and the prediction accuracy of the multi-task classification model is improved by combining the global and local features of the image. The training method and system of the present invention can use the attention mechanism as weakly supervised information to locate the key areas of interest of multiple lumbar spine diseases, thereby further improving the performance of the lumbar spine multi-task classification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A flowchart of a training method for a lumbar vertebrae multi-disease joint intelligent recognition model based on weakly supervised attention positioning provided by an embodiment of the present invention.

[0016] Figure 2 A framework diagram of a lumbar vertebrae multi-disease joint intelligent recognition model based on weakly supervised attention positioning provided by an embodiment of the present invention; Figure 3 A schematic diagram of the structure of a lumbar vertebra multi-disease joint intelligent recognition system based on weakly supervised attention positioning provided by an embodiment of the present invention; Figure 4 This is a structural block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0018] In actual clinical image diagnosis, lumbar images may show multiple common lumbar diseases. The present invention regards how to accurately and simultaneously identify multiple lumbar diseases as a multi-task learning process, and regards each lumbar disease recognition and classification task as a sub-classification task in multi-task learning. It also proposes to introduce the attention mechanism into the training of the multi-task classification model for the joint intelligent recognition of multiple common lumbar diseases, which is used for weakly supervised positioning of key areas of different levels of common lumbar diseases, and then further predicts the level of common lumbar diseases based on the local information of the positioned image.

[0019] like Figure 1 As shown, an embodiment of the present invention provides a training method for a lumbar vertebra multi-disease joint intelligent recognition model based on weakly supervised attention positioning, comprising the following steps: S101: Using lumbar magnetic resonance images labeled with multiple lumbar disease grade categories to train a preset multi-task classification network, so that the multi-task classification network can simultaneously predict the grade categories of multiple lumbar disease; S102: For each predicted lumbar disease type, calculating a class activation map corresponding to a target grade category, locating a region of interest corresponding to the lumbar disease type in the lumbar magnetic resonance image based on the class activation map, using the region of interest as a training set, and training a preset branch classification network corresponding to the lumbar disease type using the training set, so that the branch classification network can predict the grade category of the lumbar disease type; S103: The sum of the loss of the multi-task classification network and the loss of each branch classification network is taken as the total loss, and the parameters of the multi-task classification network and each branch classification network are optimized by minimizing the total loss, and finally a lumbar multi-disease joint intelligent recognition model is obtained.

[0020] The training method of the joint intelligent recognition model for multiple lumbar diseases based on weakly supervised positioning of attention provided by an embodiment of the present invention first uses the collected lumbar magnetic resonance images to train a multi-task classification network, then calculates the class activation maps of different classification tasks based on the multi-task classification network, maps the calculated class activation maps to the original image, obtains the image attention area corresponding to each classification task and extracts the located image key area (i.e., the region of interest), uses the key image area as a training set to train the branch classification network of each classification task to extract the local features of the image attention area, jointly optimizes the global and local multi-task losses, and improves the prediction accuracy of the multi-task classification model by combining the global and local features of the image. The training method of the present invention can use the attention mechanism as weakly supervised information to locate the key areas of interest of multiple lumbar diseases, thereby further improving the performance of the multi-task classification model for the lumbar spine.

[0021] Based on the above embodiments, Figure 2 As shown, the embodiment of the present invention takes four sub-classification tasks, namely, lumbar disc herniation, lumbar disc herniation MSU grading, lumbar central spinal canal stenosis grading, and grading of the relationship between disc herniation and nerve root compression, as examples to further illustrate the training method provided by the present invention. Figure 2 In the classification results for each task, different colored blocks represent different categories. Specifically, LDH corresponds to the lumbar disc herniation classification task, with classification results including "yes" and "not"; HL corresponds to the lumbar disc herniation MSU classification task, with classification results divided into grades 1-10 based on the severity of the condition; LCCS corresponds to the lumbar central spinal stenosis classification task, with classification results divided into four grades based on the severity of the condition; and NRC corresponds to the disc herniation and nerve root compression relationship classification task, with classification results divided into four grades based on the degree of nerve root compression.

[0022] In this embodiment, the multi-task classification network includes a feature extraction network and a plurality of parallel classification heads respectively connected to the feature extraction network, and one classification head corresponds to a hierarchical classification task of a type of lumbar disease. Figure 2 As shown in the upper part. Specifically, the feature extraction network can use a basic classification CNN feature extraction network. Classification CNN models include but are not limited to ResNet, Inception, DenseNet, and EfficientNet. The classification head uses a fully connected layer, and the number of output neurons in each fully connected layer is equal to the number of levels of the corresponding lumbar disease type.

[0023] Correspondingly, lumbar magnetic resonance images labeled with multiple lumbar disease grade categories are used to train a preset multi-task classification network, specifically including: collecting axial lumbar magnetic resonance image data with multiple disease grade categories labeled, dividing the data into training set, validation set, and test set according to the cases, and the three data sets have no overlap; training a multi-task classification network with multiple classification heads on the divided training set.

[0024] In this embodiment, for the prediction results of each classification task in the multi-task classification network, a class activation map method including the class activation mapping method (CAM) or the gradient weighted class activation mapping method (GradCAM) is used to calculate the class activation map corresponding to the target level category in the prediction result; a fixed threshold is set, and the class activation area with pixel values ​​greater than the threshold is first determined in the class activation map, and then the image area corresponding to the class activation area is selected in the original lumbar magnetic resonance image as the key area for predicting the target level category, thereby achieving coarse positioning of the region of interest of the classification task. The key area is the image region of interest for weakly supervised positioning.

[0025] In this embodiment, for each sub-classification task, the corresponding region of interest is used as a training set, and the training set is used to train a preset branch classification network corresponding to the lumbar disease type. Specifically, the regions of interest obtained in each sub-classification task are preprocessed, including: uniformly scaling the images to a fixed size to accommodate the input size of each branch classification network; and inputting the preprocessed region of interest images into the branch classification network to train the branch classification network. For example, based on the lumbar central spinal stenosis grading classification head in the multi-task classification network, the region of interest (ROI), i.e., the key region of interest for lumbar central spinal stenosis grading, is obtained through attention calculation. This key region image is preprocessed and used as the input to the branch classification network for lumbar central spinal stenosis grading to train the branch classification network.

[0026] In this embodiment, the branch classification network includes a feature extraction network and a classification head connected to the feature extraction network. Figure 2 The feature extraction network and classification head of the branch classification network can refer to the multi-task classification network or be designed independently, which is not limited here.

[0027] In this embodiment, the multi-task classification network and the branch classification network are trained in the following ways: the multi-task classification network can be trained first to obtain an initial model, and then the multi-task classification network and the branch classification network can be jointly trained based on the initial model, while simultaneously updating the parameters of the multi-task classification network and the branch classification network. Alternatively, the multi-task classification network and the branch classification network can be directly jointly trained, while simultaneously updating the parameters of the multi-task classification network and the branch classification network.

[0028] In this embodiment, the effect of the lumbar vertebrae multi-disease joint intelligent recognition model is tested on the validation set, and the best lumbar vertebrae multi-disease joint intelligent recognition model is selected and tested on the test set to calculate the model recognition accuracy.

[0029] The training method provided by the embodiment of the present invention trains a multi-task classification network based on large-scale magnetic resonance imaging data of common multiple lumbar degenerative diseases (including lumbar disc herniation, lumbar disc herniation MSU grade, lumbar central spinal stenosis grade, and disc herniation and nerve root compression relationship grade). Then, based on the lumbar disease category labels predicted by the multi-task classification network, the image region of interest for each sub-classification task is roughly located, and based on the located region, a branch classification network for the sub-classification task is further trained to perform a more fine-grained recognition model, thereby accurately predicting the grades of multiple lumbar diseases. This method introduces the class activation map obtained by the classification network into the training of the multi-task classification network. Through the training of the multi-task classification model, the category attention area calculated from the actual disease grade of different classification tasks is used as the rough region of interest for the classification task positioning, which is used for further training of the classification model corresponding to each subsequent classification task, thereby prompting the multi-task classification model to improve the positioning accuracy, thereby improving the performance of the multi-task classification model, and realizing intelligent graded joint diagnosis of multiple lumbar diseases, thereby assisting doctors in decision-making.

[0030] Based on the same inventive concept, Figure 3 As shown, an embodiment of the present invention also provides a lumbar vertebra multi-disease joint intelligent recognition system based on weak attention supervision positioning, including a training module and a recognition module.

[0031] The training module is used to train the lumbar vertebrae multi-disease joint intelligent recognition model using the above-mentioned training method of the lumbar vertebrae multi-disease joint intelligent recognition model based on weakly supervised positioning of attention to obtain the lumbar vertebrae multi-disease joint intelligent recognition model; the recognition module is used to input the lumbar vertebrae magnetic resonance image to be identified into the lumbar vertebrae multi-disease joint intelligent recognition model to predict the lumbar vertebrae disease type and the disease level under each disease type.

[0032] In the lumbar spine multi-disease joint intelligent recognition system provided by an embodiment of the present invention, the trained lumbar spine multi-disease joint intelligent recognition model includes a multi-task classification network for weakly supervised positioning and a branch classification network for further improving the classification accuracy of each classification task. The prediction accuracy of the multi-task classification model is improved by combining the global and local features of the image. The system can directly use the attention mechanism as weakly supervised information to perform coarse positioning of key areas of interest for multiple lumbar spine diseases, and perform more fine-grained recognition based on the positioned areas, thereby further improving the performance of the lumbar spine multi-task classification model, realizing intelligent graded joint diagnosis of multiple common lumbar spine diseases, and thus assisting doctors in decision-making.

[0033] Figure 4 An example of a schematic diagram of a physical structure of an electronic device is shown in Figure 4 As shown, the electronic device can include a processor 401, a communications interface 402, a memory 403, and a communications bus 404, wherein the processor 401, the communications interface 402, and the memory 403 complete communications with each other through the communications bus 404. The processor 401 can invoke a logical instruction in the memory 403 to execute a training method of a lumbar multi-disease joint intelligent identification model based on attention weak supervision positioning, which includes: training a preset multi-task classification network using lumbar magnetic resonance images labeled with multiple lumbar disease grade categories, so that the multi-task classification network can simultaneously predict the grade categories of multiple lumbar diseases; for each predicted lumbar disease type, calculate a class activation map corresponding to the target grade category, and locate the region of interest corresponding to the lumbar disease type in the lumbar magnetic resonance image according to the class activation map, take the region of interest as a training set, and train a preset branch classification network corresponding to the lumbar disease type using the training set, so that the branch classification network can predict the grade category of the lumbar disease type; take the sum of the loss of the multi-task classification network and the loss of each branch classification network as the total loss, and optimize the parameters of the multi-task classification network and each branch classification network by minimizing the total loss, and finally obtain the lumbar multi-disease joint intelligent identification model.

[0034] In addition, the logical instructions in the memory 403 described above are implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0035] An embodiment of the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the training method of the lumbar spine multi-disease joint intelligent recognition model based on weak attention supervision positioning provided by the above-mentioned method embodiments.

[0036] An embodiment of the present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the training method of the lumbar multi-disease joint intelligent recognition model based on weak attention supervision positioning provided by the above-mentioned method embodiments.

[0037] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0038] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A training method for a joint intelligent recognition model of multiple lumbar diseases based on weakly supervised attention positioning, characterized by: include: Using lumbar magnetic resonance images labeled with multiple lumbar disease grade categories to train a preset multi-task classification network, so that the multi-task classification network can simultaneously predict the grade categories of multiple lumbar disease; For each predicted lumbar disease type, a class activation map corresponding to a target grade category is calculated, and a region of interest corresponding to the lumbar disease type is located in the lumbar magnetic resonance image based on the class activation map. The region of interest is used as a training set, and a preset branch classification network corresponding to the lumbar disease type is trained using the training set, so that the branch classification network can predict the grade category of the lumbar disease type; The sum of the loss of the multi-task classification network and the loss of each branch classification network is taken as the total loss. The parameters of the multi-task classification network and each branch classification network are optimized by minimizing the total loss, and finally a joint intelligent recognition model for multiple lumbar diseases is obtained.

2. The training method of the lumbar vertebra multi-disease joint intelligent recognition model based on weakly supervised attention positioning according to claim 1 is characterized in that: The multi-task classification network includes a feature extraction network and a plurality of parallel classification heads respectively connected to the feature extraction network, and one classification head corresponds to a hierarchical category classification task of a type of lumbar disease.

3. The training method of the lumbar vertebra multi-disease joint intelligent recognition model based on weakly supervised attention positioning according to claim 2 is characterized in that: Each classification head uses a fully connected layer, and the number of output neurons in each fully connected layer is the same as the number of level categories of the corresponding lumbar disease type.

4. The training method of the lumbar vertebra multi-disease joint intelligent recognition model based on weakly supervised attention positioning according to claim 1 is characterized in that: The class activation map is calculated using a class activation mapping method or a gradient weighted class activation mapping method.

5. The training method of the lumbar vertebra multi-disease joint intelligent recognition model based on weakly supervised attention positioning according to claim 1 is characterized in that: Locating a region of interest corresponding to the type of lumbar disease in the lumbar magnetic resonance image according to the class activation map specifically includes: A threshold is set, and an image region located in the lumbar vertebrae magnetic resonance image and corresponding to a region in the class activation map having a pixel value greater than the threshold is used as a region of interest corresponding to the lumbar vertebrae disease type.

6. A joint intelligent recognition system for multiple lumbar diseases based on weakly supervised localization of attention, characterized by: include: A training module, configured to train a lumbar vertebrae multi-disease joint intelligent recognition model using the training method for a lumbar vertebrae multi-disease joint intelligent recognition model based on weakly supervised attention positioning as described in any one of claims 1 to 5; The identification module is used to input the lumbar vertebra magnetic resonance image to be identified into the lumbar vertebra multi-disease joint intelligent identification model to predict the lumbar vertebra disease type and the disease level under each disease type.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 5 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images