Expert knowledge-constrained deep learning method for glioma grading

Abstract

This application discloses a deep learning method, apparatus, device, and storage medium for expert knowledge-constrained glioma grading. The method includes: constructing radiomics labels that include expert knowledge; constructing a training model for discrete distance-constrained glioma grading; and inputting the radiomics labels into the glioma grading model to complete parameter optimization. The above-mentioned solution provided by this application has good grading performance and high clinical usability in the glioma grading task, thereby reducing the workload of doctors in clinical use and enabling the model to more efficiently utilize patients' brain magnetic resonance imaging data.

Description

Technical Field

[0001] The present invention relates to medical imaging technology, and in particular to a deep learning method, apparatus, device and storage medium thereof for expert knowledge constraints in grading brain gliomas. Background Art

[0002] Gliomas are the most common primary neuroepithelial malignancies. Treatment and prognosis vary significantly between patients with different glioma grades. Magnetic resonance imaging, with its superior imaging contrast and resolution, plays a crucial role in the preoperative diagnosis of gliomas. To assist physicians in achieving faster and more accurate glioma diagnosis, automated glioma grading based on brain MRI has become an important research area.

[0003] Existing glioma grading technologies primarily utilize deep learning methods such as convolutional neural networks. In these methods, brain MRI images of glioma patients are used as input, and the tumor region is selected based on a manually drawn tumor segmentation map and fed into the convolutional neural network. Using convolutional neural networks, high-dimensional features within the tumor region can be obtained, enabling accurate grading of gliomas. However, these methods present several challenges: Due to the complexity of gliomas in shape and texture, delineating the tumor requires extensive expertise and experience on the part of the physician, and the manual delineation of the tumor region is time-consuming and prone to error. Furthermore, different physicians may produce slightly different segmentation results for the same glioma MRI image, which may affect the model's clinical grading performance. Finally, the grading models derived from these methods require the physician to manually delineate the tumor region before they can be used clinically, increasing the physician's workload and reducing the model's clinical usability. Summary of the Invention

[0004] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide a deep learning method, apparatus, device and storage medium thereof for expert knowledge constraints for grading brain gliomas.

[0005] In a first aspect, an embodiment of the present application provides a deep learning method for expert knowledge constraints for glioma grading, the method comprising: constructing an imaging genomics label, wherein the imaging genomics label includes expert knowledge; constructing a training model for discrete distance-constrained glioma grading; and inputting the imaging genomics label into the glioma grading model to complete parameter optimization.

[0006] In one embodiment, constructing the radiomics label includes: manually outlining the segmentation file; extracting features of the tumor area in the segmentation file; screening the features to remove redundant information; and constructing the label using the screened features.

[0007] In one embodiment, extracting features of the tumor region in the segmentation file includes extracting high-throughput quantitative features of the tumor region, wherein the quantitative features include first-order statistics, shape and size, and texture features.

[0008] In one embodiment, the screening features and constructing labels using the screened features include: screening features through mutual information method, Student's t-test and recursive feature elimination method, and constructing radiomics labels using the screened features through a logistic regression classifier.

[0009] In one embodiment, before the training of constructing the discrete distance constrained glioma grading model, the method further includes: constructing a 3D convolutional neural network as the glioma grading model.

[0010] In the second aspect, an embodiment of the present application also provides a deep learning device for expert knowledge constraints for glioma grading, which includes: a first construction unit for constructing an imaging genomics label, wherein the imaging genomics label includes expert knowledge; a second construction unit for constructing a training model for discrete distance constraints on glioma grading; and an optimization unit for inputting the imaging genomics label into the glioma grading model to complete parameter optimization.

[0011] In one embodiment, constructing the radiomics label includes: manually outlining the segmentation file; extracting features of the tumor area in the segmentation file; screening the features to remove redundant information; and constructing the label using the screened features.

[0012] In one embodiment, extracting features of the tumor region in the segmentation file includes extracting high-throughput quantitative features of the tumor region, wherein the quantitative features include first-order statistics, shape and size, and texture features.

[0013] In a third aspect, an embodiment of the present application further provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements any of the methods described in the embodiments of the present application.

[0014] In a fourth aspect, an embodiment of the present application further provides a computer device and a computer-readable storage medium on which a computer program is stored, wherein the computer program is used to implement any method described in the embodiments of the present application when the computer program is executed by a processor.

[0015] Beneficial effects of the present invention:

[0016] The expert knowledge-constrained deep learning method for glioma grading provided by the present invention has good grading performance and high clinical usability in the glioma grading task, thereby reducing the workload of doctors in clinical use and enabling the model to more efficiently utilize the patient's brain magnetic resonance image data. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0018] Figure 1 A schematic diagram of a flow chart of an expert knowledge-constrained deep learning method for grading gliomas provided in an embodiment of the present application is shown;

[0019] Figure 2 FIG2 shows an exemplary structural block diagram of an expert knowledge-constrained deep learning apparatus 200 for grading brain gliomas according to an embodiment of the present application;

[0020] Figure 3 A schematic diagram showing the structure of a computer system suitable for implementing a terminal device according to an embodiment of the present application is shown;

[0021] Figure 4 A schematic diagram of the overall technical route provided by the embodiments of the present application is shown. DETAILED DESCRIPTION

[0022] To make the above-mentioned objects, features, and advantages of the present invention more readily apparent, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings. The following description sets forth numerous specific details to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art may make similar modifications without departing from the scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0023] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0024] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0025] In the present invention, unless otherwise specified or limited, the terms "installed," "connected," "connect," "fixed," etc. should be understood in a broad sense. For example, they can refer to fixed connection, detachable connection, or integration; mechanical connection, electrical connection; direct connection, or indirect connection through an intermediate medium; internal communication between two components, or interaction between two components, unless otherwise specified. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0026] In the present invention, unless otherwise expressly specified or limited, when a first feature is "above" or "below" a second feature, it may mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediary. Furthermore, when a first feature is "above," "above," or "above" a second feature, it may mean that the first feature is directly above or diagonally above the second feature, or simply means that the first feature is at a higher level than the second feature. When a first feature is "below," "below," or "below" a second feature, it may mean that the first feature is directly below or diagonally below the second feature, or simply means that the first feature is at a lower level than the second feature.

[0027] It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it may be directly on the other element or there may be an intermediate element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be an intermediate element. The terms "vertical," "horizontal," "upper," "lower," "left," "right," and similar expressions used herein are for illustrative purposes only and do not represent the only implementation methods.

[0028] Please refer to Figure 1 , Figure 1 A flow chart of an expert knowledge-constrained deep learning method for grading brain gliomas provided in an embodiment of the present application is shown.

[0029] like Figure 1 As shown, the method includes:

[0030] Step 110, constructing a radiomics label, wherein the radiomics label includes expert knowledge;

[0031] Step 120, constructing a training model for discrete distance-constrained glioma classification;

[0032] Step 130: Input the radiomics labels into the glioma grading model to complete parameter optimization.

[0033] Using the above technical solution, the model has good grading performance and high clinical usability in the task of glioma grading, thereby reducing the workload of doctors in clinical use and enabling the model to more efficiently utilize the patient's brain magnetic resonance image data.

[0034] In some embodiments, the construction of radiomics labels in the present application includes: manually outlining a segmentation file; extracting features of the tumor area in the segmentation file; screening features to remove redundant information; and constructing labels using the screened features.

[0035] In some embodiments, the extracting of features of the tumor region in the segmentation file in the present application includes: extracting high-throughput quantitative features of the tumor region, wherein the quantitative features include first-order statistics, shape and size, and texture features.

[0036] In some embodiments, the screening features in the present application and the use of the screened features to construct labels include: screening features through mutual information method, Student's t-test and recursive feature elimination method, and constructing radiomics labels using the screened features through a logistic regression classifier.

[0037] In some embodiments, before the training of constructing the discrete distance constrained glioma grading model, the method in the present application further includes: constructing a 3D convolutional neural network as the glioma grading model.

[0038] refer to Figure 4 As shown in the figure, the technical solution of the present invention can be divided into two parts: building expert knowledge and training convolutional neural networks constrained by expert knowledge. The technical route is shown in the attached figure. Figure 4As shown. (1) Constructing expert knowledge: In order to help the model better learn the information of the tumor area and get rid of the dependence on the segmentation file during clinical use, the present invention first constructs an imaging omics label containing expert knowledge. The entire construction process is divided into four parts: a. Manually outline the segmentation file, b. Feature extraction, c. Feature selection and d. Construction of imaging omics labels. Among them, the manual outline of the segmentation file requires the assistance of a doctor to complete in order to proceed with the subsequent steps. For the extracted tumor area, high-throughput quantitative features are extracted. These features describe the relevant characteristics of the tumor from three aspects: first-order statistics, shape size and texture. High-dimensional features may contain redundant information, which affects the label construction process. The present invention uses a cascade feature selection method to screen out key features, wherein the feature selection method consists of mutual information method, Student's t test and recursive feature elimination method. The selected key features are used to construct imaging omics labels containing expert knowledge through a logistic regression classifier.

[0039] (2) Training expert knowledge constrained convolutional neural networks: The present invention constructs a 3D convolutional neural network as the final brain glioma grading model. The model directly uses brain magnetic resonance imaging as input to avoid losing information between the tumor and the surrounding tissue. The obtained imaging genomics labels containing expert knowledge are used as prior knowledge, and a discrete distance is designed to constrain the training of the model, introducing expert knowledge into the parameter optimization process during 3D convolutional neural network training. In this way, the expert knowledge constrained deep learning method designed and implemented by the present invention can utilize data more efficiently, eliminate the model's dependence on segmentation files during clinical use, and improve the clinical usability of the grading model.

[0040] Further, refer to Figure 2 , Figure 2 An exemplary structural block diagram of an expert knowledge-constrained deep learning device 200 for glioma grading according to an embodiment of the present application is shown.

[0041] like Figure 2 As shown, the device includes: a first construction unit 210 for constructing an imaging genomics label, wherein the imaging genomics label includes expert knowledge; a second construction unit 220 for constructing a training model for discrete distance constrained glioma grading; and an optimization unit 230 for inputting the imaging genomics label into the glioma grading model to complete parameter optimization.

[0042] It should be understood that the units or modules described in the apparatus 200 are similar to those described in the reference Figure 1The various steps in the described method correspond to each other. Therefore, the operations and features described above for the method are also applicable to the device 200 and the units contained therein, and will not be repeated here. The device 200 can be pre-implemented in the browser or other security application of the electronic device, or loaded into the browser or its security application of the electronic device by downloading or other means. The corresponding units in the device 200 can cooperate with the units in the electronic device to implement the solution of the embodiment of the present application.

[0043] Reference below Figure 3 , which shows a structural diagram of a computer system 300 suitable for implementing a terminal device or server of an embodiment of the present application.

[0044] like Figure 3 As shown, the computer system 300 includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage unit 308 into a random access memory (RAM) 303. Various programs and data required for the operation of the system 300 are also stored in the RAM 303. The CPU 301, ROM 302, and RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0045] The following components are connected to the I / O interface 305: an input section 306 including a keyboard, a mouse, and the like; an output section 307 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 308 including a hard disk; and a communication section 309 including a network interface card such as a LAN card or a modem. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. Removable media 311, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 310 as needed, so that computer programs read therefrom can be installed into the storage section 308 as needed.

[0046] In particular, according to the embodiments of the present disclosure, the above reference Figure 1 The described process can be implemented as a computer software program. For example, embodiments of the present disclosure include an expert knowledge-constrained deep learning method for grading brain gliomas, which includes a computer program tangibly embodied on a machine-readable medium, the computer program including a computer program for executing Figure 1 In such an embodiment, the computer program may be downloaded and installed from a network via the communication section 309 and / or installed from the removable medium 311 .

[0047] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the aforementioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0048] The units or modules involved in the embodiments described in the present application can be implemented by software or by hardware. The described units or modules can also be set in a processor. For example, they can be described as: a processor includes a first sub-area generation unit, a second sub-area generation unit, and a display area generation unit. Among them, the names of these units or modules do not constitute a limitation of the unit or module itself under certain circumstances. For example, the display area generation unit can also be described as "a unit for generating a display area for text based on the first sub-area and the second sub-area".

[0049] As another aspect, the present application further provides a computer-readable storage medium, which may be the computer-readable storage medium included in the aforementioned apparatus in the above-mentioned embodiment, or may be a separate computer-readable storage medium not incorporated into the apparatus. The computer-readable storage medium stores one or more programs, which are used by one or more processors to execute the text generation method for a transparent window envelope described in the present application.

[0050] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the aforementioned inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A deep learning method for grading gliomas constrained by expert knowledge, characterized by: The method includes: Constructing a radiomics label, wherein the radiomics label includes expert knowledge, and constructing the radiomics label includes: Manually outline the segmentation file; Extracting features of the tumor region in the segmentation file, including: extracting high-throughput quantitative features in the tumor region, wherein the quantitative features include first-order statistics, shape and size, and texture features; Filter features and remove redundant information; Use the filtered features to construct labels; The filtering features and constructing labels using the filtered features include: Features were screened using the mutual information method, Student's t-test, and recursive feature elimination, and the screened features were used to construct radiomics labels using a logistic regression classifier. Constructing a 3D convolutional neural network as a glioma grading model; Constructing training for discrete distance-constrained glioma grading models; The radiomics labels were input into the glioma grading model to complete parameter optimization.

2. A deep learning device for expert knowledge constraint in glioma grading, characterized in that: The device includes: The first construction unit is used to construct a radiomics label, wherein the radiomics label includes expert knowledge. The construction of the radiomics label includes: Manually outline the segmentation file; Extracting features of the tumor region in the segmentation file, including: extracting high-throughput quantitative features in the tumor region, wherein the quantitative features include first-order statistics, shape and size, and texture features; Filter features and remove redundant information; Use the filtered features to construct labels; The filtering features and constructing labels using the filtered features include: Features were screened using the mutual information method, Student's t-test, and recursive feature elimination, and the screened features were used to construct radiomics labels using a logistic regression classifier. Constructing a 3D convolutional neural network as a glioma grading model; The second construction unit is used to construct a training model for discrete distance-constrained glioma grading; An optimization unit is used to input the radiomics label into the glioma grading model to complete parameter optimization.

3. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to claim 1 is implemented.

4. A computer-readable storage medium having stored thereon a computer program for: When the computer program is executed by a processor, the method according to claim 1 is implemented.

Images