MRI hippocampus 3D intelligent analysis and diagnosis platform based on deep learning

Abstract

The present invention discloses a 3D intelligent analysis and diagnosis platform for MRI hippocampus based on deep learning, which relates to the field of auxiliary medical technology based on deep learning. The platform of this application includes a user login module, a front-end monitoring module, a user demand analysis module, a data loading and preprocessing module, an inference module, a first display module, a second display module, a diagnosis module, a storage module, a query module and an interaction module; this platform combines with a 3D data visualization library to realize a 3D dynamic display of the hippocampus, which can not only observe the structure of the hippocampus more comprehensively and in detail, but also can deeply understand the data from different angles and levels through interactive operations. This platform provides a user-friendly operation interface, even users who have never been exposed to deep learning can easily upload data and quickly start segmentation work. The simple and intuitive design reduces the user's learning cost and improves work efficiency.

Description

Technical Field

[0001] The present application relates to the field of auxiliary medical technology based on deep learning, and in particular to an MRI hippocampus 3D intelligent analysis and diagnosis platform based on deep learning. Background Art

[0002] The hippocampus, a key component of the brain, plays a crucial role in cognitive functions such as memory and learning. Accurate hippocampal segmentation is crucial in medical research and clinical practice, playing a crucial role in the study of brain diseases and the diagnosis of neurological disorders. However, traditional hippocampal segmentation methods often rely on manual labor by physicians, which is not only time-consuming and labor-intensive but also susceptible to subjective factors, leading to errors and instability in the segmentation results. Consequently, the design of software that can automatically segment the hippocampus using deep learning technology has become a hot topic and a challenge in current research. This software leverages advanced deep learning frameworks to efficiently and accurately segment brain images, significantly improving segmentation accuracy and efficiency. Furthermore, the standardized and objective operation of the software effectively reduces human interference and improves the consistency and reliability of segmentation results. Furthermore, software for automatic hippocampal segmentation has broad potential applications. It can not only be used in medical research and clinical practice, providing strong support for the early detection, diagnosis, and treatment of brain diseases, but also in neuroscience research, providing important data support for a deeper understanding of the physiological functions and mechanisms of the hippocampus. In summary, designing a software that can automatically segment the hippocampus is of great significance. It will provide new technical means for the research and diagnosis of brain diseases and promote the progress and development of the medical field. Summary of the Invention

[0003] The purpose of this invention is to develop a deep learning-based MRI hippocampus 3D intelligent analysis and diagnosis platform to automatically segment the hippocampus region in MRI data and visualize the 3D and 2D hippocampus segmentation results. The platform also provides hippocampal volume analysis and diagnosis capabilities, as well as historical record query capabilities. This provides doctors with a convenient and efficient auxiliary diagnostic tool to help them better perform medical diagnoses related to the hippocampus.

[0004] The technical solution adopted by the present invention is: a deep learning-based MRI hippocampus 3D intelligent analysis and diagnosis platform, a B / S architecture platform based on a Web framework, including: a user login module, a front-end monitoring module, a user demand analysis module, a data loading and preprocessing module, a reasoning module, a first display module, a second display module, a diagnosis module, a storage module, a query module and an interaction module;

[0005] Among them, the user login module is used to implement browser-based user registration / and user login operations; if the current user is registering, the user login module stores the current user's registration information in the background database through the storage module;

[0006] The front-end monitoring module monitors the user needs initiated by the user who logs in through the browser; and sends the monitored user needs to the back-end user need parsing module;

[0007] The user demand parsing module is used to parse the received user demands, extract the user demands, match the corresponding user operation items based on the keywords in the user demands, and then map and link the corresponding operation web pages based on the user operation items so as to display the operation web pages to the currently logged-in user through the browser;

[0008] Among them, user operation items include but are not limited to: data loading and preprocessing, model reasoning, diagnosis, result viewing, data retrieval and interaction;

[0009] The background processing module corresponding to the user operation item of data loading and preprocessing is the data loading and preprocessing module, which sends relevant prompt information of the loaded data format to the currently logged-in user through the browser; when the front-end monitoring module monitors the user loading the corresponding data file, it uses the currently loaded data file as the source data for the current user's hippocampal segmentation and sends it to the data loading and preprocessing module;

[0010] The data loading and preprocessing module preprocesses the received hippocampal segmentation source data according to the established data preprocessing format so that the source data matches the input of the segmentation model preset in the inference module; and then stores the preprocessed source data in the specified table entry of the storage module;

[0011] The background processing module corresponding to the user operation item of model inference is the inference module, and the inference module is pre-installed with a trained segmentation model. When the front-end monitoring module monitors that the user's current user operation item is model inference, the current user's most recent loaded data file is read from the storage module and input into the segmentation model to perform forward reasoning operation to obtain the hippocampus segmentation result; at the same time, the hippocampus segmentation result is stored in the specified table item of the storage module, wherein the output form of the hippocampus segmentation result includes two-dimensional and three-dimensional forms; and by calling the first display module, the source data of the hippocampus segmentation and the hippocampus segmentation result are visually output to the user for three-dimensional display, and the hippocampus segmentation result is displayed in two dimensions with different sections;

[0012] The backend processing module corresponding to the user operation item of diagnosis is the diagnosis module, which is used for users (mainly medical staff) to input patient information and for diagnosing doctors to enter information describing and diagnosing the hippocampus segmentation results;

[0013] The background processing module corresponding to the user operation item of result viewing is the second display module, which is used to display the source data and hippocampus segmentation results in three dimensions, and to display the hippocampus segmentation results in two dimensions with different sections;

[0014] The backend processing module corresponding to the user operation item of data retrieval is a query module, which is used to read the content matching the retrieval requirement from the storage module based on the retrieval requirement monitored by the front-end monitoring module and realize the visual output display of the retrieval through the second display module;

[0015] The background processing module corresponding to the interactive user operation item is an interactive module, which is used to provide the user with a display selection operation of optional display content and form of the result viewing content.

[0016] Furthermore, the display selection operations provided by the interaction module include: rotation, translation and / or color change.

[0017] Furthermore, the search content visualized by the query module through the second display module includes but is not limited to: hippocampus segmentation results, hippocampus volume, 2D maximum area, patient information, etc.

[0018] Furthermore, the interactive module provides interactive functions including: mouse hover zoom and page turning functions.

[0019] Furthermore, the segmentation model preset in model inference is a segmentation model based on the nnU-Net model.

[0020] Furthermore, both the first display module and the second display module achieve three-dimensional display through the 3D data visualization library tool vtk.js.

[0021] This application is based on the nnU-Net framework and has developed a software platform for automatically segmenting the hippocampus region in brain MRI images. The platform can process MRI brain data uploaded by users and realize automatic segmentation of the hippocampus through a pre-trained hippocampus segmentation model. In addition, the platform also has the function of displaying 3D and 2D segmentation results, allowing users to intuitively view the segmentation effect. In order to improve the continuity and reference of diagnosis, the platform has also designed a historical record query function, allowing users to trace and compare past segmentation results. At the same time, in order to ensure the performance and accuracy of the model, the platform can also provide real-time feedback on model performance indicators. Combining these functions, the present invention provides doctors with an efficient auxiliary diagnostic tool, which aims to improve the diagnostic efficiency and accuracy of hippocampus-related diseases.

[0022] Compared with the existing technology, the technical solution provided by this application brings at least the following beneficial effects:

[0023] (1) 3D dynamic display technology: This application can be combined with vtk.js, a powerful 3D data visualization library, to achieve a 3D dynamic display of the hippocampus. This display method not only allows for a more comprehensive and detailed observation of the hippocampus structure, but also enables in-depth understanding of the data from different angles and levels through interactive operations. vtk.js supports a variety of common mesh representations and allows for advanced rendering features such as volume rendering, making the display of 3D models more vivid and accurate.

[0024] (2) High-precision segmentation technology: This application uses nn-Unet as the basic segmentation framework, a deep learning model widely used in the field of medical image segmentation. nn-Unet has achieved excellent results in multiple medical image segmentation competitions and is highly versatile, effective, and easy to use. On the platform of this application, this technology is used to achieve high-precision automatic segmentation of the hippocampus, ensuring high accuracy of the segmentation results.

[0025] (3) User-friendly interface: This application aims to create a simple and easy-to-use interface. Even users who have never been exposed to deep learning can easily upload data and quickly start segmentation. This simple and intuitive design reduces the user's learning cost and improves work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0027] Figure 1 An overview of the technical route of a deep learning-based MRI hippocampus 3D intelligent analysis and diagnosis platform provided in the embodiments of this application;

[0028] Figure 2 The front-end and back-end development process of a deep learning-based MRI hippocampus 3D intelligent analysis and diagnosis platform provided in the embodiment of this application;

[0029] Figure 3 The data interaction process between the homepage and result viewing page of a deep learning-based MRI hippocampus 3D intelligent analysis and diagnosis platform provided in an embodiment of the present application;

[0030] Figure 4 The data interaction process of the indicator verification page of the deep learning-based MRI hippocampus 3D intelligent analysis and diagnosis platform provided in the embodiment of the present application;

[0031] Figure 5 2D visualization flowchart provided for the embodiment of this application;

[0032] Figure 6A process diagram of the find_max_slice function in the 2D visualization processing provided in an embodiment of the present application;

[0033] Figure 7 A comparison of data before and after enhancement of a deep learning-based MRI hippocampus 3D intelligent analysis and diagnosis platform provided in an embodiment of the present application;

[0034] Figure 8 This is an example of the homepage segmentation of a deep learning-based MRI hippocampus 3D intelligent analysis and diagnosis platform provided in an embodiment of the present application;

[0035] Figure 9 The result viewing page interface of a deep learning-based MRI hippocampus 3D intelligent analysis and diagnosis platform provided in an embodiment of the present application;

[0036] Figure 10 An indicator verification interface for a deep learning-based MRI hippocampus 3D intelligent analysis and diagnosis platform provided in an embodiment of the present application;

[0037] Figure 11 A hippocampus knowledge popularization interface of an MRI hippocampus 3D intelligent analysis and diagnosis platform based on deep learning is provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to enable people skilled in the art to better understand the technical solutions in this specification, the technical solutions of the embodiments of the present application will be described in detail and completely below in combination with the drawings in the embodiments of the present application. Obviously, the embodiments described with reference to the drawings are exemplary and are intended to be used to explain the present application, and cannot be understood as limiting the present application.

[0039] The embodiment of the present application provides an MRI hippocampus 3D intelligent analysis and diagnosis platform based on deep learning, including: a user login module, a front-end monitoring module, a user demand analysis module, a data loading and preprocessing module, an inference module, a first display module, a second display module, a diagnosis module, a storage module, a query module and an interaction module; wherein the user login module is used to implement browser-based user registration / and user login operations; if the current user is registered, the user login module stores the current user's registration information in the back-end database through the storage module; the front-end monitoring module monitors the user demand initiated by the user who logged in through the browser; and sends the monitored user demand to the back-end user demand module. The user demand parsing module is used to parse the received user demand, extract the user demand, and match the corresponding user operation items based on the keywords in the user demand, and then map and link the corresponding operation web page based on the user operation item to display the operation web page to the currently logged-in user through the browser; wherein, the user operation items include but are not limited to: data loading and preprocessing, model reasoning, diagnosis, result viewing, data retrieval and interaction; the background processing module corresponding to the user operation items of data loading and preprocessing is the data loading and preprocessing module, which sends the relevant prompt information of the loading data format to the currently logged-in user through the browser; when the current end monitoring module monitors the user loading the corresponding After the data file is loaded, the currently loaded data file is used as the source data of the hippocampus segmentation of the current user and sent to the data loading and preprocessing module; the data loading and preprocessing module preprocesses the received source data of the hippocampus segmentation according to the established data preprocessing format, so that the source data matches the input of the segmentation model preset in the inference module (such as the segmentation model of the nnU-Net network); the preprocessed source data is then stored in the specified table item of the storage module; the background processing module corresponding to the user operation item of model inference is the inference module, and the inference module is pre-set with a trained segmentation model. When the front-end monitoring module detects that the user's current user operation item is model inference, it reads the segmentation model from the storage module. The most recently loaded data file of the current user is taken and input into the segmentation model to perform forward reasoning operations to obtain the hippocampus segmentation result; at the same time, the hippocampus segmentation result is stored in the specified table item of the storage module, wherein the output form of the hippocampus segmentation result includes two-dimensional and three-dimensional forms; and by calling the first display module, the source data of the hippocampus segmentation and the hippocampus segmentation result are visually output to the user for three-dimensional display, as well as the hippocampus segmentation result is displayed in two dimensions in different sections; the background processing module corresponding to the user operation item of diagnosis is the diagnosis module, which is used for users (mainly medical personnel) to input patient information, and for the diagnostic doctor to enter information describing and diagnosing the hippocampus segmentation result;The backend processing module corresponding to the user operation item for result viewing is the second display module, which is used to display the source data and hippocampus segmentation results in three dimensions, as well as the hippocampus segmentation results in two dimensions with different slices. The backend processing module corresponding to the user operation item for data retrieval is the query module, which is used to read the content matching the retrieval request from the storage module based on the retrieval request monitored by the front-end monitoring module and realize the visual output display of the retrieval through the second display module. The backend processing module corresponding to the user operation item for interaction is the interaction module, which provides the user with optional display content and format selection operations for the result viewing content.

[0040] Figure 1 The overall technical route of the MRI hippocampus 3D intelligent analysis and diagnosis platform based on deep learning in this application embodiment is as follows: Figure 1 As shown in the figure, data acquisition is first performed, followed by data augmentation techniques such as rotation, translation, Gaussian transform, and brightness adjustment to improve the model's generalization capabilities. Next, the nnU-Net network is used for model training and validation. The software platform is built on the Django framework and uses vtk.js for 3D visualization. This allows the user interface to load MRI data, invoke the optimal model for hippocampal segmentation, and display 2D and 3D segmentation results. Furthermore, the platform integrates with a MySQL database to store, manage, and view historical segmentation results. Finally, the project is packaged and the software is deployed.

[0041] To improve the generalization and robustness of the model, while increasing data diversity and simulating the changing conditions of the real world, this example uses the following data augmentation strategies to reduce the risk of overfitting, optimize the accuracy of segmentation edges, and improve the efficiency of the training process:

[0042] (1) Rotation and translation: Rotation and translation are applied together to increase the computation speed. This application reduces the number of data interpolations required to one. Rotation and translation are each applied with a probability of 0.2. The rotation angles (in degrees) x, y and z are drawn from U(-30, 30) respectively. Translation is achieved by multiplying the coordinates in the voxel grid by a scaling factor. Therefore, a scaling factor less than 1 results in a "zooming out" effect, while values ​​greater than 1 result in a "zooming in" effect. The scaling factors for all types of blocks are drawn from U(0.7, 1.4).

[0043] (2) Gaussian noise: Gaussian noise centered at zero is added to each voxel in the sample independently with a probability of 0.15. Due to intensity normalization, the voxel intensities of all samples are close to zero mean and unit variance.

[0044] (3) Gaussian blur: Blur is applied with probability 0.2 per sample. If this enhancement is triggered in a sample, blur is applied with probability 0.5 in the relevant modality (for samples with a single modality, there is only a joint probability of 0.1). The width of the Gaussian kernel (in voxels) σ is drawn from U(0.5, 1.5) independently for each modality.

[0045] (4) Brightness adjustment: With a probability of 0.15, the voxel intensity is multiplied by x~U(0.7,1.3).

[0046] (5) Contrast enhancement: With a probability of 0.15, the voxel intensity is multiplied by x~U(0.65,1.5). After the multiplication, the values ​​are truncated back to their original value range.

[0047] (6) Simulated low resolution: This augmentation is applied with a probability of 0.25 per sample and 0.5 per relevant mode. The triggered modes are downsampled by a factor of U(1,2) using nearest neighbor interpolation and then resampled back to their original size using cubic interpolation. For 2D blocks or anisotropic 3D blocks, this augmentation is applied only in 2D, preserving the out-of-plane axes (if applicable) in their original state.

[0048] (7) Gamma enhancement: This enhancement is applied with a probability of 0.15. The patch intensities are scaled to a factor of [0,1] of their respective value range. Then, a nonlinear intensity transformation is applied to each voxel:

[0049]

[0050] where γ~U(0.7,1.5) and i represents a voxel value. Subsequently, the voxel intensities are rescaled back to their original value range. With a probability of 0.15, this augmentation negates the voxel intensities before the conversion:

[0051] 1-i new =(ii old ) γ (2)

[0052] The segmentation model used in this platform is nnU-Net, a deep learning model customized for medical image segmentation. It integrates state-of-the-art knowledge and can automatically adapt to different datasets and segmentation tasks. This model has set new records on 49 tasks in 19 international competitions, surpassing many other professional segmentation models.

[0053] In order to evenly evaluate and select the optimal model across different data subsets, this example implements 5-fold cross-validation. Specifically, the training set is randomly divided into five equal parts, each of which serves as a validation set in turn, with the remaining parts serving as training sets. After the model is trained on the training set, it is evaluated on the validation set using metrics such as the Dice coefficient, Jaccard coefficient, PPV, and HD95. This process is repeated five times, with the validation set being changed each time. Finally, the optimal weight configuration is selected based on the average performance and stability of the model across all validation sets.

[0054] The embodiment of this application is based on the weight configuration trained by nnU-Net, and adopts a front-end and back-end separation architecture to develop a software platform. The front-end and back-end development process is as follows: Figure 2 As shown:

[0055] Front-end implementation: This platform is designed to be integrated with a trained nnU-Net model to achieve the goal of hippocampal segmentation. To this end, a navigation bar is designed at the top of the platform's user interface, providing the following functions:

[0056] Homepage: Provides an overview of the platform and quick access to key features. Users can upload brain MRI data and reconstruct and display it in 3D. Click Start Segmentation to automatically segment the hippocampus using the optimal model in the backend, displaying the results in 3D and 2D slices.

[0057] Result View: allows users to review and analyze historical segmentation results, making it easier to compare and analyze in depth. The data interaction between the home page and the result view page is as follows: Figure 3 shown.

[0058] Indicator Verification: Users can view indicators such as Dice coefficient, Jaccard coefficient and model training log to verify and evaluate the performance of the segmentation model. The data interaction process of the indicator verification page is as follows: Figure 4 shown.

[0059] Hippocampus Knowledge Popularization: Provides educational information about the hippocampus to enhance users' understanding of its structure and function, and help users better prevent hippocampus-related diseases.

[0060] Backend Implementation: In the platform's backend development process, this embodiment uses the Django framework, which adopts the Model-View-Template (MVT) architectural pattern to facilitate rapid development. MySQL, an efficient and stable open source relational database management system, is also used. The core functions of the backend include:

[0061] Automated data processing: Automated processing of MRI image data for accurate hippocampal segmentation.

[0062] Image visualization: Convert the segmentation results into 2D images for intuitive display in the front-end interface.

[0063] Data persistence: Ensure that the segmentation results are effectively stored in the MySQL database to facilitate subsequent queries.

[0064] Data retrieval: Provides the function of retrieving data from the database so that the front end can display the stored segmented data.

[0065] Performance evaluation: Calculate key segmentation metrics to demonstrate model performance in the front-end interface.

[0066] The implementation of these comprehensive functions provides powerful data interaction support for the front end, ensuring the smooth operation of the entire software platform and excellent user interaction experience.

[0067] In the embodiment of the present application, when performing 2D visualization on the NIfTI 3D segmentation data of the hippocampus, it is a challenge to accurately locate the position of the hippocampus, because improper section selection may cause the hippocampus to be missing in the 2D view. In order to overcome this problem, the embodiment of the present application proposes a new 2D visualization method, which is based on finding the section with the largest 2D hippocampus area. By determining this optimal section and appropriately expanding around it, it is possible to ensure that the hippocampus is included in the 2D visualization image, thereby improving the accuracy and reliability of visualization.

[0068] See also Figure 5 The 2D visualization processing flow of the embodiment of the present application specifically includes:

[0069] At the beginning of the process, first use the load_nii_file function (file loading function) to import the NIfTI file;

[0070] Then, verify that the loaded image and data are valid.

[0071] Next, the find_max_slice function (identification function) is used to identify the pixels and the largest slice in the given hippocampus 3D array.

[0072] Then, use the plot function to plot the slice and its three adjacent slices, for a total of four images.

[0073] Finally, save these drawn images to files, and the 2D visualization processing process is completed.

[0074] In the embodiment of the present application, the specific processing process of the above find_max_slice function is:

[0075] Assume that data is a three-dimensional array, where data[k,i,j] represents the pixel value of the kth layer (slice), the ith row, and the jth column. The goal of the function is to find the slice index k that maximizes the following sum:

[0076]

[0077] Where M and N are the number of rows and columns of the slice, respectively.

[0078] To increase the speed of the algorithm, in this particular function, only slices of specified consecutive layers may be considered. Based on experience, this embodiment only considers slices from the 100th layer to the 129th layer. Therefore, the formula becomes:

[0079]

[0080] Finally, the function returns the k value that maximizes the above sum, that is, max_slice.

[0081] In the embodiment of the present application, the specific execution process of the find_max_slice function is as follows: Figure 6 Shown, including:

[0082] Let A = data[k,i,j], then A is a three-dimensional array, and the execution goal is to find the index of the slice with the largest value in this matrix;

[0083] First, without changing the Y and Z coordinates, we find the maximum value along the X axis using find_max_slice, and then find the maximum value and its index in each slice. This step does not require rotation because we are only working in the original coordinate system.

[0084] Next, rotate the entire 3D matrix 90° around the Z axis. Mathematically, this can be achieved by applying a rotation matrix. The rotation matrix Rz(90°) for a point (x, y, z) in 3D space that rotates 90° around the Z axis is:

[0085]

[0086] A ` =A*R z (90°) (6)

[0087] After applying this rotation matrix, we can get a new three-dimensional matrix A`. In the rotated matrix A`, find_max_slice is used to find the maximum value along the X axis again.

[0088] Then, rotate the matrix A' by 90° around the Y axis. The rotation matrix Ry(90°) for a 90° rotation around the Y axis is:

[0089]

[0090] A``=A`*R z (90°) (8)

[0091] After applying this rotation matrix, we can get another new three-dimensional matrix A``.

[0092] We then continue searching for the maximum value in the rotated matrix A using find_max_slice. This time, we work in a coordinate system that has been rotated twice.

[0093] Finally, the maxima found in all rotated and unrotated coordinate systems are compared, and their indices are used for plotting, i.e., 2D visualization.

[0094] In order to evaluate the performance of the segmentation model used in the deep learning-based MRI hippocampus 3D intelligent analysis and diagnosis platform of this application, this embodiment uses the Dice coefficient, Jaccard coefficient, Positive Predictive Value (PPV) and HD95 (95% Hausdorff distance) as evaluation indicators. These indicators can fully reflect the accuracy, consistency and edge processing ability of the model in automatically segmenting the hippocampus area. These indicators will be explained one by one below:

[0095] (1) The Dice coefficient is a statistical tool used to evaluate the similarity between two samples. It can measure the consistency between automatic segmentation results and manual segmentation by experts and is a key indicator for evaluating the performance of segmentation algorithms. The Dice coefficient is defined as:

[0096]

[0097] Among them, A represents the hippocampus area manually segmented by experts, and B represents the hippocampus area automatically segmented.

[0098] (2) Jaccard coefficient is widely used in the field of image segmentation and can measure the similarity between the segmentation result and the true annotation. The definition of Jaccard coefficient is:

[0099]

[0100] (3) Positive Predictive Value (PPV) is also an indicator to measure the performance of segmentation. It calculates the ratio of the number of pixels correctly predicted as positive to all pixels predicted as positive. PPV is defined as:

[0101]

[0102] Among them, TP is the number of true positives, FP is the number of false positives, and the higher the PPV, the fewer misjudgments in the hippocampal area pixels predicted by the model, that is, the more accurate the model's prediction.

[0103] (4) HD95 (95% Hausdorff distance) is an indicator for evaluating image segmentation quality. It measures the maximum distance between the edge of the predicted segmentation result and the ground truth. The farthest 5% of points are ignored during the calculation to reduce the impact of extreme values ​​and more accurately reflect the accuracy of the segmentation result. The smaller the HD95, the closer the boundary between the predicted segmentation and the ground truth segmentation, that is, the stronger the model's ability in edge processing.

[0104] Table 1, Table 3.1 lists the performance of the nnU-net 3D model across five validation sets. Each validation set tested the model independently to ensure reliable evaluation. The fourth fold achieved the best results in terms of three key metrics: the Dice coefficient, Jaccard coefficient, and PPV. The average Dice coefficient was 0.9554, demonstrating the model's high recognition accuracy; the Jaccard coefficient was 0.9146, demonstrating its efficiency in identifying overlapping regions; and the PPV value of 0.9736, reflecting the model's high true positive prediction rate. The model also performed well in terms of edge accuracy, maintaining an HD95 of approximately 1 mm, which is crucial for segmenting the delicate hippocampal structure.

[0105] Table 1

[0106]

[0107] Figure 7 This paper compares the Dice coefficient, Jaccard coefficient, and PPV of the nnU-Net 3D model before and after data augmentation. The results show that each coefficient significantly improves after data augmentation. Figure 8 An example of homepage segmentation of the platform proposed in the embodiment of this application is shown. Figure 9 This is the result viewing page interface of this platform. Figure 10 and Figure 11 The following diagrams illustrate the platform's indicator verification interface and hippocampus knowledge interface. The platform's homepage allows users to explore segmentation results in multiple dimensions using 3D and 2D views. Furthermore, a history display function is included on the results review page, allowing doctors to easily review and compare past analysis results, thereby improving diagnostic accuracy and efficiency.

[0108] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0109] Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the technical features being referred to. Thus, a feature defined as "first," "second," etc., may explicitly or implicitly include at least one of the features.

[0110] Any process or method description described in this specification may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed in a different order than shown or discussed, including performing functions in a substantially simultaneous manner or in a reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0111] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0112] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

[0114] The above are only some embodiments of the present application. For those skilled in the art, several modifications and improvements can be made without departing from the inventive concept of the present application, and these all fall within the scope of protection of the present application.

Claims

1. MRI hippocampus 3D intelligent analysis and diagnosis platform based on deep learning, characterized by: The platform is a B / S architecture platform based on the Web framework, including: user login module, front-end monitoring module, user demand analysis module, data loading and pre-processing module, reasoning module, first display module, second display module, diagnosis module, storage module, query module and interaction module; Among them, the user login module is used to implement browser-based user registration / and user login operations; if the current user is registering, the user login module stores the current user's registration information in the background database through the storage module; The front-end monitoring module monitors the user needs initiated by the user who logs in through the browser; and sends the monitored user needs to the back-end user need parsing module; The user demand parsing module is used to parse the received user demands, extract the user demands, match the corresponding user operation items based on the keywords in the user demands, and then map and link the corresponding operation web pages based on the user operation items so as to display the operation web pages to the currently logged-in user through the browser; Among them, user operation items include: data loading and preprocessing, model reasoning, diagnosis, result viewing, data retrieval and interaction; The background processing module corresponding to the user operation item of data loading and preprocessing is the data loading and preprocessing module, which sends relevant prompt information of the loaded data format to the currently logged-in user through the browser; when the front-end monitoring module monitors the user loading the corresponding data file, it uses the currently loaded data file as the source data for the current user's hippocampal segmentation and sends it to the data loading and preprocessing module; The data loading and preprocessing module preprocesses the received hippocampal segmentation source data according to the established data preprocessing format so that the source data matches the input of the segmentation model preset in the inference module; and then stores the preprocessed source data in the specified table entry of the storage module; The background processing module corresponding to the user operation item of model inference is the inference module, and the inference module is pre-installed with a trained segmentation model. When the front-end monitoring module monitors that the user's current user operation item is model inference, the current user's most recent loaded data file is read from the storage module and input into the segmentation model to perform forward reasoning operation to obtain the hippocampus segmentation result; at the same time, the hippocampus segmentation result is stored in the specified table item of the storage module, wherein the output form of the hippocampus segmentation result includes two-dimensional and three-dimensional forms; and by calling the first display module, the source data of the hippocampus segmentation and the hippocampus segmentation result are visually output to the user for three-dimensional display, and the hippocampus segmentation result is displayed in two dimensions with different sections; The background processing module corresponding to the user operation item of diagnosis is the diagnosis module, which is used for the user to input patient information and for the diagnosing doctor to enter information describing and diagnosing the hippocampus segmentation results; The backend processing module corresponding to the user operation item for viewing the results is the second display module, which is used to display the source data and hippocampus segmentation results in three dimensions, as well as to display the hippocampus segmentation results in two dimensions with different sections. The first and second display modules both implement three-dimensional display through the 3D data visualization library tool vtk.js. The backend processing module corresponding to the user operation item of data retrieval is a query module, which is used to read the content matching the retrieval requirement from the storage module based on the retrieval requirement monitored by the front-end monitoring module and realize the visual output display of the retrieval through the second display module; The background processing module corresponding to the interactive user operation item is an interactive module, which is used to provide the user with a display selection operation of optional display content and form of the result viewing content; The two-dimensional display of the hippocampus segmentation results in different sections specifically includes: Use the file loading function to import the NIfTI file and verify whether the loaded image and data are valid; Identify the pixels and the largest slice in the given hippocampus 3D array using the recognition function, draw the slice and its three adjacent slices using the drawing function, and save these drawn images to a file. The specific processing process of the recognition function is as follows: define a three-dimensional array data[k,i,j] to represent the pixel value of the k-th slice, the i-th row, and the j-th category. The goal of the recognition function is to find the pixel value such that the sum The largest slice index k; where M and N are the number of rows and columns of the slice respectively; The specific execution process of the recognition function includes: Let the three-dimensional matrix A = data[k,i,j], without changing the Y and Z coordinates, find the maximum value along the X axis through the identification function, and find the maximum value and its index in each slice; Rotate matrix A 90° around the Z axis to obtain a new three-dimensional matrix A′. In matrix A′, use the identification function to find the maximum value along the X axis again. Then rotate the matrix A′ 90° around the Y axis to obtain the rotated matrix A″, and then find the maximum value in the matrix A″ through the identification function; Finally, the maxima found in all rotated and unrotated coordinate systems are compared, and their indices are used for plotting.

2. The intelligent analysis and diagnosis platform according to claim 1, characterized in that: The display selection operations provided by the interactive module include: rotation, translation and / or color change.

3. The intelligent analysis and diagnosis platform according to claim 1, characterized in that: The query module visualizes the search content output by the second display module, including: hippocampus segmentation results, hippocampus volume, 2D maximum area, and patient information.

4. The intelligent analysis and diagnosis platform according to claim 1, characterized in that: The interactive functions provided by the interactive module also include: mouse hover zoom and page turning functions.

5. The intelligent analysis and diagnosis platform according to claim 1, characterized in that: The preset segmentation model in model inference is a segmentation model based on the nnU-Net model.

Images