Predictive apparatus and method for vascular fluid characteristics, data segmentation apparatus

Abstract

The embodiment of the present application provides a kind of blood vessel fluid characteristic prediction device and method, data segmentation device.Vessel fluid characteristic prediction method includes: obtaining the medical image data to be analyzed;Using the neural network based on multiscale, the blood vessel medical image data is segmented from the medical image data;The blood vessel medical image data is converted into surface reconstruction data, to obtain the discrete point data of blood vessel;And using the neural network based on full connection, the fluid characteristic of each discrete point of the blood vessel is predicted.Thereby, not only can the blood vessel fluid characteristic be quickly predicted, but also simple operation, without manual intervention and strong stability.

Description

Technical Field

[0001] This application relates to the field of medical device technology, and in particular to a device and method for predicting vascular fluid characteristics, and a data segmentation device. Background Technology

[0002] Vascular diseases are the leading cause of death worldwide, with very high morbidity and mortality rates. They are recognized as killers that endanger human health. Therefore, timely and accurate diagnosis of vascular diseases is of great significance for improving patient prognosis, enhancing quality of life, and reducing mortality and medical costs.

[0003] Hemodynamics is a branch of biomechanics. One of its main tasks is to apply the theories and methods of fluid mechanics to study the causes, conditions, states, and various influencing factors of blood flow along blood vessels, in order to elucidate the laws of blood flow, its physiological significance, and its relationship with diseases.

[0004] In biomedicine, hemodynamics focuses on certain fluid characteristics of the human circulatory system, often used to characterize changes under physiological and pathological conditions. These fluid characteristics include wall shear stress (WSS), fluid pressure, and fluid velocity. Utilizing these fluid characteristics can not only detect and classify certain vascular diseases but also help in developing adjunctive treatment solutions.

[0005] For example, the fractional flow reserve (FFR) calculated using fluid characteristics can be used to detect and classify coronary atherosclerosis, while the maximum wall shear stress (MWSS) and wall shear stress gradient (WSSG) calculated using fluid characteristics can be used to assess the risk of rupture of cerebral aneurysms.

[0006] This demonstrates the close relationship between fluid characteristics and the development of certain vascular diseases. Traditional hemodynamic analysis uses hydrodynamic methods to calculate these fluid characteristics.

[0007] It should be noted that the above introduction to the technical background is only for the purpose of providing a clear and complete explanation of the technical solution of this application and facilitating the understanding of those skilled in the art. Summary of the Invention

[0008] However, the inventors discovered that fluid dynamics-based analysis has many shortcomings and problems in practical applications:

[0009] For example, the operation is complex: before performing fluid dynamics simulation, it is necessary to accurately segment blood vessels from the data, define outlet boundary conditions and inlet boundary conditions to construct a hemodynamic model, and set the initial parameters of the solver and blood vessels. The whole process requires experienced operators to complete.

[0010] For example, computation time: fluid dynamics analysis obtains the fluid feature values ​​of each discrete pixel by solving the Navier-Stokes equations of discrete points. The whole process requires multiple iterations of optimization, so the computation time is very long. The analysis time for a single data point may be as long as several hours.

[0011] For example, poor stability: the results of fluid dynamics analysis are affected by many factors, including image quality, the quality of blood vessel segmentation, misalignment artifacts, image noise, etc. These factors can affect the determination of blood vessel boundaries, and thus affect the accuracy and repeatability of fluid feature calculation results.

[0012] To address at least one of the aforementioned technical problems, embodiments of this application provide a device and method for predicting vascular fluid characteristics, and a data segmentation device. The aim is to rapidly predict vascular fluid characteristics, with simple operation, no manual intervention required, and high stability.

[0013] According to one aspect of the embodiments of this application, a device for predicting vascular fluid characteristics is provided, comprising:

[0014] The acquisition unit acquires the medical image data to be analyzed.

[0015] The segmentation unit uses a multi-scale neural network to segment vascular medical image data from the medical image data.

[0016] A conversion unit that converts the vascular medical imaging data into surface reconstruction data to obtain discrete point data of the blood vessels; and

[0017] The prediction unit uses a fully connected neural network to predict the fluid characteristics of each discrete point in the blood vessel.

[0018] In some embodiments, the multi-scale based neural network includes a densely connected neural network having an input layer and a dense block, wherein the input layer and the dense block include multi-scale neural network modules.

[0019] In some embodiments, the multi-scale neural network module includes multiple branches and at least one merging node; wherein each branch uses one or more convolutional layers, and any two branches have different numbers of convolutional layers and / or at least some of the convolutional kernels of the convolutional layers are different; the merging node performs a cascaded combination of the processing results of the multiple branches.

[0020] In some embodiments, during the training process of the multi-scale based neural network, the training loss function includes: cross-entropy loss between the real labels and the segmentation results, and Dice similarity coefficient loss between the real labels and the segmentation results;

[0021] The formula for calculating the loss function is as follows:

[0022]

[0023] Among them, g i Let p represent the label of the i-th pixel, γ represent the proportional weight, and p represent the weight of the i-th pixel. i This represents the probability value predicted by the neural network for the i-th pixel.

[0024] In some embodiments, the segmentation unit includes:

[0025] The extraction unit extracts the axial plane data, coronal plane data, and sagittal plane data of the medical imaging data;

[0026] The input unit inputs the axial plane data, the coronal plane data, and the sagittal plane data into the corresponding multi-scale axial plane neural network, multi-scale coronal plane neural network, and multi-scale sagittal plane neural network, respectively; and

[0027] An integration unit integrates the segmentation results output by the multi-scale axial plane neural network, the multi-scale coronal plane neural network, and the multi-scale sagittal plane neural network to obtain the pixel-based vascular medical image data.

[0028] In some embodiments, the integration unit uses a majority voting strategy to integrate the segmentation results and obtain the binarized vascular medical image data.

[0029] In some embodiments, the conversion unit includes:

[0030] The reconstruction unit converts the vascular medical imaging data into surface reconstruction data.

[0031] The registration unit uses nearest-point search registration to register the surface reconstruction data to a standard template space;

[0032] An extraction unit performs an extraction operation to reduce the discrete points in the surface reconstruction data;

[0033] The sorting unit sorts the discrete points of the surface reconstruction data in a predetermined order according to their spatial positions.

[0034] In some embodiments, the fluid characteristics include one or any combination of the following: fluid pressure, wall shear force, fluid velocity, fractional blood flow reserve, and blood flow rate.

[0035] In some embodiments, the fully connected neural network includes: a shape encoding neural network, a nonlinear mapping neural network, and multiple fluid feature decoding neural networks.

[0036] In some embodiments, during the training of a shape neural network including the shape encoding neural network, the nonlinear mapping neural network, or a fluid feature neural network including the fluid feature decoding neural network, grayscale mean square error is used to train the shape neural network, the nonlinear mapping neural network, or the fluid feature neural network.

[0037] The formula for calculating the mean square error of gray levels is as follows:

[0038]

[0039] Where N represents the number of three-dimensional spatial coordinates; R i M represents the training label, which is the spatial coordinate of the i-th discrete point in the input; i This represents the spatial coordinates of the i-th discrete point output by the neural network.

[0040] According to another aspect of the embodiments of this application, a method for predicting vascular fluid characteristics is provided, comprising:

[0041] Acquire the medical image data to be analyzed;

[0042] A multi-scale neural network is used to segment vascular medical image data from the medical image data;

[0043] The vascular medical imaging data is converted into surface reconstruction data to obtain discrete point data of the blood vessels;

[0044] A fully connected neural network is used to predict the fluid characteristics of each discrete point in the blood vessel.

[0045] According to another aspect of the embodiments of this application, a data segmentation apparatus is provided, comprising:

[0046] The acquisition unit acquires the medical image data to be analyzed.

[0047] A preprocessing unit that preprocesses the medical image data;

[0048] The segmentation unit uses a multi-scale neural network to segment organ medical image data from the medical image data.

[0049] In some embodiments, the multi-scale based neural network includes a densely connected neural network having an input layer and a dense block, wherein the input layer and the dense block include multi-scale neural network modules.

[0050] In some embodiments, the multi-scale neural network module includes multiple branches and at least one merging node; wherein each branch uses one or more convolutional layers, and any two branches have different numbers of convolutional layers and / or at least some of the convolutional layers have different convolutional kernels;

[0051] The merging node performs a cascaded union of the processing results of the multiple branches.

[0052] In some embodiments, the segmentation unit includes:

[0053] The extraction unit extracts the axial plane data, coronal plane data, and sagittal plane data of the medical imaging data;

[0054] The input unit inputs the axial plane data, the coronal plane data, and the sagittal plane data into the corresponding multi-scale axial plane neural network, multi-scale coronal plane neural network, and multi-scale sagittal plane neural network, respectively; and

[0055] An integration unit integrates the segmentation results output by the multi-scale axial plane neural network, the multi-scale coronal plane neural network, and the multi-scale sagittal plane neural network to obtain pixel-based organ medical image data.

[0056] One of the beneficial effects of this application's embodiments is that it employs a multi-scale neural network to segment vascular medical image data from medical image data; converts the vascular medical image data into surface reconstruction data to obtain discrete point data of the blood vessels; and employs a fully connected neural network to predict the fluid characteristics of each discrete point of the blood vessels. Therefore, it can not only quickly predict vascular fluid characteristics, but also is simple to operate, requires no manual intervention, and is highly stable.

[0057] Referring to the following description and accompanying drawings, specific implementation methods of the embodiments of this application are disclosed in detail, indicating how the principles of the embodiments of this application can be adopted. It should be understood that the implementation methods of this application are not limited in scope. Within the spirit and scope of the appended claims, the implementation methods of this application include many changes, modifications, and equivalents. Attached Figure Description

[0058] The accompanying drawings, which form part of the specification, are used to provide a further understanding of the embodiments of this application and illustrate the implementation methods of this application, together with the textual description, to explain the principles of this application. Obviously, the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other implementation methods based on these drawings without creative effort. In the drawings:

[0059] Figure 1 This is a schematic diagram of a CT imaging device according to an embodiment of this application;

[0060] Figure 2 This is a schematic diagram of a CT imaging system according to an embodiment of this application;

[0061] Figure 3 This is a schematic diagram of a method for predicting vascular fluid characteristics according to an embodiment of this application;

[0062] Figure 4 This is a schematic diagram of a multi-scale neural network based on an embodiment of this application;

[0063] Figure 5 This is a schematic diagram of a dense block in a densely connected neural network according to an embodiment of this application;

[0064] Figure 6 This is a schematic diagram of the structure of a multi-scale neural network module according to an embodiment of this application;

[0065] Figure 7 This is a schematic diagram of vascular medical image data segmentation according to an embodiment of this application;

[0066] Figure 8 This is a schematic diagram of vascular medical image data segmentation based on multi-scale densely connected neural networks and ensemble learning, according to an embodiment of this application.

[0067] Figure 9 This is a schematic diagram of fluid characteristic prediction according to an embodiment of this application;

[0068] Figure 10 This is a schematic diagram of a shape neural network according to an embodiment of this application;

[0069] Figure 11 This is a schematic diagram of a fluid feature neural network according to an embodiment of this application;

[0070] Figure 12 This is a schematic diagram of the segmentation and prediction in an embodiment of this application;

[0071] Figure 13 This is an example diagram of a blood vessel according to an embodiment of this application;

[0072] Figure 14This is a schematic diagram of a device for predicting vascular fluid characteristics according to an embodiment of this application;

[0073] Figure 15 This is a schematic diagram of a segmentation unit according to an embodiment of this application;

[0074] Figure 16 This is a schematic diagram of a conversion unit according to an embodiment of this application;

[0075] Figure 17 This is a schematic diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0076] Referring to the accompanying drawings, the foregoing and other features of the embodiments of this application will become apparent from the following description. Specific embodiments of this application are specifically disclosed in the description and drawings, illustrating partial implementations in which the principles of the embodiments of this application can be adopted. It should be understood that this application is not limited to the described embodiments; rather, the embodiments of this application include all modifications, variations, and equivalents falling within the scope of the appended claims.

[0077] In the embodiments of this application, the terms "first," "second," etc., are used to distinguish different elements by name, but do not indicate the spatial arrangement or chronological order of these elements, and these elements should not be limited by these terms. The term "and / or" includes any one or more of the terms listed in association and all combinations thereof. The terms "comprising," "including," "having," etc., refer to the presence of the stated features, elements, components, or assemblies, but do not exclude the presence or addition of one or more other features, elements, components, or assemblies.

[0078] In the embodiments of this application, the singular forms "a," "the," etc., including the plural forms, should be broadly understood as "a kind" or "a class" rather than limited to the meaning of "an." Furthermore, the term "the" should be understood to include both the singular and plural forms, unless the context explicitly indicates otherwise. Additionally, the term "according to" should be understood as "at least partially based on…," and the term "based on" should be understood as "at least partially based on…," unless the context explicitly indicates otherwise.

[0079] Features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, combined with features in other embodiments, or substituted for features in other embodiments. The term "comprising / including" as used herein means the presence of a feature, integral, step, or component, but does not exclude the presence or addition of one or more other features, integrals, steps, or components.

[0080] The devices described herein for acquiring medical imaging data are applicable to a variety of medical imaging modalities, including but not limited to computed tomography (CT) devices, magnetic resonance imaging (MRI) devices, C-arm imaging devices, positron emission tomography (PET) devices, single photon emission computed tomography (SPECT) devices, or any other suitable medical imaging devices.

[0081] The system for acquiring medical imaging data may include the aforementioned medical imaging equipment, a separate computer device connected to the medical imaging equipment, or a computer device connected to an internet cloud, which is connected via the internet to the medical imaging equipment or a storage device for storing medical images. The imaging method may be implemented independently or in combination by the aforementioned medical imaging equipment, the computer device connected to the medical imaging equipment, and the computer device connected to the internet cloud.

[0082] Exemplary examples are described below with reference to X-ray computed tomography (CT) equipment. Those skilled in the art will understand that embodiments of this application can also be applied to other medical imaging equipment.

[0083] Figure 1 This is a schematic diagram of a CT imaging device according to an embodiment of this application, illustrating the CT imaging device 100. For example... Figure 1 As shown, the CT imaging device 100 includes a scanning gantry 101 and a patient table 102; the scanning gantry 101 has an X-ray source 103 that projects an X-ray beam toward a detector assembly or collimator 104 on the opposite side of the scanning gantry 101. The subject 105 can lie flat on the patient table 102 and move into the scanning gantry opening 106 as the patient table 102 moves; medical image data of the subject 105 can be obtained by scanning with the X-ray source 103.

[0084] Figure 2 This is a schematic diagram of a CT imaging system according to an embodiment of this application, illustrating a block diagram of the CT imaging system 200. Figure 2 As shown, the detector assembly 104 includes multiple detector units 104a and a data acquisition system (DAS) 104b. The multiple detector units 104a sense projected X-rays passing through the object being detected 105.

[0085] The DAS 104b converts the collected information into projection data for subsequent processing based on the sensing of the detector unit 104a. During the scan that acquires the X-ray projection data, the scanning gantry 101 and the components mounted thereon rotate around the rotation center 101c.

[0086] The rotation of the scanning gantry 101 and the operation of the X-ray source 103 are controlled by the control mechanism 203 of the CT imaging system 200. The control mechanism 203 includes an X-ray controller 203a that provides power and timing signals to the X-ray source 103, and a scanning gantry motor controller 203b that controls the rotational speed and position of the scanning gantry 101. The image reconstruction unit 204 receives projection data from the DAS 104b and performs image reconstruction. The reconstructed image is transmitted as input to the computer 205, which stores the image in a mass storage device 206.

[0087] Computer 205 also receives commands and scanning parameters from the operator via console 207. Console 207 has some form of operator interface, such as a keyboard, mouse, voice-activated controller, or any other suitable input device. An associated display 208 allows the operator to view reconstructed images and other data from computer 205. Commands and parameters provided by the operator are used by computer 205 to provide control signals and information to DAS 104b, X-ray controller 203a, and scanning gantry motor controller 203b. Additionally, computer 205 operates patient table motor controller 209, controlling patient table 102 to position the subject 105 and scanning gantry 101. Specifically, patient table 102 moves the subject 105, wholly or partially, through... Figure 1 The scanning rack opening is 106.

[0088] The above illustrations depict devices and systems for acquiring medical imaging data (or medical images or medical image data) according to embodiments of this application, but this application is not limited thereto. Medical imaging devices may be CT devices, MRI devices, PET devices, SPECT devices, or any other suitable imaging devices. Storage devices may be located within the medical imaging device, on a server outside the medical imaging device, in a separate medical image storage system (such as a PACS, Picture Archiving and Communication System), and / or in a remote cloud storage system.

[0089] Furthermore, medical imaging workstations can be located locally on the medical imaging equipment, meaning they are situated close to the equipment, and both can be located in the same scanning room, radiology department, or within the same hospital. Meanwhile, the medical image cloud platform analysis system can be located away from the medical imaging equipment, for example, in the cloud where it communicates with the equipment.

[0090] As an example, after a medical institution completes an imaging scan using medical imaging equipment, the scanned data is stored in a storage device. A medical imaging workstation can directly read the scanned data and perform image processing through its processor. As another example, a medical image cloud platform analysis system can remotely access medical images stored in the storage device to provide "Software as a Service" (SaaS). SaaS can exist between hospitals, between hospitals and imaging centers, or between hospitals and third-party online medical service providers.

[0091] The embodiments of this application are described in detail below.

[0092] First aspect of the embodiments

[0093] This application provides a method for predicting vascular fluid characteristics. Figure 3 This is a schematic diagram of a method for predicting vascular fluid characteristics according to an embodiment of this application, as shown below. Figure 3 As shown, the method includes:

[0094] 301. Obtain the medical image data to be analyzed;

[0095] 302. Vascular medical image data is segmented from the medical image data using a multi-scale neural network (also known as a multi-scale deep learning neural network).

[0096] 303, The vascular medical imaging data is converted into surface reconstruction data to obtain discrete point data of the blood vessels;

[0097] 304. A fully connected neural network (also known as a fully connected deep learning neural network) is used to predict the fluid characteristics of each discrete point of the blood vessel.

[0098] It is worth noting that the above appendix Figure 3 The embodiments of this application have only been illustrated schematically, and the application is not limited thereto. For example, the execution order between various operations can be appropriately adjusted, and other operations can be added or some operations can be removed. Those skilled in the art can make appropriate modifications based on the above description, and are not limited to the above-described embodiments. Figure 3 The records.

[0099] In some embodiments, the medical imaging data is medical data obtained through computed tomography (CT), magnetic resonance imaging (MR), digital subtraction angiography (DSA), or X-ray detection. However, this application is not limited to these; for example, it may also be other imaging data or image data including organs such as blood vessels.

[0100] In some embodiments, the medical image data may be preprocessed by resampling, denoising, and numerical normalization before being input into a multi-scale neural network (hereinafter referred to as a segmentation model).

[0101] For example, you can first obtain the raw image of the blood vessel to be analyzed; then, you can perform the following preprocessing on the image:

[0102] Resampling: Perform linear interpolation (e.g., 1x1x1 bilinear interpolation) on the original image;

[0103] Noise Removal: Bilateral filtering is used to remove noise from the resampled image, thereby eliminating the influence of noise factors in the image;

[0104] Numerical normalization: Performs numerical normalization on the denoised image to make its pixel values ​​fall into a standard normal distribution with a mean of 0 and a variance of 1.

[0105] The above description of preprocessing is based on resampling, denoising, and numerical normalization as examples. However, this application is not limited to these. For example, only one or any two of the resampling, denoising, and numerical normalization operations may be performed, or other preprocessing operations may be added.

[0106] Preprocessing improves the quality of data input to the segmentation model, resulting in more accurate segmentation. Furthermore, it reduces the variability between different blood vessels, enhancing the stability and generalization of the segmentation.

[0107] In some embodiments, the multi-scale based neural network includes a densely connected neural network (DenseNet) having an input layer and a dense block, wherein the input layer and the dense block include multi-scale neural network modules.

[0108] Figure 4 This is a schematic diagram of a multi-scale neural network structure according to an embodiment of this application. Figure 4As shown, for example, a multi-scale neural network module can be added to the input layer of the densely connected neural network and each dense block to extract features at different levels, thereby constructing a multi-scale based densely connected neural network (segmentation model).

[0109] As Figure 4 shown, the multi-scale based densely connected neural network includes a left branch (which can be referred to as a downsampling branch for example). The left branch can include an input layer 401, and the input layer 401 includes a multi-scale neural network module 402. In addition, the left branch further includes a convolutional layer 403, a dense block 404, and a transition layer 405. The transition layer 405 of the left branch includes Transition Down. The left branch further includes a dense block 406 and a transition layer 407. The transition layer 407 of the left branch includes Transition Down. The processing results after the dense block 404 and the dense block 406 can be concatenated respectively, as Figure 4 shown in C of

[0110] As Figure 4 shown, the multi-scale based densely connected neural network further includes a dense block 408 and a right branch (which can be referred to as an upsampling branch for example). The right branch includes a transition layer 409. The transition layer 409 of the right branch includes Transition Up. The right branch further includes a dense block 410 and a transition layer 411. The transition layer 411 of the right branch includes Transition Up. In addition, the right branch further includes a dense block 412 and a convolutional layer 413. The processing results after the transition layer 409 and the transition layer 411 can be concatenated respectively, as Figure 4 shown in C of

[0111] The appendix Figure 4 only schematically illustrates the embodiments of the present application, but the present application is not limited thereto. For example, the positions between each layer (or block) can be appropriately adjusted. In addition, some other layers (or blocks) can be added or some of the layers (or blocks) can be reduced. Those skilled in the art can make appropriate modifications according to the above content, not limited to the record of the above appendix Figure 4 only.

[0112] Figure 5 is a schematic structural diagram of a dense block in the densely connected neural network of the embodiments of the present application. As Figure 5As shown, for example, a dense block can include multiple convolutional layers, such as Figure 5 The diagram shows convolutional layers 2, 3, 4, and 5, etc.; the processing results can be concatenated (or joined), such as... Figure 5 As shown in C. (As in...) Figure 5 As shown, a multi-scale neural network module 501 can be added after the input layer of the dense block.

[0113] Appendix Figure 5 The embodiments of this application have only been illustrated schematically, and the application is not limited thereto. For example, the positions of the various layers (or modules) can be appropriately adjusted, and additional layers (or modules) can be added or some layers (or modules) can be removed. Those skilled in the art can make appropriate modifications based on the above description, and are not limited to the above-described embodiments. Figure 5 The records.

[0114] The above provides an illustrative description of densely connected neural networks and dense blocks, but the embodiments of this application are not limited thereto. Furthermore, for specific details regarding densely connected neural networks, dense blocks, transition layers, cascaded unions, and convolutional layers, please refer to relevant technologies, which will not be elaborated upon here.

[0115] In some embodiments, the multi-scale neural network module includes multiple branches; wherein each branch uses one or more convolutional layers, and at least some of the convolutional layers in any two branches have different convolutional kernels and / or different numbers of convolutional layers. The multi-scale neural network module also includes at least one merging node, which performs a cascaded combination of the processing results of the multiple branches.

[0116] Figure 6 This is a schematic diagram of the structure of a multi-scale neural network module according to an embodiment of this application. Figure 6 As shown, for example, this multi-scale neural network module may include four branches. Branch 1 includes three convolutional layers, using 3*3, 3*3 and 1*1 convolutional kernels respectively; Branch 2 includes two convolutional layers, using 3*3 and 1*1 convolutional kernels respectively; Branch 3 includes a pooling layer and a convolutional layer using a 1*1 convolutional kernel; Branch 4 uses a convolutional layer using a 1*1 convolutional kernel.

[0117] It is worth noting that, Figure 6 The multi-scale neural network module of the embodiments of this application has been illustrated, but the application is not limited thereto. For example, more branches can be used, or only two or three branches can be used; the convolution kernel is not limited to 1*1 and 3*3, for example, 5*5, 7*7, etc. can also be used.

[0118] Therefore, by using a multi-scale densely connected neural network (segmentation model), vascular medical image data can be segmented quickly and accurately.

[0119] The above illustration demonstrates a multi-scale densely connected neural network (segmentation model). The following section will further explain the segmentation operation of vascular medical image data.

[0120] Figure 7 This is a schematic diagram of vascular medical image data segmentation according to an embodiment of this application, as shown below. Figure 7 As shown, in some embodiments, vascular medical image data segmentation may include:

[0121] 701. Extract axial (also known as horizontal or transverse) plane data, coronal plane data, and sagittal plane data from medical imaging data. For details on the axial, coronal, and sagittal planes, and how to extract axial, coronal, and sagittal plane data, please refer to relevant technologies.

[0122] 702, The axial plane data, the coronal plane data, and the sagittal plane data are respectively input into the corresponding multi-scale axial plane neural network, multi-scale coronal plane neural network, and multi-scale sagittal plane neural network.

[0123] For example, three multi-scale densely connected neural networks can be trained and used: a multi-scale axial plane neural network, a multi-scale coronal plane neural network, and a multi-scale sagittal plane neural network. The structures of these three neural networks can be as follows: Figures 4 to 6 As shown, by inputting the 3D images of the axial, coronal, and sagittal planes into their respective multi-scale densely connected neural networks for feature extraction and segmentation, coarse segmentation results of blood vessels at different cross-sections can be obtained.

[0124] 703, The segmentation results output by the multi-scale axial plane neural network, the multi-scale coronal plane neural network, and the multi-scale sagittal plane neural network are integrated to obtain the vascular medical image data based on pixels.

[0125] Figure 8 This is a schematic diagram of vascular medical image data segmentation based on multi-scale densely connected neural networks and ensemble learning, according to an embodiment of this application.

[0126] For example, ensemble learning can be used to integrate and optimize coarse segmentation results of blood vessels from different cross-sections to obtain the final segmentation result. The idea behind ensemble learning is to first construct multiple base classifiers according to certain rules, then combine them using a certain ensemble strategy, and finally output the final result through comprehensive judgment.

[0127] In some embodiments, a majority voting strategy can be used to integrate the segmentation results and obtain binarized vascular medical image data. For example, for the same pixel, if two or more segmentation results classify it as a blood vessel, the final segmentation result for that pixel will be classified as a blood vessel; otherwise, it will be classified as a non-blood vessel.

[0128] The above example uses the majority voting strategy for integration, but this application is not limited to this. For example, other ensemble learning methods such as the knapsack method can also be used.

[0129] In some embodiments, during the training process of a multi-scale neural network (a multi-scale densely connected neural network), the training loss function may include two parts: the cross-entropy loss between the true labels and the segmentation results, and the Dice similarity coefficient loss between the true labels and the segmentation results.

[0130] For example, the formula for calculating the loss function is as follows:

[0131]

[0132] Among them, g i Let p represent the label of the i-th pixel, γ represent the proportional weight, and p represent the weight of the i-th pixel. i This represents the probability value predicted by the neural network for the i-th pixel. Here, γ is a value between 0 and 1, for example, it can be set to 0.2, 0.3, 0.5, 0.7, etc.

[0133] The above illustrations demonstrate the segmentation of vascular medical image data. The following sections will explain the transformation operations and fluid feature prediction.

[0134] In some embodiments, post-processing operations are performed on the segmented vascular medical image data, including:

[0135] - Surface reconstruction: Converting vascular medical imaging data into surface reconstruction data;

[0136] -ICP Registration: The surface reconstruction data is registered to the standard template space using Iterative Closest Point (ICP) registration.

[0137] -Decimate operation: Performs a decimate operation to reduce the discrete points in the surface reconstruction data;

[0138] - Discrete point sorting: Sort the discrete points of the surface reconstruction data in a predetermined order (e.g., from top to bottom, from left to right) according to their spatial location.

[0139] The above explanation only uses ICP registration and Decimate operations as examples to illustrate post-processing, but this application is not limited to these. For example, only one or any two of the above operations can be performed, or other post-processing operations can be added. Furthermore, for specific details regarding surface reconstruction data, ICP registration, and Decimate operations, please refer to relevant technologies.

[0140] Post-processing can improve the quality of data input into the prediction model, making the prediction results more accurate. Furthermore, it can reduce the variability between different blood vessels, improving the stability and generalization of the prediction.

[0141] In some embodiments, the fluid characteristics include one or any combination of the following: fluid pressure (also referred to as pressure), wall shear force (WSS), fluid velocity (also referred to as flow rate), fractional flow reserve (FFR), and blood flow rate. However, this application is not limited to these, and other fluid characteristics may also be included.

[0142] In some embodiments, the fully connected neural network (also referred to as the prediction model) includes: a shape encoding neural network, a nonlinear mapping neural network, and multiple fluid feature decoding neural networks. For example, the multiple fluid feature decoding neural networks include: a fluid pressure decoding neural network, a wall shear force decoding neural network, and a fluid velocity decoding neural network.

[0143] For example, a shape-encoding neural network uses fully connected layers for shape encoding, a nonlinear mapping neural network uses fully connected layers for nonlinear mapping, and a fluid feature decoding neural network uses fully connected layers for fluid feature decoding. For more details on fully connected layers, shape encoding, nonlinear mapping, and fluid feature decoding, please refer to relevant technical resources.

[0144] The following will use fluid pressure, wall shear force, and fluid velocity as examples to illustrate the prediction of fluid characteristics. The three-dimensional spatial coordinates of the sorted discrete points can be extracted and input into a fully connected fluid feature neural network to predict fluid feature values ​​such as pressure, wall shear force, and flow velocity.

[0145] Figure 9 This is a schematic diagram of fluid characteristic prediction according to an embodiment of this application. Figure 9 As shown, the shape encoding neural network performs feature dimensionality reduction on the three-dimensional coordinates of discrete points. For example, this shape encoding neural network is input with the sorted three-dimensional coordinates of 5000 discrete points (e.g., ...). Figure 9 As shown, there are a total of 15,000 data points. After encoding through two fully connected layers, the dimensions are reduced to obtain 16 shape codes.

[0146] like Figure 9As shown, the nonlinear mapping neural network maps 16 shape codes to a fluid feature coding space. For example, this nonlinear mapping neural network takes the 16 shape codes as input, and after mapping through three fully connected layers, it obtains 48 fluid feature codes, which are 16 pressure codes, 16 flow velocity codes, and 16 wall shear stress codes.

[0147] like Figure 9 As shown, the pressure decoding neural network decodes the pressure codes to obtain the pressure value for each discrete point. For example, this pressure decoding neural network takes 16 pressure codes as input and outputs 5000 discrete pressure values, which correspond to the pressure values ​​of the 5000 input discrete points.

[0148] like Figure 9 As shown, the wall shear force decoding neural network decodes the wall shear force code to obtain the wall shear force value at each discrete point. For example, this wall shear force decoding neural network takes 16 wall shear force codes as input and outputs 5000 discrete wall shear force values, which correspond to the wall shear force values ​​at the 5000 input discrete points.

[0149] like Figure 9 As shown, the flow rate decoding neural network decodes the flow rate codes to obtain the flow rate value for each discrete point. For example, this flow rate decoding neural network takes 16 flow rate codes as input and outputs the flow rate values ​​for 5000 discrete points, which correspond to the flow rate values ​​of the 5000 input discrete points.

[0150] Therefore, by using a fully connected neural network (prediction model), vascular fluid characteristics can be predicted quickly, and the operation is simple and requires no manual intervention.

[0151] The training of fully connected neural networks will be explained below.

[0152] For example, a shape encoding neural network is a part of a shape neural network, which consists of two parts: a shape encoding neural network and a shape decoding neural network. During training, the entire shape neural network, including the shape encoding neural network and the shape decoding neural network, is trained.

[0153] Similarly, the fluid feature (pressure, velocity, or wall shear force) neural network also comprises two parts: a fluid feature (pressure, velocity, or wall shear force) encoding neural network and a fluid feature (pressure, velocity, or wall shear force) decoding neural network. During training, the entire fluid feature (pressure, velocity, or wall shear force) neural network, including the fluid feature (pressure, velocity, or wall shear force) encoding neural network and the fluid feature (pressure, velocity, or wall shear force) decoding neural network, is trained.

[0154] Figure 10 This is a schematic diagram of a shape neural network according to an embodiment of this application. Figure 10 As shown, the shape neural network is input with the three-dimensional spatial coordinates of 5000 discrete points (e.g., Figure 10 As shown, there are a total of 15,000 data points. These are processed by a shape encoding neural network to obtain 16 shape-encoded values, and then by a shape decoding neural network to obtain the three-dimensional spatial coordinates of 5,000 discrete points (e.g., ...). Figure 10 As shown, there are a total of 15,000 data points.

[0155] Figure 11 This is a schematic diagram of a neural network for fluid characteristics (pressure, velocity, or wall shear force) according to an embodiment of this application. Figure 11 As shown, the fluid feature (pressure, velocity, or wall shear force) neural network is input with 5000 data points, which are then processed by the fluid feature (pressure, velocity, or wall shear force) encoding neural network to obtain 16 fluid feature (pressure, velocity, or wall shear force) encoded values, and then processed by the fluid feature (pressure, velocity, or wall shear force) decoding neural network to obtain 5000 data points.

[0156] During training, the gray-level mean squared error is used to train the network. The formula for calculating the gray-level mean squared error is as follows:

[0157]

[0158] Where N represents the number of three-dimensional spatial coordinates; R i M represents the training label, which is the spatial coordinate of the i-th discrete point in the input; i This represents the spatial coordinates of the i-th discrete point output by the neural network.

[0159] For example, after training the shape neural network, pressure neural network, flow velocity neural network, and wall shear force neural network separately, the four parts—shape encoding neural network, pressure decoding neural network, flow velocity decoding neural network, and wall shear force neural network—are retained, and then the gray-level mean square error is used to train the nonlinear mapping neural network. The trained shape encoding neural network, pressure decoding neural network, flow velocity decoding neural network, wall shear force neural network, and nonlinear mapping neural network can be used as prediction models to predict the fluid characteristics of blood vessels.

[0160] The embodiments of this application have been illustrated above. The following examples will further illustrate the embodiments.

[0161] Figure 12 This is a schematic diagram of the segmentation and prediction in an embodiment of this application, as shown below. Figure 12 As shown, after acquiring the medical image data to be analyzed, preprocessing operations (including resampling, noise reduction, and numerical normalization) can be performed; then, the vascular medical image data can be automatically segmented using a multi-scale densely connected neural network; after that, post-processing (including surface reconstruction, ICP registration, Decimate, and discrete point sorting) can be performed; and then, based on a fully connected neural network, the fluid characteristics of the blood vessels can be automatically predicted.

[0162] Figure 13 This is an example diagram of a blood vessel according to an embodiment of this application, such as... Figure 13 As shown on the left, vascular medical image data can be automatically segmented using a multi-scale densely connected neural network; such as... Figure 13 As shown on the right, the fluid characteristics of blood vessels can be automatically predicted based on a fully connected neural network. Compared to the left part, the right part, after adding fluid characteristic data, can intuitively display the fluid characteristics of this segment of blood vessel.

[0163] The above embodiments are merely illustrative examples of embodiments of this application, but this application is not limited thereto, and appropriate modifications can be made based on the above embodiments. For example, the above embodiments can be used alone, or one or more of the above embodiments can be combined.

[0164] As can be seen from the above embodiments, a multi-scale neural network is used to segment vascular medical image data from medical image data; the vascular medical image data is converted into surface reconstruction data to obtain discrete point data of the blood vessels; and a fully connected neural network is used to predict the fluid characteristics of each discrete point of the blood vessels. Therefore, it can not only quickly predict vascular fluid characteristics, but also is simple to operate, requires no manual intervention, and has strong stability.

[0165] Second aspect of the embodiments

[0166] This application provides a device for predicting vascular fluid characteristics, and the same content as the first aspect of the embodiment will not be repeated.

[0167] Figure 14 This is a schematic diagram of a device for predicting vascular fluid characteristics according to an embodiment of this application. Figure 14 As shown, the vascular fluid characteristic prediction device 1400 includes:

[0168] Acquisition unit 1401 acquires medical image data to be analyzed;

[0169] The segmentation unit 1402 uses a multi-scale neural network to segment vascular medical image data from the medical image data.

[0170] The conversion unit 1403 converts the vascular medical imaging data into surface reconstruction data to obtain discrete point data of the blood vessels; and

[0171] Prediction unit 1404 uses a fully connected neural network to predict the fluid characteristics of each discrete point in the blood vessel.

[0172] In some embodiments, the medical imaging data is medical data obtained through computed tomography, magnetic resonance imaging, digital subtraction angiography, or X-ray detection.

[0173] In some embodiments, such as Figure 14 As shown, the device 1400 further includes:

[0174] The preprocessing unit 1405 performs preprocessing on the detected image data, including resampling, noise reduction, and numerical normalization, before inputting the medical image data into the multi-scale neural network.

[0175] In some embodiments, the multi-scale based neural network includes a densely connected neural network having an input layer and a dense block, wherein the input layer and the dense block include multi-scale neural network modules.

[0176] In some embodiments, the multi-scale neural network module includes multiple branches and at least one merging node; wherein each branch uses one or more convolutional layers, and any two branches have different numbers of convolutional layers and / or at least some of the convolutional layers have different convolutional kernels;

[0177] The merging node performs a cascaded union of the processing results of the multiple branches.

[0178] In some embodiments, during the training process of the multi-scale based neural network, the training loss function includes: cross-entropy loss between the real labels and the segmentation results, and Dice similarity coefficient loss between the real labels and the segmentation results;

[0179] The formula for calculating the loss function is as follows:

[0180]

[0181] Among them, g i Let p represent the label of the i-th pixel, γ represent the proportional weight, and p represent the weight of the i-th pixel. i This represents the probability value predicted by the neural network for the i-th pixel.

[0182] Figure 15 This is a schematic diagram of a segmentation unit according to an embodiment of this application, as shown below. Figure 15 As shown, the segmentation unit 1402 includes:

[0183] Extraction unit 1501 extracts axial plane data, coronal plane data and sagittal plane data of the medical image data;

[0184] Input unit 1502 inputs the axial plane data, the coronal plane data, and the sagittal plane data into corresponding multi-scale axial plane neural networks, multi-scale coronal plane neural networks, and multi-scale sagittal plane neural networks, respectively; and

[0185] The integration unit 1503 integrates the segmentation results output by the multi-scale axial plane neural network, the multi-scale coronal plane neural network, and the multi-scale sagittal plane neural network to obtain the vascular medical image data based on pixels.

[0186] In some embodiments, the integration unit 1503 uses a majority voting strategy to integrate the segmentation results and obtain binarized vascular medical image data.

[0187] Figure 16 This is a schematic diagram of a conversion unit according to an embodiment of this application, as shown below. Figure 16 As shown, the conversion unit 1403 includes:

[0188] Reconstruction unit 1601 converts the vascular medical imaging data into surface reconstruction data;

[0189] Registration unit 1602 uses nearest-point search registration to register the surface reconstruction data to a standard template space;

[0190] Extraction unit 1603 performs extraction operations to reduce the discrete points of the surface reconstruction data;

[0191] The sorting unit 1604 sorts the discrete points of the surface reconstruction data in a predetermined order according to their spatial positions.

[0192] In some embodiments, the fluid characteristics include one or any combination of the following: fluid pressure, wall shear force, fluid velocity, fractional blood flow reserve, and blood flow rate.

[0193] In some embodiments, the fully connected neural network includes: a shape encoding neural network, a nonlinear mapping neural network, and multiple fluid feature decoding neural networks.

[0194] In some embodiments, during the training of a shape neural network including the shape encoding neural network, the nonlinear mapping neural network, or a fluid feature neural network including the fluid feature decoding neural network, gray-level mean square error is used to train the shape neural model, the nonlinear mapping neural network, or the fluid feature neural network.

[0195] The formula for calculating the mean square error of gray levels is as follows:

[0196]

[0197] Where N represents the number of three-dimensional spatial coordinates; R i M represents the training label, which is the spatial coordinate of the i-th discrete point in the input; i This represents the spatial coordinates of the i-th discrete point output by the neural network.

[0198] For the sake of simplicity, Figures 14 to 16 The diagram only exemplifies the connection relationships or signal flow between various components or modules; however, those skilled in the art should understand that various related technologies, such as bus connections, can be employed. The aforementioned components or modules can be implemented using hardware facilities such as processors and memory; this application does not limit the scope of the embodiments.

[0199] The above embodiments are merely illustrative examples of embodiments of this application, but this application is not limited thereto, and appropriate modifications can be made based on the above embodiments. For example, the above embodiments can be used alone, or one or more of the above embodiments can be combined.

[0200] As can be seen from the above embodiments, a multi-scale neural network is used to segment vascular medical image data from medical image data; the vascular medical image data is converted into surface reconstruction data to obtain discrete point data of the blood vessels; and a fully connected neural network is used to predict the fluid characteristics of each discrete point of the blood vessels. Therefore, it can not only quickly predict vascular fluid characteristics, but also is simple to operate, requires no manual intervention, and has strong stability.

[0201] Third aspect of the embodiments

[0202] This application provides a data segmentation method and apparatus, and the contents that are the same as those in the first and second aspects will not be repeated. The detection objects of this application include, but are not limited to, human patients and animals, and the organs are not limited to the aforementioned blood vessels, but may include, but are not limited to, the brain, liver, heart, lungs, pancreas, and bones, etc.

[0203] In some embodiments, medical image data to be analyzed is acquired; the medical image data is preprocessed; and organ medical image data is segmented from the medical image data using a multi-scale neural network.

[0204] In some embodiments, the medical imaging data is medical data obtained through computed tomography (CT), magnetic resonance imaging (MR), digital subtraction angiography (DSA), or X-ray detection.

[0205] In some embodiments, the preprocessing includes resampling, noise reduction, and numerical normalization.

[0206] In some embodiments, the multi-scale based neural network includes a densely connected neural network having an input layer and a dense block, wherein the input layer and the dense block include multi-scale neural network modules.

[0207] In some embodiments, the multi-scale neural network module includes multiple branches; wherein each branch uses one or more convolutional layers, and at least some of the convolutional layers in any two branches have different convolutional kernels and / or different numbers of convolutional layers.

[0208] In some embodiments, the multi-scale neural network module further includes at least one merging node, which performs a cascaded combination of the processing results of the multiple branches.

[0209] In some embodiments, during the training process of the multi-scale based neural network, the training loss function includes: the cross-entropy loss between the real labels and the segmentation results, and the Dice similarity coefficient loss between the real labels and the segmentation results.

[0210] In some embodiments, the loss function is calculated using the following formula:

[0211]

[0212] Among them, g i Let p represent the label of the i-th pixel, γ represent the proportional weight, and p represent the weight of the i-th pixel. i This represents the probability value predicted by the neural network for the i-th pixel.

[0213] In some embodiments, the segmentation includes: extracting axial plane data, coronal plane data, and sagittal plane data from the medical image data; inputting the axial plane data, coronal plane data, and sagittal plane data into corresponding multi-scale axial plane neural networks, multi-scale coronal plane neural networks, and multi-scale sagittal plane neural networks, respectively; and integrating the segmentation results output by the multi-scale axial plane neural network, the multi-scale coronal plane neural network, and the multi-scale sagittal plane neural network to obtain pixel-based organ medical image data.

[0214] In some embodiments, a majority voting strategy is used to integrate the segmentation results and obtain binarized organ medical image data.

[0215] The above embodiments are merely illustrative examples of embodiments of this application, but this application is not limited thereto, and appropriate modifications can be made based on the above embodiments. For example, the above embodiments can be used alone, or one or more of the above embodiments can be combined.

[0216] As demonstrated in the above embodiments, organ medical image data can be segmented from medical image data using a multi-scale neural network. This not only enables rapid segmentation of organ medical image data but also offers advantages such as simple operation, no manual intervention required, and high stability.

[0217] Fourth aspect of the embodiment

[0218] This application provides an electronic device including a vascular fluid characteristic prediction device 1400 as described in the second aspect of the embodiment, the contents of which are incorporated herein by reference. This electronic device may be, for example, a computer, server, workstation, laptop computer, smartphone, etc.; however, this application is not limited thereto.

[0219] Figure 17 This is a schematic diagram of an electronic device according to an embodiment of this application. For example... Figure 17 As shown, the electronic device 1700 may include one or more processors (e.g., a central processing unit, CPU) 1710 and one or more memories 1720; the memories 1720 are coupled to the processors 1710. The memories 1720 may store various types of data; in addition, they may store a program 1721 for information processing, and execute the program 1721 under the control of the processors 1710.

[0220] In some embodiments, the functionality of the vascular fluid feature prediction device 1400 is integrated into the processor 1710. The processor 1710 is configured to implement the vascular fluid feature prediction method as described in the first aspect embodiment.

[0221] In some embodiments, the vascular fluid feature prediction device 1400 is configured separately from the processor 1710. For example, the vascular fluid feature prediction device 1400 can be configured as a chip connected to the processor 1710, and the function of the vascular fluid feature prediction device 1400 can be realized through the control of the processor 1710.

[0222] For example, the processor 1710 is configured to perform the following control: acquire medical image data to be analyzed; segment vascular medical image data from the medical image data using a multi-scale neural network; convert the vascular medical image data into surface reconstruction data to obtain discrete point data of the blood vessels; and predict the fluid characteristics of each discrete point of the blood vessels using a fully connected neural network.

[0223] For example, the processor 1710 is configured to perform the following control: acquire medical image data to be analyzed; preprocess the medical image data; and segment organ medical image data from the medical image data using a multi-scale neural network.

[0224] In addition, such as Figure 17 As shown, the electronic device 1700 may also include: an input / output (I / O) device 1730 and a display 1740, etc.; the functions of the above components are similar to those in the prior art, and will not be described in detail here. It is worth noting that the electronic device 1700 is not necessarily required to include... Figure 17 All components shown; in addition, the electronic device 1700 may also include Figure 17 For components not shown, please refer to relevant technologies.

[0225] This application also provides a computer-readable program, wherein when the program is executed in an electronic device, the program causes the computer in the electronic device to perform a method for predicting vascular fluid characteristics as described in the first aspect embodiment or a data segmentation method as described in the third aspect embodiment.

[0226] This application also provides a storage medium storing a computer-readable program, wherein the computer-readable program causes a computer in an electronic device to perform a method for predicting vascular fluid characteristics as described in the first aspect embodiment or a data segmentation method as described in the third aspect embodiment.

[0227] The apparatus and methods described above in this application can be implemented in hardware or in combination with software. This application relates to a computer-readable program that, when executed by a logic component, enables the logic component to implement the apparatus or components described above, or to implement the various methods or steps described above. This application also relates to storage media for storing the above programs, such as hard disks, magnetic disks, optical disks, DVDs, flash memory, etc.

[0228] The methods / apparatus described in conjunction with the embodiments of this application can be directly embodied in hardware, software modules executed by a processor, or a combination of both. For example, one or more and / or combinations of one or more functional block diagrams shown in the figures can correspond to various software modules in a computer program flow, or to various hardware modules. These software modules can correspond to the various steps shown in the figures, respectively. These hardware modules can be implemented, for example, using a field-programmable gate array (FPGA) to embed these software modules.

[0229] The software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. A storage medium can be coupled to the processor, enabling the processor to read information from and write information to the storage medium; or the storage medium can be an integral part of the processor. The processor and storage medium can reside in an ASIC. The software module can be stored in the memory of a mobile terminal or in a memory card that can be inserted into the mobile terminal. For example, if the device (such as a mobile terminal) uses a high-capacity MEGA-SIM card or a high-capacity flash memory device, the software module can be stored in the MEGA-SIM card or the high-capacity flash memory device.

[0230] One or more and / or one or more combinations of functional blocks described in the accompanying drawings can be implemented as a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, or any suitable combination thereof for performing the functions described herein. One or more and / or one or more combinations of functional blocks described in the accompanying drawings can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in communication with a DSP, or any other such configuration.

[0231] The present application has been described above with reference to specific embodiments. However, those skilled in the art should understand that these descriptions are exemplary and not intended to limit the scope of protection of the present application. Those skilled in the art can make various modifications and variations to the present application based on the principles thereof, and these modifications and variations are also within the scope of the present application.

[0232] Regarding the implementation methods including the above embodiments, the following notes are also disclosed:

[0233] Appendix 1. A method for predicting vascular fluid characteristics, comprising:

[0234] Acquire the medical image data to be analyzed;

[0235] A multi-scale neural network is used to segment vascular medical image data from the medical image data;

[0236] The vascular medical imaging data is converted into surface reconstruction data to obtain discrete point data of the blood vessels; and

[0237] A fully connected neural network is used to predict the fluid characteristics of each discrete point in the blood vessel.

[0238] Note 2. According to the method described in Note 1, the medical imaging data is medical data obtained by computed tomography (CT), magnetic resonance (MR), digital subtraction angiography (DSA), or X-ray detection.

[0239] Appendix 3. The method according to Appendix 1 or 2, wherein the method further comprises:

[0240] Before inputting the medical image data into the multi-scale neural network, the medical image data undergoes preprocessing including resampling, noise reduction, and numerical normalization.

[0241] Appendix 4. The method according to any one of Appendices 1 to 3, wherein the multi-scale based neural network comprises a densely connected neural network (DenseNet) having an input layer and a dense block, wherein the input layer and the dense block include multi-scale neural network modules.

[0242] Note 5. According to the method described in Note 4, the multi-scale neural network module includes multiple branches; wherein each branch uses one or more convolutional layers, and at least some of the convolutional layers in any two branches have different convolutional kernels and / or different numbers of convolutional layers.

[0243] Note 6. According to the method described in Note 5, the multi-scale neural network module further includes at least one merging node, which performs a cascaded union (or splicing) of the processing results of the multiple branches.

[0244] Note 7. The method according to any one of Notes 1 to 6, wherein, during the training process of the multi-scale based neural network, the training loss function includes: cross-entropy loss between the true labels and the segmentation results, and Dice similarity coefficient loss between the true labels and the segmentation results.

[0245] Appendix 8. According to the method described in Appendix 7, the formula for calculating the loss function is as follows:

[0246]

[0247] Among them, g i Let p represent the label of the i-th pixel, γ represent the proportional weight, and p represent the weight of the i-th pixel. i This represents the probability value predicted by the neural network for the i-th pixel.

[0248] Appendix 9. The method according to any one of Appendices 1 to 8, wherein the segmentation comprises:

[0249] Extract the axial, coronal, and sagittal plane data from the medical imaging data;

[0250] The axial plane data, the coronal plane data, and the sagittal plane data are respectively input into the corresponding multi-scale axial plane neural network, multi-scale coronal plane neural network, and multi-scale sagittal plane neural network; and

[0251] The segmentation results output by the multi-scale axial plane neural network, the multi-scale coronal plane neural network, and the multi-scale sagittal plane neural network are integrated to obtain the vascular medical image data based on pixels.

[0252] Note 10. The method according to Note 9, wherein the segmentation results are integrated using a majority voting strategy to obtain the binarized vascular medical image data.

[0253] Note 11. The method according to any one of Notes 1 to 10, wherein the conversion comprises:

[0254] The vascular medical imaging data is converted into surface reconstruction data;

[0255] The surface reconstruction data was registered to the standard template space using nearest-neighbor (ICP) registration.

[0256] Perform a decimate operation to reduce the discrete points in the surface reconstruction data; and

[0257] The discrete points of the surface reconstruction data are sorted by spatial location according to a predetermined order.

[0258] Note 12. The method according to any one of Notes 1 to 11, wherein the fluid characteristics include one or any combination of the following: fluid pressure, wall shear force (WSS), fluid velocity, fractional flow reserve (FFR), and blood flow rate.

[0259] Note 13. The method according to any one of Notes 1 to 12, wherein the fully connected neural network comprises: a shape encoding neural network, a nonlinear mapping neural network, and multiple fluid feature decoding neural networks.

[0260] Note 14. According to the method described in Note 13, the plurality of fluid feature decoding neural networks include: a fluid pressure decoding neural network, a wall shear force decoding neural network, and a fluid velocity decoding neural network.

[0261] Note 15. According to the method described in Note 13, wherein, during the training of the shape neural network including the shape encoding neural network, the nonlinear mapping neural network, and the fluid feature neural network including the fluid feature decoding neural network, the gray-level mean square error is used to train the shape neural network, the nonlinear mapping neural network, or the fluid feature neural network.

[0262] Appendix 16. According to the method described in Appendix 15, the formula for calculating the mean square error of gray levels is as follows:

[0263]

[0264] Where N represents the number of three-dimensional spatial coordinates; R i M represents the training label, which is the spatial coordinate of the i-th discrete point in the input; i This represents the spatial coordinates of the i-th discrete point output by the neural network.

[0265] Appendix 17. A data partitioning method, comprising:

[0266] Acquire the medical image data to be analyzed;

[0267] Preprocessing the medical image data; and

[0268] Organ medical image data is segmented from the medical image data using a multi-scale neural network.

[0269] Note 18. The method according to Note 17, wherein the medical imaging data is medical data obtained by computed tomography (CT), magnetic resonance (MR), digital subtraction angiography (DSA), or X-ray detection.

[0270] Note 19. The method according to Note 17 or 18, wherein the preprocessing includes: resampling, denoising, and numerical normalization.

[0271] Note 20. The method according to any one of Notes 17 to 19, wherein the multi-scale based neural network comprises a densely connected neural network (DenseNet) having an input layer and a dense block, wherein the input layer and the dense block include multi-scale neural network modules.

[0272] Note 21. According to the method described in Note 20, the multi-scale neural network module includes multiple branches; wherein each branch uses one or more convolutional layers, and at least some of the convolutional layers in any two branches have different convolutional kernels and / or different numbers of convolutional layers.

[0273] Note 22. According to the method described in Note 21, the multi-scale neural network module further includes at least one merging node, which performs a cascaded union (or splicing) of the processing results of the multiple branches.

[0274] Note 23. The method according to any one of Notes 17 to 22, wherein, during the training process of the multi-scale based neural network, the training loss function includes: cross-entropy loss between the true labels and the segmentation results, and Dice similarity coefficient loss between the true labels and the segmentation results.

[0275] Appendix 24. According to the method described in Appendix 23, the formula for calculating the loss function is as follows:

[0276]

[0277] Among them, g i Let p represent the label of the i-th pixel, γ represent the proportional weight, and p represent the weight of the i-th pixel. i This represents the probability value predicted by the neural network for the i-th pixel.

[0278] Note 25. The method according to any one of Notes 17 to 24, wherein the segmentation comprises:

[0279] Extract the axial, coronal, and sagittal plane data from the medical imaging data;

[0280] The axial plane data, the coronal plane data, and the sagittal plane data are respectively input into the corresponding multi-scale axial plane neural network, multi-scale coronal plane neural network, and multi-scale sagittal plane neural network; and

[0281] The segmentation results output by the multi-scale axial plane neural network, the multi-scale coronal plane neural network, and the multi-scale sagittal plane neural network are integrated to obtain pixel-based medical image data of the organ.

[0282] Note 26. The method according to Note 25, wherein the segmentation results are integrated using a majority voting strategy to obtain binarized organ medical image data.

[0283] Appendix 27. An electronic device comprising a memory and a processor, the memory storing a computer program, the processor being configured to execute the computer program to implement a method for predicting vascular fluid characteristics as described in any one of Appendices 1 to 16, or a data segmentation method as described in any one of Appendices 17 to 26.

[0284] Note 28. A storage medium storing a computer-readable program, wherein the computer-readable program causes a computer in an electronic device to perform a method for predicting vascular fluid characteristics as described in any one of Notes 1 to 16, or a data segmentation method as described in any one of Notes 17 to 26.

Claims

1. A device for predicting vascular fluid characteristics, characterized in that, The device includes: The acquisition unit acquires the medical image data to be analyzed. The segmentation unit uses a multi-scale neural network to segment vascular medical image data, including blood vessels with different cross-sections, from the medical image data. A conversion unit converts the vascular medical imaging data into surface reconstruction data and sorts them by spatial location to obtain discrete point data of the blood vessels; and The prediction unit extracts the three-dimensional spatial coordinates of the sorted discrete points and inputs them into a fully connected neural network to predict the fluid characteristics of each discrete point of the blood vessel. The fully connected neural network includes a shape encoding neural network, a nonlinear mapping neural network, and multiple fluid feature decoding neural networks.

2. The apparatus according to claim 1, characterized in that, The multi-scale based neural network includes a densely connected neural network with an input layer and dense blocks, wherein the input layer and the dense blocks include multi-scale neural network modules.

3. The apparatus according to claim 2, characterized in that, The multi-scale neural network module includes multiple branches and at least one merging node; wherein each branch uses one or more convolutional layers, and any two branches have different numbers of convolutional layers and / or at least some of the convolutional kernels of the convolutional layers are different; The merging node performs a cascaded union of the processing results of the multiple branches.

4. The apparatus according to claim 1, characterized in that, During the training process of the multi-scale neural network, the training loss function includes: cross-entropy loss between the true label and the segmentation result, and Dice similarity coefficient loss between the true label and the segmentation result; The formula for calculating the loss function is as follows: in, Indicates the first i A label of 1 pixel, Indicates proportional weight. This indicates that the neural network predicts the first... i The probability value of each pixel.

5. The apparatus according to claim 1, characterized in that, The segmentation unit includes: The extraction unit extracts the axial plane data, coronal plane data, and sagittal plane data of the medical imaging data; The input unit inputs the axial plane data, the coronal plane data, and the sagittal plane data into the corresponding multi-scale axial plane neural network, multi-scale coronal plane neural network, and multi-scale sagittal plane neural network, respectively; and An integration unit integrates the segmentation results output by the multi-scale axial plane neural network, the multi-scale coronal plane neural network, and the multi-scale sagittal plane neural network to obtain the pixel-based vascular medical image data.

6. The apparatus according to claim 5, characterized in that, The integration unit uses a majority voting strategy to integrate the segmentation results and obtain the binarized vascular medical image data.

7. The apparatus according to claim 1, characterized in that, The conversion unit includes: The reconstruction unit converts the vascular medical imaging data into surface reconstruction data. The registration unit uses nearest-point search registration to register the surface reconstruction data to a standard template space; An extraction unit performs an extraction operation to reduce the discrete points in the surface reconstruction data; The sorting unit sorts the discrete points of the surface reconstruction data in a predetermined order according to their spatial positions.

8. The apparatus according to claim 1, characterized in that, The fluid characteristics include one or any combination of the following: fluid pressure, wall shear force, fluid velocity, fractional blood flow reserve, and blood flow rate.

9. The apparatus according to claim 1, characterized in that, During the training process of the shape neural network including the shape encoding neural network, the nonlinear mapping neural network, and the fluid feature neural network including the fluid feature decoding neural network, the gray-level mean square error is used to train the shape neural network, the nonlinear mapping neural network, or the fluid feature neural network. The formula for calculating the mean square error of gray levels is as follows: Where N represents the number of three-dimensional spatial coordinates; Represents the training label, which is the input label of the first generation. i Spatial coordinates of discrete points; The first term represents the output of the neural network. i Spatial coordinates of discrete points.

10. A method for predicting vascular fluid characteristics, characterized in that, The method includes: Acquire the medical image data to be analyzed; A multi-scale neural network is used to segment vascular medical image data, including blood vessels with different cross-sections, from the medical image data. The vascular medical imaging data is converted into surface reconstruction data and sorted by spatial location to obtain discrete point data of the blood vessels; and The three-dimensional spatial coordinates of the sorted discrete points are extracted and input into a fully connected neural network to predict the fluid characteristics of each discrete point of the blood vessel. The fully connected neural network includes a shape encoding neural network, a nonlinear mapping neural network, and multiple fluid feature decoding neural networks.

Images