A real-time measurement method of cardiac outflow tract wall shear stress based on neural network

Abstract

The application belongs to the technical field of biomedical and computer vision, and discloses a real-time measurement method of cardiac outflow tract wall shear force based on a neural network. An image is collected by using an SD-OCT imaging system to obtain a structure image and a flow velocity image. A Doppler angle is recorded by changing the incident angle of the probe beam through a protractor head rotation. An ALSegNet network is used to extract the flow area of the cardiac outflow tract in the structure image; the flow velocity image is combined with the extracted flow area to calculate the cardiac outflow tract wall shear force through a laminar flow model flow velocity gradient and a three-dimensional rotation correction algorithm. The ALSegNet network designed in the method has small complexity, can be migrated to a mobile device, and has fast inference speed, wherein the parallel sub-network and the PDAM can ensure that the target of the OFT blood flow area is not lost to a certain extent, that is, the inference accuracy is relatively high. The method can obtain the cardiac outflow tract wall shear force in real time at a relatively fast speed.

Description

Technical Field

[0001] This invention relates to the fields of biomedicine and computer vision, and in particular to a method for real-time measurement of cardiac outflow tract wall shear force based on neural networks. Background Technology

[0002] Wall shear stress is the shear stress generated between a flowing object and its contact surface. In early embryonic development, the growth and morphological changes of the heart are closely related to the biomechanical environment. Within blood vessels, blood flow generates mechanical stimulation on the vessel walls, particularly wall shear stress (WSS), which is the frictional force acting on the surface of endothelial cells. The magnitude and direction of this force are related to changes in blood flow velocity as blood flows through the cardiovascular system. WSS can induce morphological and functional changes in endothelial cells, thereby regulating heart development. Since the cardiac outflow tract (OFT) connects the ventricles and arterial system in early heart development, many congenital heart diseases are associated with abnormal changes in the OFT during heart development. These abnormalities can lead to developmental defects in heart valves and major blood vessels, such as aortic misalignment. Therefore, measuring the WSS of the OFT is of great significance for the study of congenital heart disease.

[0003] Animal models are frequently used to explore embryonic development because their early stages of embryonic heart development are similar to those in humans. Real-time measurement of the hemodynamic saturation (WSS) within the OFT (Out-of-Flight) structure is valuable for capturing real-time feedback information generated during the heartbeat in animal embryos. This is crucial for researchers to accurately grasp changes in the hemodynamic environment of the embryonic heart and conduct experiments effectively. However, due to the small size and rapid heartbeat of early embryonic hearts, current techniques struggle to accurately and in real-time measure WSS within the OFT. Optical coherence tomography (OCT) possesses ultra-high spatial and temporal resolution and is a mature, non-invasive real-time imaging technique. Spectral domain OCT (SD-OCT) enables high-speed, high signal-to-noise ratio imaging and real-time monitoring of sample structure and flow velocity information, making it ideal for animal embryo imaging. Calculating WSS using OCT requires obtaining the absolute blood flow velocity within the OFT, but current WSS measurements are post-processed and real-time measurement is still not feasible. However, WSS measurement requires calculation based on the OFT structure. Traditional image processing-based OFT structure segmentation suffers from problems such as inability to automatically locate the structure, low segmentation accuracy, and long time consumption in real-time measurement.

[0004] With the widespread application of deep learning technology in medical imaging, segmentation methods based on convolutional neural networks (CNNs) can be used to automatically extract any target region within the imaging range. These advancements in deep learning technologies have enabled real-time measurement of the heart's peripheral blood flow (WSS) in the cardiac organ's out-of-focus area (OFT), particularly in the crucial step of automatically extracting blood flow regions. However, due to the high complexity and significant individual specificity of the structures surrounding the OFT, the direct transfer and application of traditional deep learning-based segmentation techniques may lead to problems such as target loss.

[0005] Given the aforementioned challenges, further research is needed on how to use deep learning methods to measure the wall shear stress of the cardiac outflow tract in real time. Summary of the Invention

[0006] In order to at least partially solve one of the technical problems existing in the prior art, the purpose of this invention is to propose a real-time measurement method for cardiac outflow tract wall shear force based on neural networks.

[0007] The technical solution adopted in this invention is: a real-time measurement method for cardiac outflow tract wall shear force based on neural networks, as detailed below:

[0008] Images were acquired using the SD-OCT imaging system that was built, and structural and flow velocity maps were obtained.

[0009] In the SD-OCT imaging system, the probe arm is mounted on a graduated indexing head, and the Doppler angle is recorded by changing the incident angle of the probe beam by rotating the indexing head.

[0010] The ALSegNet network is used to extract the flow region of the cardiac outflow tract from the structural map; the velocity map is combined with the extracted flow region, and the shear force of the cardiac outflow tract wall is calculated using the laminar flow model velocity gradient and the three-dimensional rotation correction algorithm.

[0011] The ALSegNet network includes a CNN backbone network, a holed spatial pyramid pooling-squeezing and excitation module, a decoder, a classification and box regression sub-network, a non-maximum suppression module (NMS), and a probability-dependent attention module (PDAM).

[0012] The ALSegNet network takes a structure map as input and extracts features through a CNN backbone network as an encoder. The resulting feature maps are then input to a hollow spatial pyramid pooling-squeezing and excitation module to obtain multi-scale feature maps, which are then input to a decoder. The decoder's feature layer is divided into three parts. The feature maps from the first part of the feature layer are input to a classification and bounding box regression sub-network, and the resulting bounding boxes are input to a non-maximum suppression module to obtain the cardiac outflow tract bounding boxes. The feature maps from the second part of the decoder's feature layer are upsampled and then input together with the cardiac outflow tract bounding boxes to a probability-dependent attention module. The feature maps from the probability-dependent attention module are then input to the third part of the decoder's feature layer to output the extracted flow region.

[0013] The CNN backbone network consists of five coding layers; the decoder consists of three upsampled residual blocks (URBs). 1-3 The encoder consists of a deep feature layer, a 1×1 convolutional layer, and a sigmoid activation function. The deepest feature layer obtained from the encoder is input to the dilated spatial pyramid pooling-squeezing and activation module. The feature map output from the dilated spatial pyramid pooling-squeezing and activation module is concatenated with the L4 output of the encoder's fourth feature layer in the channel dimension, and this concatenation is used as the fifth feature layer of the decoder. The fifth feature layer of the decoder uses a 3×3 convolution with a stride of 2 to generate the deep feature layer of the decoder. The fifth feature layer of the decoder is input to the first upsampled residual block URB1. The output feature of URB1 is concatenated with the L3 channel dimension output of the encoder's third feature layer, and this concatenation is called the fourth feature layer of the decoder. The fourth feature layer of the decoder is input to the second upsampled residual block URB2. The output feature of URB2 is concatenated with the L2 channel dimension output of the encoder's second feature layer, and this concatenation is called the third feature layer of the decoder. The third feature layer of the decoder is input to the third upsampled residual block URB3. The output feature of URB3 is concatenated with the L1 channel dimension output of the encoder's first feature layer, and this concatenation is called the second feature layer of the decoder.

[0014] The deep feature layer, fifth feature layer, and fourth feature layer of the decoder are input into the classification and box regression sub-network. The outputs of the classification and box regression sub-network are merged and input into the non-maximum suppression module to generate the cardiac outflow tract bounding box.

[0015] The cardiac outflow tract bounding box serves as the first input branch of the probability-dependent attention module. The third and second feature layers of the decoder are upsampled and then used as the second input branches of the probability-dependent attention module. Both input branches are simultaneously fed into the probability-dependent attention module, and the output features are denoted as the first feature layer of the decoder. The first feature layer of the decoder is finally processed by a 1×1 convolution and sigmoid activation to obtain the extracted flow velocity region.

[0016] The first input branch of the probability-dependent attention module uses the superposition of Gaussian functions to obtain a two-dimensional OFT probability distribution function; the two-dimensional OFT probability distribution function is based on the distance from the point to the center point of the cardiac outflow tract bounding box, and the two-dimensional OFT probability distribution function is based on:

[0017]

[0018]

[0019] F = Normalize(F target +F backward )

[0020] Where, ||·|| represents the Euclidean distance, P1 represents a point within the bounding box of the cardiac outflow tract; P2 represents a point within the background region but outside the bounding box of the cardiac outflow tract; D represents a vertex of the bounding box of the cardiac outflow tract, and C represents the center point of the bounding box of the cardiac outflow tract; α and β are constants used to adjust the level of attention; the total probability distribution function Fp is used to adjust the level of attention. target and F backward Normalize, α < β;

[0021] The second input branch of the probability-dependent attention module is split into two convolutional branches; the first convolutional branch uses a 1×1 convolutional layer and multiplies it point-by-point with the two-dimensional OFT probability distribution, while the second convolutional branch uses several 3×3 convolutional layers to extract features; the feature maps obtained from the first and second convolutional branches are concatenated along the channel dimension to obtain the output of the probability-dependent attention module.

[0022] The three upsampling residual blocks URB 1-3 Using the same structure, the input is first upsampled and then subjected to two 3×3 convolutions, batch normalization, and ReLU activation to obtain the main feature layer. Then, the result of the input upsampling is used as a residual connection and added point by point to the main feature layer to obtain the output.

[0023] The dilated spatial pyramid pooling-squeezing and activation module ASPP-SE applies n parallel dilated convolutional layers with an expansion rate of 6n to the input feature map. Each dilated convolutional layer includes batch normalization and ReLU activation. Simultaneously, feature extraction is performed on the input feature map. The features extracted from each layer are concatenated and then integrated using 1×1 convolutions with the same channels, resulting in an intermediate feature map. The intermediate feature map undergoes global average pooling and is input to two fully connected layers. The first fully connected layer uses ReLU activation, and the second fully connected layer uses sigmoid activation. After completing the two fully connected processes, the resulting feature vectors are combined with the intermediate feature map, and weights are redistributed.

[0024] The ALSegNet network performs cardiac outflow tract localization and pixel-level semantic segmentation. The ALSegNet network is trained using three different types of loss functions: segmentation loss L... seg Regression loss L box and classification loss L cls ;

[0025] The segmentation loss is the Dice loss L. Dice Weighted binary cross-entropy loss L WBCE The combination of these factors results in this loss acting at the encoder's output.

[0026] The classification loss uses focal loss to determine the category of the generated OFT bounding box; the regression loss uses smoothL1 loss to evaluate the localization accuracy; the classification loss and regression loss are applied to the cardiac outflow tract bounding box obtained by nonmaximum suppression.

[0027] L loss =ζ1L cls +ζ2L box +ζ3L seg

[0028] L seg =L Dice +L WBCE

[0029]

[0030] Adjustable coefficients ζ1, ζ2, and ζ3 are used to control the degree of attention the ALSegNet network pays to different tasks.

[0031] The shear force on the cardiac outflow tract wall is calculated as follows:

[0032] The velocity distribution within the extracted flow region is projected onto a plane perpendicular to the flow direction to obtain a projected image. The rotation angle is based on the Doppler angle θ recorded by the dial at the time of obtaining the velocity map.

[0033] OFT WSS ​​calculations are performed on the projected image. Based on the absolute velocity gradient of the edge region, the geometric center of the velocity region is set as the maximum velocity point. An annular region is extracted from the edge to a certain distance from the maximum velocity point. The velocity points within the annular region are discretized and the velocity gradient is calculated by traversing the direction of the maximum velocity point. in The line width of the annular region intercepted by the line connecting the edge point to the point of maximum flow velocity is calculated.

[0034] Finally, the calculated WSS is projected back onto the original OCT scan plane.

[0035] The beneficial effects of this invention are:

[0036] 1. The method for real-time measurement of cardiac outflow tract wall shear force proposed in this invention, which combines structural diagrams and flow velocity diagrams, can acquire cardiac outflow tract wall shear force in real time at a relatively fast speed.

[0037] 2. The ALSegNet network designed in this invention has low complexity, can be transferred to mobile devices, and has fast inference speed. The parallel subnetwork and the probability-dependent attention module can ensure that the target in the OFT blood flow region is not lost to a certain extent, that is, it has high inference accuracy. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating the real-time measurement of cardiac outflow tract wall shear force based on neural networks in an example of the present invention.

[0039] Figure 2 This is a schematic diagram of the SD-OCT structure in an example of the present invention.

[0040] Figure 3(a) is a schematic diagram of the ALSegNet network structure in an example of the present invention;

[0041] Figure 3(b) shows the upsampled decoded block (URB).

[0042] Figure 3(c) shows the void space pyramid pooling-squeezing and excitation module (ASPP-SE).

[0043] Figure 4 This is a schematic diagram of the Probability Dependency Module (PDAM) in an example of the present invention.

[0044] Figure 5 This is a schematic diagram of the ALSegNet network training process in an example of the present invention.

[0045] Figure 6 This is a schematic diagram of the shear force calculation process in an example of the present invention.

[0046] Figures 7(a)-7(h) This is a 2D view of the WSS test results of a chicken embryo heart at different stages of its beating cycle in an example of the present invention.

[0047] Figure 8 shows a 3D view of the WSS test results of chicken embryo heart at different beating cycle stages in an example of the present invention. Figure 8(a) is a structural diagram of the M-section, Figure 8(b) is the absolute flow velocity diagram corresponding to Figure 8(a), Figure 8(c) is a 3D view of the OFT blood flow region changing over time, and Figure 8(d) is a 3D view of the calculated WSS changing over time. Detailed Implementation

[0048] The embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0049] In the description of this invention, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., are based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0050] In the description of this invention, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0051] In the description of this invention, unless otherwise explicitly defined, terms such as "set up," "install," and "connect" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0052] Congenital heart disease is the most common birth defect in newborns and children, highlighting the importance of cardiac development. In the early stages of development, the biomechanical environment, particularly the outflow tract (OFT), plays a crucial role in cardiac morphological changes. The OFT is an important part of the embryonic heart, and a large proportion of congenital heart defects originate from the OFT. However, real-time measurement of the WSS in the OFT of animal models remains challenging.

[0053] Images were acquired using SD-OCT and blood flow regions were extracted using ALSegNet. Finally, flow velocity maps were combined to measure OFT wall shear force in real time, including the following steps:

[0054] Step 1: Setting up the SD-OCT imaging system: Broadband light is emitted from a broadband light source and coupled to a fiber-optic Michelson interferometer via a fiber optic circulator. The optical interference signal between the backscattered light from the probe and the reference arm is transmitted to a spectrometer. The interference spectrum is imaged onto a line-scan camera (CCD) using a spectrometer, and the Doppler angle θ is quantized using a calibrated separator head. This process ultimately yields OFT structural maps and flow velocity maps of the heart at several locations.

[0055] The broadband light source is a broadband superluminescent diode (SLD) with a center wavelength of 1310 nm and a bandwidth of 52 nm, providing an axial resolution of approximately 14 μm when irradiating samples exposed to air. The probe includes a collimator, an X-Y2D galvanometer scanning system, and an objective lens (focal length 50 mm, lateral resolution 16 μm); the spectrometer consists of a collimating lens (f = 5 mm), a 1145 line / mm transmission grating, and a Fourier lens (f = 100 mm); the line scan camera is a 1024-pixel infrared InGaAs.

[0056] Step 2: Construct the ALSegNet network, which includes a CNN backbone network, a decoder, a holed spatial pyramid pooling-squeezing and activation module (ASPP-SE), a parallel sub-network, non-maximum suppression (NMS), and a probability-dependent attention module (PDAM).

[0057] The decoder includes three identical upsampled decoding blocks (URBs), a deep feature layer, a 1×1 convolutional layer, and a sigmoid activation function.

[0058] The URB includes upsampling, convolutional layers, and residual connection layers. The convolutional layers include 3×3 convolution, ReLU activation function, and batch normalization, while the residual connection layers include 1×1 convolution, ReLU activation function, and batch normalization.

[0059] The ASPP-SE comprises: n parallel dilated convolutional layers with an expansion rate of 6n applied to the input; feature extraction is performed on the input feature map, and the features extracted from each layer are combined using a 1×1 convolution with the same channel after passing through a concatenation process; global average pooling is applied to the combined feature map, followed by two fully connected layers (FC1 and FC2), with ReLU activation functions for FC1 and sigmoid activation functions for FC2, and the weights are redistributed in combination with the combined feature map; wherein, the dilated convolutional layers include batch normalization and ReLU activation.

[0060] The parallel subnetwork (classification and box regression subnetwork) may use, but is not limited to, several 3×3 convolutional layers and 1×1 convolutional layers.

[0061] The PDAM includes: Input 1 is the OFT bounding box after NMS, and a two-dimensional probability distribution function is defined using Input 1, which is based on a mixture of two-dimensional Gaussian distributions; Input 2 is the result of the last two shallow decoder layers after bilinear interpolation upsampling. Input 2 is split into two branches. For branch 1, a 1×1 convolutional layer is used and multiplied pointwise with the defined two-dimensional probability distribution. For branch 2, several 3×3 convolutional layers are used to extract features, and the feature maps of the two branches are output after concatenation. The 3×3 convolutional layer includes 3×3 convolution, batch normalization, and ReLU function activation, and the 1×1 convolutional layer includes 1×1 convolution, batch normalization, and ReLU function activation.

[0062] In this example, the CNN backbone network can be, but is not limited to, ResNet.

[0063] Step 3: Set the loss function for network training. This loss function has three parts: segmentation loss, regression loss, and classification loss. The segmentation loss is the sum of the Dice loss and the weighted binary cross-entropy loss, the classification loss is the focal loss, and the regression loss is the smooth L1 loss.

[0064] Step 4: Preprocess the structural image acquired in Step 1 and resize it to a uniform size. In this example, an image of size 512×300 can be used, but is not limited to. Use annotation tools such as Labelme to manually annotate it at the pixel level. The annotated image is used as the segmentation label and is a binary image. Next, draw the marker box at the top, bottom, left and right endpoints of the segmentation label annotation area, and use the coordinates of a pair of diagonal points as the position label.

[0065] Step 5: Pair the original image, segmentation labels, and location labels and encapsulate them into a dataset. Iterate the training until the network converges to obtain the trained ALSegNet network.

[0066] Step 6: Load the pre-trained ALSegNet network weights and extract the blood flow region in real time from the input heart OFT structural map.

[0067] Step 7: Combine the blood flow region extracted in Step 6 with the cardiac OFT velocity map, and use the laminar flow model velocity gradient and three-dimensional rotation correction algorithm to calculate the cardiac outflow tract wall shear force. That is, rotate the sample on the XY plane until the X scan direction of OCT is perpendicular to the flow direction. The rotation angle is based on the Doppler angle θ recorded when the velocity map was obtained in Step 1.

[0068] like Figure 1 As shown, this embodiment provides a real-time measurement method for cardiac outflow tract wall shear force based on ALSegNet, specifically including the following steps:

[0069] S1. Construct the SD-OCT imaging system, as shown in the structural diagram. Figure 2 A broadband light source emits broadband light, which is coupled to a fiber-optic Michelson interferometer via a fiber optic circulator. The optical interference signal between the backscattered light from the probe and the reference arm is transmitted to a spectrometer. The interference spectrum is then imaged onto a line-scan camera (CCD) using a spectrometer.

[0070] The broadband light source is a broadband superluminescent diode (SLD) with a center wavelength of 1310 nm and a bandwidth of 52 nm, which can provide an axial resolution of approximately 14 μm in air; the probe includes a collimator, an XY 2D galvanometer scanning system, and an objective lens (focal length 50 mm, lateral resolution 16 μm); the spectrometer consists of a collimating lens (f = 5 mm), a 1145 line / mm transmission grating, and a Fourier lens (f = 100 mm); the line scan camera is a 1024 pixel infrared InGaAs.

[0071] In SD-OCT, information along one axis is acquired in parallel, and the interference spectrum encodes the backscattering information of the sample, with different frequencies corresponding to different depths. Therefore, by performing a wavenumber space Fourier transform on the interference spectrum, the backscattering distribution at different depths of the sample can be obtained:

[0072]

[0073] Where A is the Fourier transform of the spectral light source S(k), This represents the convolution operation; The curve represents the coherence function, with the half-width at half-maximum (FWHM) being the coherence length, located near z = 0. The backscattering amplitude distribution at different depths of the sample is given by a(z) and a * (-z) is symmetric about z = 0; This is a Fourier transform of the interference spectrum at different depths of the sample. The Fast Fourier Transform of the interference spectrum is complex and contains phase information. Phase changes are introduced when red blood cells flow in blood vessels. These phase changes can be extracted by performing conjugate multiplication between the two sequential A lines at each pixel.

[0074]

[0075] Where λ is the center wavelength of the light source; n is the refractive index of the tissue; τ is the time interval between two consecutive A-scans; θ is the Doppler angle, i.e., the angle between the probe beam and the flow direction; and V is the absolute velocity. Therefore, the absolute velocity can be obtained as follows:

[0076]

[0077] To accurately measure the Doppler angle θ, this example mounts the probe arm on a graduated head. Therefore, the incident angle of the probe beam can be changed by rotating the graduated head.

[0078] The process ultimately yields OFT structural maps and flow velocity maps of the heart at several locations.

[0079] S2. Construct the ALSegNet network, as shown in Figure 3. The ALSegNet network includes a CNN backbone network as an encoder and decoder, a hollow spatial pyramid pooling-squeezing and activation module (ASPP-SE), multiple parallel sub-networks (classification and regression) for OFT localization, non-maximum suppression (NMS), and a probability-dependent attention module (PDAM).

[0080] S2.1 To capture features at different depths, this example can use, but is not limited to, an 18-layer ResNet as the CNN backbone network, with the decoder consisting of three upsampled residual blocks (URBs). Each URB includes upsampling, convolutional layers, and residual connection layers. The convolutional layers include 3×3 convolutions, ReLU activation functions, and batch normalization, while the residual connection layers include 1×1 convolutions, ReLU activation functions, and batch normalization.

[0081] Furthermore, the ALSegNet network adds skip connections from the first to fourth feature layers (L1-L4) of the CNN backbone network to the decoder, fully utilizing the semantic features extracted by the backbone network. The structure of each decoding unit is shown in Figure 3(a). This module upsamples the input feature map using bilinear interpolation and extracts features using two 3×3 convolutions and a 1×1 convolution as a residual connection. The network's decoder generates multiple different feature layers at different depths.

[0082] S2.2 To locate the Oft First (OFT) in an OCT image, global feature information of the target is required. Since the first two layers of the decoder contain global feature information, this example uses multiple parallel sub-networks (classification and bounding box regression sub-networks) for OFT localization. These parallel sub-networks can use, but are not limited to, several 3×3 convolutional layers and 1×1 convolutional layers. To generate more global information, this example downsamples the deeper layers of the decoder to generate additional deep feature layers, which are then used as input to the parallel sub-networks. In each layer of the parallel sub-network, this example sets the aspect ratio of the prior bounding boxes to [(1:0.5), (1:1), (1:2)], using these three ratios to generate 9 bounding boxes for each feature point. Since the output of the parallel sub-network consists of multiple bounding box coordinates, non-maximum suppression (NMS) needs to be further applied to the output of the parallel sub-network to output a unique bounding box, thus achieving OFT localization.

[0083] S2.3 Since the parallel sub-network only uses global features, the output is an approximate OFT localization. To improve the extraction accuracy of OFT bounding boxes, this example designs a Hollow Spatial Pyramid Pooling-Squeezing and Activation Module (ASPP-SE) at the junction of the encoder and decoder. The detailed architecture of ASPP-SE is shown in Figure 3(b). This module extracts and integrates features of different scales from the deep feature map generated by the CNN backbone network and introduces an attention mechanism in the channel dimension. The ASPP-SE module comprises: n parallel dilated convolutional layers with an expansion rate of 6n applied to the input; simultaneous feature extraction from the input feature map, with the extracted features from each layer being concatenated and then integrated using 1×1 convolutions with the same channels; global average pooling of the integrated feature map to generate global information; followed by two fully connected layers (FC1 and FC2), with ReLU activation functions for FC1 and sigmoid activation functions for FC2, which helps the network learn the nonlinear interactions between channels and capture their correlations; and redistributing weights based on the integrated feature map. The dilated convolutional layers include batch normalization and ReLU activation. The integration of the ASPP-SE module not only reduces the localization bias of the parallel sub-networks but also significantly improves the efficiency of global information learning within the network.

[0084] S2.4 Following the above process, the network roughly locates the OFT in the OCT image. However, it cannot guarantee that all blood flow regions are within the obtained OFT bounding box. Since performing pixel-level segmentation directly within the OFT bounding box leads to relatively low accuracy, this example designs a probability-dependent attention module (PDAM) to guide high-precision segmentation. Figure 4 ).

[0085] The PDAM includes: a first input branch is the OFT bounding box of the parallel subnetwork after passing through NMS; a two-dimensional OFT probability distribution function is constructed based on the first input branch to constrain the segmentation of the flow region; the distribution principle is as follows:

[0086]

[0087]

[0088] F = Normalize(F target +F backward )

[0089] Where, |·| represents the Euclidean distance, P1 represents a point inside the OFT bounding box, P2 represents a point within the background region (outside the bounding box), D represents a vertex of the OFT bounding box, and C represents the center point of the OFT bounding box; α and β are constants used to adjust the level of attention. The total probability distribution function (F) is related to F target and F backward Normalization was performed, indicating that points outside the bounding box can still be identified. Typically, background points receive less attention than points within the bounding box (α < β). This process reduces the strong constraint of the OFT bounding box on pixel-level semantic segmentation. In other words, this process effectively extends the bounding box to some extent.

[0090] The second input branch is the result of bilinear interpolation upsampling from the last two shallow decoder layers. This second input branch is split into two branches. For the first branch, a 1×1 convolutional layer is used and multiplied point-by-point with a defined two-dimensional probability distribution. For the second branch, several 3×3 convolutional layers are used to extract features. The feature maps from both branches are then concatenated and output. The 3×3 convolutional layers include 3×3 convolution, batch normalization, and ReLU activation; the 1×1 convolutional layers include 1×1 convolution, batch normalization, and ReLU activation. Using two branches avoids excessive constraints on the OFT bounding box by the compensation mechanism. The second branch utilizes features from the decoder surface to compensate for the OFT localization bias in the first branch. The PDAM output is segmented using 1×1 convolution and a sigmoid function.

[0091] S2.5. Set the loss function for network training. Since the ALSegNet network includes OFT localization and pixel-level semantic segmentation, the network's loss function (L... loss It consists of three parts (segmentation loss, regression loss, and classification loss):

[0092] L loss =ζ1L cls +ζ2L box +ζ3L seg

[0093] L seg =L Dice +L WBCE

[0094]

[0095] Among them, classification loss (L clsFocal loss is used to determine the category (true or false) of the generated OFT bounding box; regression loss (L) box The smooth L1 loss is used to evaluate localization accuracy. Segmentation loss (L...) seg ) is Dice loss (L Dice ) and weighted binary cross-entropy loss (L WBCE The combination of ζ1, ζ2, and ζ3 represents adjustable loss coefficients. t i y represents the pixel value of the real image. i σ represents the pixel value of the predicted image, and is used as a weighting parameter to adjust the attention given to the OFT region.

[0096] S3. Preprocess the structure image acquired in S1 and resize it to a uniform size. This example can use, but is not limited to, images of 512×320 pixels. Use annotation tools such as LabelMe to manually annotate it at the pixel level. The annotated image serves as the segmentation label and is a binary image. Next, draw bounding boxes at the top, bottom, left, and right endpoints of the segmentation label annotation region, using the coordinates of a pair of diagonal points as location labels. Pair and encapsulate the original image, segmentation labels, and location labels into a dataset. Iteratively train the network until it converges to obtain the trained ALSegNet network. The training process is described in [link to training instructions]. Figure 5 Load the pre-trained ALSegNet network weights and extract the blood flow region in real time from the input heart OFT structural map.

[0097] S4. To achieve quantitative measurement of the shear stress on the OFT (Out-of-Flight) wall of the chicken embryo heart, blood is typically considered a Newtonian fluid with a low Reynolds number. Therefore, blood flow in the OFT can be regarded as laminar flow. Viscous blood flow suspends red blood cells, white blood cells, platelets, and other particles (at 37°C, the viscosity coefficient of blood μ = 2.5–4 × 10⁻⁶). -3 The endocardium (Pa·s) in contact with the endocardium of the cardiovascular wall generates a frictional force, known as wall shear stress. To better understand the influence of embryonic morphology on hemodynamics, this example quantifies WSS and defines it based on general fluid dynamics principles:

[0098]

[0099] in Let μ represent the absolute velocity gradient and μ represent blood viscosity. Theoretically, WSS calculations are performed on a cross-section perpendicular to the flow direction. Figure 6 (a) The red plane with coordinates (x, y, z). However, Doppler OCT only provides velocity images in planes not perpendicular to blood flow (Doppler angle is not equal to π / 2). Figure 6(a) The blue plane with coordinates (x′, y′, z′). Because OCT measures the velocity component parallel to the probe beam, to calculate the WSS, the velocity distribution within the extracted flow region (blue plane) needs to be projected onto a plane perpendicular to the flow direction (red plane). However, the coordinate systems of these two planes are independent, making projection difficult. To simplify the calculation, this example rotates the sample in the XY plane until the X-scan direction of the OCT is perpendicular to the flow direction, i.e., the x-axis of the red plane and the x′-axis of the blue plane are in the same direction. The projection principle is as follows:

[0100]

[0101] Where θ is the Doppler angle recorded when S1 obtains the flow velocity map. Figure 6 (d) shows the result after projection. Furthermore, the Doppler angle is used to correct the flow velocity (velocity component) acquired by the OCT to absolute velocity, and OFT WSS ​​is calculated on the projected image (6(e)). Finally, the calculated WSS is projected back onto the original OCT scan plane (6(f)).

[0102] The embodiments of the real-time measurement method for cardiac outflow tract wall shear force based on neural networks of the present invention can be applied to any device with data processing capabilities, such as a computer or other similar equipment. The device embodiments can be implemented through software, hardware, or a combination of both. Since the device embodiments essentially correspond to the method embodiments, relevant details can be found in the descriptions of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this invention without any creative effort.

[0103] Supplementary Experiment: Since chicken embryos at Hamburger-Hamilton (HH) stage 18 are relatively easy to obtain and develop rapidly compared to most other animal models, this example uses HH stage 18 chicken embryo hearts to provide additional experimental validation of the method. The preparation of the chicken embryos followed a standard procedure: fertilized white Leghorn eggs were cultured in a rotating incubator at 38°C and 85% humidity for approximately 3 days. After opening the eggshell at the air sac end, a small section of the inner shell membrane was carefully removed. The egg was then placed in a specially designed acrylic box. During image acquisition, the temperature in the acrylic box was maintained at 37.5°C using a heating blanket. Figure 7 shows a 2D view of the WSS test results using chicken embryo hearts at different beating cycle stages in this example of the invention. Figure 8 shows a 3D view of the WSS test results using chicken embryo hearts at different beating cycle stages in this example of the invention. (a) is a structural diagram of the M-section, (b) is the absolute velocity diagram corresponding to (a), (c) is a 3D view of the OFT blood flow region changing over time, and (d) is a 3D view of the calculated WSS changing over time.

Claims

1. A method for real-time measurement of cardiac outflow tract wall shear force based on neural networks, characterized in that, Specifically as follows: Images were acquired using the SD-OCT imaging system that was built, and structural and flow velocity maps were obtained. In the SD-OCT imaging system, the probe arm is mounted on a graduated indexing head, and the Doppler angle is recorded by changing the incident angle of the probe beam by rotating the indexing head. The ALSegNet network is used to extract the flow region of the cardiac outflow tract from the structural map; the velocity map is combined with the extracted flow region, and the shear force of the cardiac outflow tract wall is calculated by the laminar flow model velocity gradient and the three-dimensional rotation correction algorithm. The ALSegNet network includes a CNN backbone network, a holed spatial pyramid pooling-squeezing and excitation module, a decoder, a classification and box regression sub-network, a non-maximum suppression module (NMS), and a probability-dependent attention module (PDAM). The input to the ALSegNet network is a structure map. Features are extracted by the CNN backbone network as an encoder. The resulting feature map is input to the hollow space pyramid pooling-squeezing and excitation module to obtain multi-scale feature maps. The multi-scale feature maps are then input to the decoder. The feature layer of the decoder is divided into three parts. The feature map obtained from the first part of the feature layer is input to the classification and bounding box regression sub-network. The resulting bounding boxes are input to the non-maximum suppression module to obtain the cardiac outflow tract bounding boxes. The feature map obtained from the second feature layer of the decoder is upsampled and then input together with the cardiac outflow tract bounding box into the probability-dependent attention module. The feature map obtained from the probability-dependent attention module is input into the third feature layer of the decoder to obtain the extracted flow region.

2. The method for real-time measurement of cardiac outflow tract wall shear force based on neural network according to claim 1, characterized in that, The CNN backbone network consists of five coding layers; the decoder consists of three upsampled residual blocks (URBs). 1-3 The deep feature layer, 1×1 convolutional layer and sigmoid activation function are used. The deepest feature obtained by the encoder is input to the hole space pyramid pooling-squeezing and activation module. The feature map output by the hole space pyramid pooling-squeezing and activation module is concatenated with the L4 output of the fourth feature layer of the encoder in the channel dimension and used as the fifth feature layer of the decoder. The fifth feature layer of the decoder uses a 3×3 convolution with a stride of 2 to generate the deep feature layer of the decoder; decoder The fifth feature layer is input to the first upsampled residual block URB1. The output features of URB1 are concatenated with the L3 channel dimension output of the third feature layer of the encoder, which is denoted as the fourth feature layer of the decoder. decoder The fourth feature layer is input to the second upsampled residual block URB2. The output features of URB2 are concatenated with the L2 channel dimension output of the second feature layer of the encoder, which is called the third feature layer of the decoder. The third feature layer of the decoder is input to the third upsampled residual block URB3. The output features of URB3 are concatenated with the L1 channel dimension output of the first feature layer of the encoder, which is called the second feature layer of the decoder. The deep feature layer, fifth feature layer, and fourth feature layer of the decoder are input into the classification and box regression sub-network. The outputs of the classification and box regression sub-network are merged and input into the non-maximum suppression module to generate the cardiac outflow tract bounding box. The cardiac outflow tract bounding box serves as the first input branch of the probability-dependent attention module. The third feature layer and the second feature layer of the decoder are upsampled and then used as the second input branch of the probability-dependent attention module. Both input branches are simultaneously input into the probability-dependent attention module, and the output features are denoted as the first feature layer of the decoder. The first feature layer of the decoder is finally processed by 1×1 convolution and sigmoid activation to obtain the extracted flow velocity region.

3. The method for real-time measurement of cardiac outflow tract wall shear force based on neural network according to claim 2, characterized in that, The first input branch of the probability-dependent attention module uses the superposition of Gaussian functions to obtain a two-dimensional OFT probability distribution function; the two-dimensional OFT probability distribution function is based on the distance from the point to the center point of the cardiac outflow tract bounding box, and the two-dimensional OFT probability distribution function is based on: ； in, Represents Euclidean distance. This represents a point within the bounding box of the cardiac outflow tract; This represents a point within the background area but outside the bounding box of the cardiac outflow tract. Represents the vertices of the bounding box of the cardiac outflow tract. Indicates the center point of the bounding box of the cardiac outflow tract; It is a constant used to adjust the level of attention; the total probability distribution function. right Normalize, ; The second input branch of the probability-dependent attention module is split into two convolutional branches; the first convolutional branch uses a 1×1 convolutional layer and multiplies it point-by-point with the two-dimensional OFT probability distribution, while the second convolutional branch uses several 3×3 convolutional layers to extract features; the feature maps obtained from the first and second convolutional branches are concatenated along the channel dimension to obtain the output of the probability-dependent attention module.

4. The method for real-time measurement of cardiac outflow tract wall shear force based on neural network according to claim 3, characterized in that, The three upsampling residual blocks URB 1-3 Using the same structure, the input is first upsampled and then subjected to two 3×3 convolutions, batch normalization, and ReLU activation to obtain the main feature layer. Then, the result of the input upsampling is used as a residual connection and added point by point to the main feature layer to obtain the output.

5. The method for real-time measurement of cardiac outflow tract wall shear force based on neural network according to claim 4, characterized in that, The dilated spatial pyramid pooling-squeezing and activation module ASPP-SE applies n parallel dilated convolutional layers with an expansion rate of 6n to the input feature map. Each dilated convolutional layer includes batch normalization and ReLU activation. Simultaneously, feature extraction is performed on the input feature map. The features extracted from each layer are concatenated and then integrated using 1×1 convolutions with the same channels, resulting in an intermediate feature map. The intermediate feature map undergoes global average pooling and is input to two fully connected layers. The first fully connected layer uses ReLU activation, and the second fully connected layer uses sigmoid activation. After completing the two fully connected processes, the resulting feature vectors are combined with the intermediate feature map, and weights are redistributed.

6. The method for real-time measurement of cardiac outflow tract wall shear force based on neural network according to claim 5, characterized in that, The ALSegNet network performs cardiac outflow tract localization and pixel-level semantic segmentation. The ALSegNet network is trained using three different types of loss functions: segmentation loss... Regression loss and classification loss ; Segmentation loss is Dice loss Weighted binary cross-entropy loss The combination of these factors results in this loss acting at the encoder's output. The classification loss uses focal loss to determine the category of the generated OFT bounding box; the regression loss uses smooth L1 loss to evaluate the localization accuracy; the classification loss and regression loss are applied to the cardiac outflow tract bounding box obtained by nonmaximum suppression. ； Use adjustable coefficients This is used to control the degree of attention the ALSegNet network pays to different tasks.

7. A method for real-time measurement of cardiac outflow tract wall shear force based on neural networks according to any one of claims 1-6, characterized in that, The shear force on the cardiac outflow tract wall is calculated as follows: The velocity distribution within the extracted flow region is projected onto a plane perpendicular to the flow direction to obtain a projected image. The rotation angle is based on the Doppler angle recorded by the dial at the time the velocity map was acquired. ; OFT WSS ​​calculations are performed on the projected image. Based on the absolute velocity gradient of the edge region, the geometric center of the velocity region is set as the maximum velocity point. An annular region is extracted from the edge to a certain distance from the maximum velocity point. The velocity points within the annular region are discretized and the velocity gradient is calculated by traversing the direction of the maximum velocity point. ;in The line width of the annular region intercepted by the line connecting the edge point to the point of maximum flow velocity is calculated. Finally, the calculated WSS is projected back onto the original OCT scan plane.

Images