Fetal heart uterine contraction automatic interpretation system and method based on network representation

Abstract

The invention discloses a fetal heart uterine contraction automatic interpretation system based on network characterization, which comprises a network model, and the network model comprises a CV module, an AT module, an SW module and a TC module which are connected in sequence; the CV module encodes low-level spatio-temporal information in the CTG signal through three convolutional layers; the AT module comprises a plurality of self-attention layers; the TC module uses a time convolution network to extract advanced time features in a time sequence, and inputs the advanced time features into a full connection layer with a Sigmoid classifier. According to the method, a multi-representation and multi-model integration thought is adopted, related features beneficial to CTG interpretation are fully extracted, the problem that a manual feature extraction algorithm is insufficient in capturing a long-time-span dependency relationship is solved, subsequent various analysis and research on CTG signals are facilitated, and the method is suitable for popularization and application. More accurate and reliable diagnosis information is provided for doctors and applied to the field of medical signal processing, and a more accurate and reliable auxiliary diagnosis tool is provided for clinicians.

Description

Technical Field

[0001] The present invention relates to the field of medical signal processing technology, and more specifically, to a fetal heart rate and uterine contraction chart automatic interpretation system method based on network representation Background Art

[0002] Cardiotocography (CTG), as the core means of perinatal monitoring, aims to assess the health of the fetus by real-time monitoring of fetal heart rate signals and uterine contraction signals. CTG plays a vital role before and during delivery. It is a standard method for assessing the health of the fetus in utero, and it is also the only intuitive tool for clinicians and midwives to make a preliminary diagnosis by naked eye observation alone. However, the accuracy of this visual interpretation is highly dependent on detailed interpretation guidelines and the rich clinical experience of doctors. Due to the high subjectivity of this method, there may be differences in interpretation between different doctors, which increases the risk of misdiagnosis and poses a potential threat to the health of pregnant women and fetuses. In order to reduce this subjectivity and misdiagnosis rate, it is urgent to introduce more accurate and objective diagnostic aids. These tools can provide doctors with more accurate and reliable diagnostic information based on key physiological parameters such as fetal heart rate (FHR) through advanced algorithms and data analysis technologies. However, there are currently technical difficulties, which are mainly reflected in:

[0003] Data complexity: CTG images not only include changes in fetal heart rate, but also incorporate a variety of physiological information such as uterine contraction activity. This complexity of data requires intelligent judgment tools to have strong data processing and analysis capabilities to accurately extract key information related to the health status of the fetus.

[0004] Difficulty in pattern recognition: The fluctuations in fetal heart rate and the patterns of uterine contraction activity are varied and may be affected by many factors, such as the pregnant woman’s body position, mood, medication use, etc. Intelligent judgment tools need to be able to recognize these complex patterns and accurately distinguish between normal and abnormal situations.

[0005] Real-time requirement: Before and during delivery, the health status of the fetus may change at any time. Therefore, the intelligent judgment tool needs to have the ability to process and analyze CTG data in real time to provide accurate diagnostic information to clinicians and midwives in a timely manner.

[0006] Personalized needs: The situation of each pregnant woman and fetus is unique, and intelligent judgment tools need to be able to adapt to this personalized need and provide accurate judgment results for different situations. This requires the tool to fully consider factors such as the medical history of the pregnant woman and the development of the fetus. Summary of the invention

[0007] The purpose of the present invention is to provide a network-characterized fetal heart and uterine contraction monitoring signal recognition accuracy improvement system and method, which can promote the development of CTG signal analysis.

[0008] In order to achieve the above object, the technical solution adopted by the present invention is as follows: an automatic interpretation system of fetal heart and uterine contraction chart based on network representation, including a network model, the network model includes a CV module, an AT module, a SW module and a TC module connected in sequence;

[0009] The CV module is a convolution module, which encodes low-level spatiotemporal information in the CTG signal through three convolutional layers, and each convolutional layer is respectively provided with different filters to extract spatial features in each temporal feature map;

[0010] The SW module is a convolution-based sliding window module, which divides the samples of length into windows, then uses the integration of sliding windows and convolution layers, and finally performs a convolution calculation on all windows;

[0011] The AT module is an attention module. The AT module is based on an attention score method and consists of a multi-head attention mechanism, including multiple self-attention layers, where each self-attention layer consists of three different components, namely query (Query, Q), key (Key, K) and value (Value, V). According to their respective functions, the interaction between Q and K generates an attention score that guides the selection bias of V, and finally obtains the most important information in the highlighted time series;

[0012] The TC module is a temporal convolution module, which consists of two residual modules. The TC module is used to extract high-level temporal features from the time series using a temporal convolutional network and input them into a fully connected layer with a Sigmoid classifier.

[0013] Preferably, the CV module is cascaded by three convolutional layers, the first convolutional layer extracts the temporal feature map, the second convolutional layer extracts the spatial features in each temporal feature map, and the third convolutional layer controls the length of the time series generated by CV.

[0014] Preferably, the SW module, drawing on the idea of ​​sliding windows, uses a sliding window of length to divide the time series w into multiple windows, where represents the window index and is the total number of windows.

[0015] Preferably, the AT module consists of an MSA layer, which includes multiple self-attention layers, called heads, each of which consists of three main components: query (Query, Q), key (Key, K) and value (Value, V). The interaction between Q and K produces an attention score that guides the selection bias of V.

[0016] Preferably, the TC module consists of two residual blocks, each residual block consists of two dilated causal convolutional layers, each convolutional layer is followed by batch normalization and ELU activation function, and the TC module can expand its receptive field size exponentially while its parameter amount only increases linearly.

[0017] A method for automatically interpreting fetal heart rate and uterine contraction chart based on network representation, comprising:

[0018] The CV module encodes the low-level spatiotemporal information in the CTG signal through three convolutional layers, and each convolutional layer sets different filters to extract the spatial features in each temporal feature map;

[0019] The SW module adopts the idea of ​​sliding window. First, the samples of length are divided into windows, then the sliding window and convolution layer are integrated, and finally a convolution calculation is performed on all windows.

[0020] The AT module is based on an attention score method, which consists of a multi-head attention mechanism, including multiple self-attention layers, where each self-attention layer consists of three different components, namely query (Query, Q), key (Key, K) and value (Value, V). Then, according to their respective functions, the interaction between Q and K produces an attention score that guides the selection bias of V, and finally obtains the most important information in the highlighted time series;

[0021] The TC module consists of two residual modules, which are used to extract high-level temporal features from time series using a temporal convolutional network and input them into a fully connected layer with a sigmoid classifier.

[0022] A computer-readable storage medium, characterized in that a computer program is stored thereon, and when the computer program is executed in a computer, the computer is caused to execute the method described in claim 6.

[0023] Compared with the prior art, the advantages of the present invention are:

[0024] The automatic interpretation system and algorithm of fetal heart and uterine contraction chart based on network representation of the present invention have high accuracy and universality, adopt the idea of ​​multi-representation and multi-model integration, fully extract relevant features that are helpful for CTG interpretation, and make up for the deficiency of manual feature extraction algorithm in capturing long-time span dependencies, which facilitates various subsequent analyses and studies of CTG signals, provides doctors with more accurate and reliable diagnostic information for application in the field of medical signal processing, and provides clinicians with more accurate and reliable auxiliary diagnostic tools. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 It is a network structure diagram of the fetal heart and uterine contraction chart automatic interpretation system based on network representation of the present invention;

[0027] Figure 2 It is a schematic diagram of the convolution (CV) module structure of the present invention;

[0028] Figure 3 Schematic diagram of the structure of the multi-head self-attention mechanism (AT) module of the present invention;

[0029] Figure 4 It is a schematic diagram of the structure of the temporal convolution (TC) module of the present invention;

[0030] Figure 5 It is a data table showing the influence of different classification weights on model performance of the present invention;

[0031] Figure 6 This is a data table showing the impact of different attention mechanisms on model performance in the present invention;

[0032] Figure 7 It is a data diagram of the effects of different head dimensions and head numbers on the QI and ACC of the model of the present invention;

[0033] Figure 8 It is a data table showing the influence of different numbers of sliding windows on the model performance of the present invention;

[0034] Fig. 9 is a model reasoning visualization heat map of normal CTG signals of the present invention;

[0035] Fig.10 is a model reasoning visualization heat map of abnormal CTG signals of the present invention;

[0036] Fig.11 This is a table of classification performance data of different methods of the present invention on the CTU-UHB Data1 dataset. DETAILED DESCRIPTION

[0037] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.

[0038] See also Figure 1-4As shown, the present invention provides a fetal heart and uterine contraction chart automatic interpretation system based on network representation, the system includes a network model, and the network model includes a CV module, an AT module, a SW module and a TC module connected in sequence;

[0039] The CV module and CTC signals are fed into the network and enter the convolution (CV) module, which is composed of three convolutional layers cascaded to encode the low-level spatiotemporal information of the CTG signal. The first convolutional layer consists of F1 1*KC filters, where F1 represents the number of temporal feature maps output by this layer. The second layer is a deep convolutional layer, which uses F2 filters of size C*1 to extract spatial features in each temporal feature map. This layer outputs F_1*D feature maps, where D is the number of filters associated with each temporal feature map of the previous layer, and D is set to 2 based on experience. The deep convolutional layer is followed by an average pooling layer of size 1*8, which abstracts the temporal data at an 8-fold scale. The third convolutional layer consists of F2 filters of size 1*KC2, where KC2 is set to 16. Finally, an average pooling layer of 1*P2 is used, where P2 is used to control the length of the time series generated by the CV module. The second and third convolutional layers are followed by batch normalization to speed up network training, and then nonlinear transformations are performed using Exponential Linear Unit (ELU) activations.

[0040] The output of the V module is composed of a T C Time series composed of time vectors The dimension of each vector is d = F2 = F2 × D. In this section, d is set to 32, and the length of the time series is:

[0041]

[0042] Wherein, T is the length of the input CTG signal (7200).

[0043] The SW module and the SW module draw on the idea of ​​sliding window and set the length of T C The samples are divided into n windows. To reduce the amount of calculation as much as possible, the sliding window and the convolution layer are integrated, that is, a convolution calculation is performed on all windows. The training and reasoning time are reduced through the parallelization process. This integrated method can effectively reduce the calculation time and improve the efficiency of the model. This algorithm uses a length of T w The sliding window of the time series z i Split into multiple windows Where w = 1, ..., n represents the window index, and n is the total number of windows. Each window z i w Enter the subsequent AT module to extract important information, and then enter the TC module. Window length T w The definitions are as follows:

[0044]

[0045] If the CV module performs two temporal poolings of size P1=8 and P2=7, the CV module will produce a pool consisting of T c = a time series of 20 vectors z i (where T is 7200). Each vector will represent the original CTG signal x i 56 (8*7) time points in . Therefore, performing a one-step slide in is equivalent to performing 56 time steps of sliding in the original signal. i Performing a one-step slide in is equivalent to sliding the original signal x i A sliding of 56 time steps is performed in .

[0046] AT modules and attention mechanisms can be implemented based on attention scores or through different ML algorithms (such as reinforcement learning). In this algorithm, we draw on the great success of multi-headed attention mechanisms in fields such as natural language processing and computer vision, and adopt an attention score-based method, namely Multi-headed Self Attention (MSA).

[0047] The AT module consists of an MSA layer, which includes multiple self-attention layers, called heads, each of which consists of three main components: query (Query, Q), key (Key, K), and value (Value, V). The interaction between Q and K produces an attention score that guides the selection bias of V.

[0048] TC module,TC module consists of two residual blocks, each residual block consists of two dilated causal convolutional layers, each convolutional layer is followed by batch normalization and ELU activation function, TC module can expand its receptive field size exponentially, while its parameter amount increases only linearly. In the residual block, this algorithm uses an identity mapping because the input and output dimensions are 32, otherwise a linear transformation, i.e., 1×1 convolution, is required. Since the dilation coefficient D in each subsequent module increases exponentially, the receptive field size (RFS) of the temporal convolution increases exponentially with the number of stacked residual blocks L. RFS is controlled by two parameters: the number of residual blocks L and the convolution kernel size K_T, as defined below:

[0049] RFS=1+2(K T -1)(2 L -1)

[0050] In the network structure of this algorithm, the TC module consists of two temporal convolutions with residual blocks and 32 filters of size KT=4. With this module, the RFS is 19, that is, TCN can process up to 19 sequence elements. Therefore, the time series entering the TC module should be less than or equal to 19 to allow the temporal convolution to process all the temporal information without loss. For sequences longer than RFS, they can be divided into multiple windows, each of which has a length less than RFS. Then, each window is input into the TC module separately. In this paper, RFS is fixed to 19, and the length of the window is adjusted according to Tw.

[0051] In order to explore the impact of different attention mechanisms on model performance, this paper compares three attention mechanisms commonly used in classification tasks, MSA, SE and CBAM. When the attention mechanism is CBAM, the SPE value of the model classification is the highest, indicating that CBAM has a good effect on the recognition of normal samples, but its recognition effect on abnormal samples is poor, and the SEN is only 75.00%. When the attention mechanism is SE, the SEN of the model classification reaches 80.31%, that is, the model has a good effect on the recognition of abnormal samples, but its recognition effect on normal samples is poor, only 66.80%. Further analysis shows that when the attention mechanism is MSA, the QI, ACC, AUC and BS values ​​of the model reach the maximum, that is, the model achieves the best comprehensive classification effect on positive and negative samples. In order to further verify the impact of different head dimensions and number of heads on the model, relevant experiments were conducted. The experimental results are as follows. Figure 5 and Figure 6 As shown. The experimental results show that when the head dimension of the multi-head attention mechanism is 8 and the number of heads is 4. The SEN value of the model reaches the highest 80.94%, that is, the model has the best recognition effect on abnormal samples at this time, but the recognition accuracy of normal samples is low, and the SPE value is only 64.26%. When the head dimension of the multi-head attention mechanism is 16 and the number of heads is 2, the model performs best in 6 indicators, among which QI = 73.34%. Figure 7 As shown in the figure, when the head dimension is 8 and the number of heads is 2, the QI and ACC values ​​are both optimal, which are 74.76% and 75.68% respectively.

[0052] To explore the impact of the number of windows on model performance, this paper compares the impact of 6 different numbers of windows on model performance, such as Figure 8As shown in the figure, when the number of sliding windows is 1, the model has the best classification effect on normal signals, with SPE reaching a maximum of 75.32%, but its recognition effect on abnormal samples is poor. When the number of sliding windows is 2, the model has the best recognition effect on abnormal samples, with SEN reaching 80.93%, but its SPE value is only 62.98%, that is, the model has low recognition accuracy for normal samples. When the sliding window is 5, the model's ACC, QI, F1, AUC and BS are the best, which are 75.68%, 74.76%, 76.11%, 0.8407 and 0.2432 respectively.

[0053] The network-representation-based fetal heart and uterine contraction automatic interpretation algorithm constructed by the present invention is used to visualize the model reasoning heat map representation of normal CTG signals and abnormal CTG signals, such as Fig. 9 , as shown in 10.

[0054] To further demonstrate the effectiveness and universality of the algorithm, the method proposed in this chapter is compared with previous work using the CTU-UHB Data1 database. The results are as follows: Fig.11 As shown, P-value is the statistical indicator of QI between the comparison method and the method in this chapter.

[0055] Although the embodiments of the present invention are described in conjunction with the accompanying drawings, the patent owner may make various variations or modifications within the scope of the appended claims. As long as they do not exceed the protection scope described in the claims of the present invention, they should be within the protection scope of the present invention.

Claims

1. A fetal heart rate and uterine contraction chart automatic interpretation system based on network representation, including a network model, characterized in that: The network model includes a CV module, an AT module, a SW module, and a TC module connected in sequence; The CV module is a convolution module, which encodes low-level spatiotemporal information in the CTG signal through three convolutional layers, and each convolutional layer is respectively provided with different filters to extract spatial features in each temporal feature map; The SW module is a convolution-based sliding window module, which divides the samples of length into windows, then uses the integration of sliding windows and convolution layers, and finally performs a convolution calculation on all windows; The AT module is an attention module. The AT module is based on an attention score method and consists of a multi-head attention mechanism, including multiple self-attention layers, where each self-attention layer consists of three different components, namely query (Query, Q), key (Key, K) and value (Value, V). According to their respective functions, the interaction between Q and K generates an attention score that guides the selection bias of V, and finally obtains the most important information in the highlighted time series; The TC module is a temporal convolution module, which consists of two residual modules. The TC module is used to extract high-level temporal features from the time series using a temporal convolutional network and input them into a fully connected layer with a Sigmoid classifier.

2. The automatic interpretation system of fetal heart and uterine contraction chart based on network representation according to claim 1, characterized in that: The CV module is composed of three cascaded convolutional layers. The first convolutional layer extracts the temporal feature map, the second convolutional layer extracts the spatial features in each temporal feature map, and the third convolutional layer controls the length of the time series generated by CV.

3. The automatic interpretation system of fetal heart and uterine contraction chart based on network representation according to claim 1, characterized in that: The SW module, drawing on the idea of ​​sliding windows, uses a sliding window of length to divide the time series w into multiple windows, where represents the window index and is the total number of windows.

4. The automatic interpretation system of fetal heart and uterine contraction chart based on network representation according to claim 1, characterized in that: The AT module consists of an MSA layer, which includes multiple self-attention layers, called heads, each of which consists of three main components: query (Query, Q), key (Key, K) and value (Value, V). The interaction between Q and K produces an attention score that guides the selection bias of V.

5. The automatic interpretation system of fetal heart and uterine contraction chart based on network representation according to claim 1, characterized in that: The TC module consists of two residual blocks, each of which consists of two dilated causal convolutional layers, each of which is followed by batch normalization and ELU activation function. The TC module can expand its receptive field size exponentially while its parameter amount only increases linearly.

6. A method for automatically interpreting fetal heart rate and uterine contraction chart based on network representation, characterized in that: include: The CV module encodes the low-level spatiotemporal information in the CTG signal through three convolutional layers, and each convolutional layer sets different filters to extract the spatial features in each temporal feature map; The SW module adopts the idea of ​​sliding window. First, the samples of length are divided into windows, then the sliding window and convolution layer are integrated, and finally a convolution calculation is performed on all windows. The AT module is based on an attention score method, which consists of a multi-head attention mechanism, including multiple self-attention layers, where each self-attention layer consists of three different components, namely query (Query, Q), key (Key, K) and value (Value, V). Then, according to their respective functions, the interaction between Q and K produces an attention score that guides the selection bias of V, and finally obtains the most important information in the highlighted time series; The TC module consists of two residual modules, which are used to extract high-level temporal features from time series using a temporal convolutional network and input them into a fully connected layer with a sigmoid classifier.

7. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed in a computer, the computer executes the method described in claim 6.

Images