Method and system for recognizing acceleration and deceleration of fetal heart rate monitoring graph during delivery based on neural network

Abstract

The present invention discloses a method and system for identifying acceleration and deceleration of fetal heart rate monitoring images during delivery based on a neural network. The method comprises: after obtaining a training image set of multiple fetal heart rate monitoring images annotated, cross-checked and reviewed by senior fetal doctors, a model training is performed using the training image set to obtain a 1D-CNN neural network; for the fetal heart rate monitoring images to be identified during delivery, the images are first subjected to signal separation and image preprocessing, and then the trained 1D-CNN neural network is called to perform image recognition processing according to a combination of two scale visual windows and three characteristic images of fetal heart rate, uterine contraction and fetal movement, and the recognition results are acceleration, deceleration and background. The present invention can complete the automatic recognition of acceleration and deceleration, two important fetal monitoring features during delivery, and the accuracy, sensitivity, specificity and F1-score indicators are significantly improved compared with traditional algorithms.

Description

Background Art

[0002] Fetal heart rate monitoring (CTG) is an important tool for prenatal fetal health monitoring. It is used to continuously record the changes in fetal heart rate, the occurrence and pressure of uterine contractions during pregnancy over a period of time, forming a time series graph consisting of three characteristic images: fetal heart rate (FHR), uterine contraction (UC) and fetal movement (FM).

[0003] According to medical guidelines / expert consensus, fetal heart rate monitoring images can interpret multiple types of features such as baseline, baseline variation, acceleration, deceleration, and sine waveform. By accurately identifying the image areas where fetal heart rate acceleration and deceleration (hereinafter referred to as A / D) occur in the intrapartum fetal heart rate monitoring image, and combining the corresponding acceleration and deceleration characteristics, it can assist doctors in assessing the intrauterine health status of the fetus during delivery and provide powerful help in reducing the risk of fetal delivery. For example, combining the interpretation of "deceleration" and "fetal tachycardia" is of great reference value for predicting whether the newborn will have acidosis delivery outcomes. In clinical practice, if certain A / D features are ignored, serious consequences or even fetal death may occur, and excessive interpretation of A / D may lead to unnecessary cesarean sections.

[0004] Since the International Federation of Gynecology and Obstetrics (FIGO) Perinatal Medicine Standards Committee formulated the "Guidelines for Fetal Monitoring (1987 Edition)" in 1985, the industry has been trying to develop various types of algorithms, models and systems based on various computer technologies for decades to achieve automated, rapid, efficient and high-accuracy recognition of various characteristic images and appearance areas such as A / D in intrapartum fetal heart rate monitoring images, thereby assisting doctors to improve the accuracy and consistency of judgments and reduce the heavy clinical workload of doctors.

[0005] At present, several manufacturers of fetal monitoring equipment have built-in some automated interpretation functions in their electronic fetal monitoring equipment (hereinafter referred to as EFM) for clinical use. The core algorithms of these functions are mostly based on the definitions and parameters in medical guidelines / expert consensus such as FIGO, and use "rule-based" signal feature processing (Signal Processing) related technologies. Since the specific characteristics of A / D images are not obvious, the interpretation effect of this type of algorithm is unsatisfactory, and the corresponding "automatic recognition and interpretation" function cannot meet actual clinical needs. (AH de l'Aulnoit, S. Boudet, R. Demailly, A. Delgranche, MG'enin, L. Peyrodie, R. Beuscart, and D. H. de l'Aulnoit, "Automated fetal heart rate analysis for baseline determination and acceleration / deceleration detection: A comparison of methods versus expert consensus", Biomedical Signal Processing and Control, vol.49, pp.113–123, 2019).

[0006] In the past period of time, the industry has also begun to use "deep learning / neural network" technology for research. However, due to the huge workload of manual annotation, the "circular definition" of baseline and A / D in medical guidelines, and the imbalance of sampling data of various feature graphics, there are not many research results, and the relevant results have not been able to enter the more rigorous clinical trial link for further verification. Summary of the invention

[0007] The present invention proposes a method and system for identifying acceleration and deceleration of fetal heart monitoring images during delivery based on a neural network. The method can separate the fetal heart monitoring image into three characteristic images of fetal heart rate, uterine contraction and fetal movement, and use a trained 1D-CNN neural network to identify the characteristic images according to visual windows of different sizes, thereby improving the accuracy, sensitivity and specificity of identifying the fetal heart monitoring image.

[0008] A first aspect of an embodiment of the present invention provides a method for identifying acceleration and deceleration of a fetal heart rate monitoring graph during delivery based on a neural network, the method comprising:

[0009] After obtaining a training image set of a plurality of annotated fetal heart monitoring images, the training image set is used to perform model training to obtain a 1D-CNN neural network;

[0010] After obtaining the intrapartum fetal heart rate monitoring image to be identified, performing signal separation and image preprocessing on the intrapartum fetal heart rate monitoring image to obtain a characteristic image, wherein the characteristic image includes: a fetal heart rate characteristic image, a uterine contraction characteristic image and a fetal movement characteristic image, and the image preprocessing includes: image filtering, curve smoothing and data standardization and regularization;

[0011] The 1D-CNN neural network is called to perform image capture and encoding processing on the feature image according to two-scale visual windows to obtain one-dimensional sequence encoding data, and then recognition processing is performed on the one-dimensional sequence encoding data to obtain an image recognition result.

[0012] In a possible implementation manner of the first aspect, the image recognition result includes: an accelerated image, a decelerated image, and a background image.

[0013] In a possible implementation manner of the first aspect, acquiring a training image set of multiple labeled fetal heart monitoring images includes:

[0014] A plurality of fetal heart rate monitoring images are collected and displayed to obstetric clinicians for annotation by the obstetric clinicians based on medical guidelines and / or expert consensus as interpretation criteria, and cross-checking and reviewing are performed with each other after annotation to obtain a plurality of fetal heart rate monitoring images with annotations, wherein the annotations include: acceleration images, deceleration images, start time and end time of acceleration images, and start time and end time of deceleration images.

[0015] In a possible implementation of the first aspect, scoring parameters of the 1D-CNN neural network training are evaluated, including: accuracy, sensitivity, specificity and F1-score.

[0016] A second aspect of an embodiment of the present invention provides a neural network-based intrapartum fetal heart rate monitoring image acceleration and deceleration recognition system, the system comprising:

[0017] Signal acquisition interface module, used for collecting fetal heart monitoring images;

[0018] A data preprocessing module, used for preprocessing the fetal heart monitoring image, wherein the preprocessing includes: signal separation, image filtering, curve smoothing, and data standardization and regularization;

[0019] AI recognition and judgment module, used to identify the acceleration image, deceleration image and background image from the pre-processed fetal heart monitoring image;

[0020] The data output interface module is used to output the recognized accelerated image, decelerated image and background image to a display screen or output device.

[0021] Compared with the prior art, the embodiment of the present invention provides a method and system for identifying acceleration and deceleration in fetal heart rate monitoring images during delivery based on a neural network, and its beneficial effects are as follows: the present invention separates the image signal into three characteristic images of fetal heart rate, uterine contraction and fetal movement, and after image preprocessing, uses a trained 1D-CNN neural network to identify acceleration and deceleration in the characteristic image according to visual windows of different sizes, and the recognition accuracy, sensitivity and specificity are greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a flowchart of a method for identifying acceleration and deceleration of a fetal heart rate monitoring image during delivery based on a neural network provided by an embodiment of the present invention;

[0023] Figure 2 1 is a schematic diagram of the effect of smoothing a fetal heart monitoring image provided by an embodiment of the present invention;

[0024] Figure 3 It is a comparative analysis diagram of recognition effect scoring parameters obtained by using different residual network layer numbers and fusion strategy combinations provided by an embodiment of the present invention;

[0025] Figure 4 This is a comparison diagram of the recognition effects of the present invention and the SOTA (State-Of-The-Art) algorithm provided by an embodiment of the present invention;

[0026] Figure 5 is a schematic diagram of the structure of a residual block provided by an embodiment of the present invention;

[0027] Figure 6 It is a flowchart of the input fusion strategy processing of fetal heart monitoring images provided by an embodiment of the present invention;

[0028] Figure 7 It is a flowchart of an early fusion strategy processing of fetal heart monitoring images provided by an embodiment of the present invention;

[0029] Figure 8 It is a flowchart of a mid-term fusion strategy processing of fetal heart monitoring images provided by an embodiment of the present invention;

[0030] Fig. 9 It is a flowchart of a late fusion strategy process of fetal heart monitoring images provided by an embodiment of the present invention;

[0031] Fig.10 The present invention is a schematic diagram of a neural network-based acceleration and deceleration recognition system for fetal heart rate monitoring during delivery provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] In order to solve the above problems, a method for identifying acceleration and deceleration of fetal heart rate monitoring images during delivery based on a neural network provided in an embodiment of the present application will be introduced and explained in detail through the following specific embodiments.

[0034] Reference Figure 1 , showing a flow chart of a method for identifying acceleration and deceleration of a fetal heart rate monitoring chart during delivery based on a neural network provided by an embodiment of the present invention.

[0035] Wherein, as an example, the neural network-based method for identifying acceleration and deceleration of a fetal heart rate monitoring graph during delivery may include:

[0036] S10. Obtain fetal heart monitoring images required for training.

[0037] A plurality of fetal heart rate monitoring images are collected and displayed to obstetric clinicians for annotation by the obstetric clinicians based on medical guidelines and / or expert consensus as interpretation criteria. After annotation, the obstetric clinicians cross-check and review each other to obtain a plurality of annotated fetal heart rate monitoring images, which serve as a training image set for training a 1D-CNN neural network, wherein the annotated contents include: an accelerated image, a decelerated image, a start time and an end time of the accelerated image, and a start time and an end time of the decelerated image.

[0038] In one embodiment, there are two types of image annotations, including acceleration and deceleration, to determine whether the image is an accelerated image or a decelerated image.

[0039] Specifically, in order to determine the characteristics of acceleration and deceleration, the start time and end time of acceleration are marked in the acceleration image, and the start time and end time of deceleration are marked in the deceleration image.

[0040] For example, a total of 667 fetal heart rate monitoring images selected from 2019-2021 were annotated, each image had at least one page, and each page had a recording time of 20 minutes. A total of 2,944 pages were annotated, including: 4,018 acceleration events and 2,805 deceleration events.

[0041] In one implementation, three senior doctors and midwives with more than 10 years of experience in obstetrics and gynecology from a tertiary teaching hospital are responsible for the annotation, cross-checking and review of the images.

[0042] It should be noted that the basis for each doctor's marking can be the application guidelines of obstetric clinicians and / or industry consensus, such as: FIGO consensus guidelines on intrapartum fetal monitoring: Cardiotocography (2015), and "2015 Expert Consensus on the Application of Electronic Fetal Heart Monitoring" of the Perinatal Medicine Branch of the Chinese Medical Association.

[0043] In one embodiment, the training image data set is divided into a training set, a validation set, and a test set, wherein 80% of the training data set images are used as the training set and the validation set, and the remaining 20% ​​are used as the test set.

[0044] In one embodiment, the signal separation operation of the present invention may be as follows:

[0045] Reference Figure 1 , showing that before formal model training and image recognition, it is necessary to perform signal separation processing on the training image set and each intrapartum fetal heart rate monitoring image to be identified (hereinafter referred to as "multimodal CTG image").

[0046] In one embodiment, there are three types of characteristic images that require signal separation, namely fetal heart rate, uterine contraction and fetal movement.

[0047] The signal separation is to separate the three characteristic images of fetal heart rate, uterine contraction and fetal movement contained in the multi-modal CTG image separately into three independent characteristic image files.

[0048] Specifically, medical staff routinely view intrapartum fetal heart rate monitoring images through the EFM screen or the "paper tape" output by the printer connected to the EFM. The image will contain three characteristic images of fetal heart rate, fetal movement and uterine contraction at the same time. Manual interpretation requires the combination of the characteristics of these three characteristic images to determine whether there are acceleration, deceleration and other intrapartum fetal monitoring characteristics. Separating the three characteristic images separately through signal separation will help with subsequent processing and analysis.

[0049] Specifically, the built-in data export function of EFM can be used to export the proprietary format data stored in the device by each EFM manufacturer into a graphic-like file (such as .PDF) that can be directly visualized and read by doctors and complies with medical guidelines and / or expert consensus requirements. Then, based on the position, waveform and other characteristic information of the text, graphics, data, etc. in the graphic file, it can be converted into a graphic dot matrix format file.

[0050] In one embodiment, the image preprocessing operation of the present invention may be as follows:

[0051] Reference Figure 1 , showing that before formal model training and image recognition, the three characteristic images of the multimodal CTG image that have completed "signal separation" need to be preprocessed separately.

[0052] In one embodiment, the image preprocessing required includes: image filtering, curve smoothing, and data standardization and regularization.

[0053] In one embodiment, the signals of the training image set and the intrapartum fetal heart rate monitoring image to be identified are obtained by sampling the signals at a fixed frequency, and each signal sampling is a frame.

[0054] For example, the signal sampling frequency is 8 Hz, that is, the signal is sampled 8 times per second.

[0055] The image filtering is to remove the obvious noise interference signal in the fetal heart rate characteristic image.

[0056] Specifically, for the fetal heart rate characteristic image, between adjacent signal frames, the fetal heart rate signal with an increase or decrease of more than 25 bpm is considered as a noise interference signal, which can be filtered and excluded using standard filtering techniques in the art. Through the above operation, the noise signal exceeding 25 bpm between two adjacent signal frames can be removed.

[0057] The curve smoothing is to smooth out some small high-frequency signals in the fetal heart rate characteristic image and the uterine contraction characteristic image.

[0058] Reference Figure 2 , which shows a schematic diagram of the effect of smoothing a fetal heart monitoring image provided by an embodiment of the present invention.

[0059] Specifically, according to the technology in this field, most of the tiny local fluctuations in the signal come from the noise generated by the equipment. Therefore, the present invention performs curve smoothing processing on the fetal heart rate characteristic image and the uterine contraction characteristic image. Figure 2 , Figure 2 a in the figure is the curve without smoothing. Figure 2 The b in the figure is the smoothed curve. Figure 2 In a and b, the three curves from top to bottom are the fetal heart rate characteristic image, uterine contraction characteristic image and fetal movement characteristic image.

[0060] The data standardization and regularization is to standardize the three characteristic signals so that they can be within a certain range. The characteristic values ​​of the fetal heart rate characteristic image and the uterine contraction characteristic image are integer values, and the characteristics of the uterine contraction characteristic image are binary. Therefore, they need to be standardized and regularized to ensure the stability of the model training process.

[0061] Specifically, the data of each channel are standardized using the empirical mean and standard deviation, respectively, and then normalized to the range of [0.5, 0.5].

[0062] Specifically, using data augmentation techniques to increase and diversify training data, a popular machine learning technique that helps reduce overfitting of data-driven models. This data augmentation process can be applied only to the training set and not to the validation or test sets.

[0063] In one embodiment, the structure and processing steps of the 1D-CNN neural network of the present invention can be as follows: Figure 1 As shown, please refer to the following 1D-CNN neural network structure and internal data processing analysis.

[0064] The CNN (Convolutional Neural Network) is a type of deep learning neural network structure, more precisely, a type of feedforward neural network that includes convolution calculations and has a deep structure. Convolutional neural networks are proposed based on the biological mechanism of receptive fields. Convolutional neural networks are neural networks specifically used to process data with a grid-like structure. For example, time series data (which can be considered as a one-dimensional grid formed by regular sampling on the time axis) and image data (which can be considered as a two-dimensional pixel grid) are mainly used in image recognition and computer vision tasks. Generally, CNN uses a convolutional layer to extract the features of the input data, and reduces the number of parameters and the amount of calculation through a pooling layer, and finally performs classification or regression prediction through a fully connected layer. The advantage of CNN is that it can automatically learn multi-level abstract features, has good performance and generalization capabilities, and has been successfully applied in many fields.

[0065] The 1D-CNN (1D Convolutional Neural Network) is a variant of the convolutional neural network. 1D-CNN is mainly used to process one-dimensional sequence data, such as time series images, audio, text, etc. Compared with the traditional fully connected neural network, 1D-CNN can better process local relations in sequence data, so it performs better in tasks such as speech recognition, natural language processing, and time series prediction.

[0066] In one embodiment, the 1D-CNN neural network includes an input layer, a convolutional layer, a residual network and a fully connected layer connected in sequence.

[0067] As an example, the recognition process using the 1D-CNN neural network of the above structure may include the following steps:

[0068] The first step is to use the one-dimensional sequence encoding data as the input layer.

[0069] The second step is to use the convolutional layer to convert the one-dimensional sequence encoding data into a feature vector.

[0070] The third step is to use the residual network to optimize the 1D-CNN to improve the processing efficiency of the 1D-CNN.

[0071] The fourth step is to aggregate and fuse the feature vectors at different stages to form an integrated feature vector based on a fusion strategy.

[0072] Step 5: Use the fully connected layer to convert the feature vector into a distribution category probability value, and determine the image recognition result according to the distribution category probability value.

[0073] The first step is to use the one-dimensional sequence encoding data as the input layer.

[0074] In one embodiment, the input layer data of the 1D-CNN neural network of the present invention is obtained through two steps: selecting an image window to be processed, and generating visual window encoding data of two scale windows.

[0075] Step 1: Select the image window to be processed.

[0076] In one embodiment, for a multimodal CTG image that has completed signal separation and image preprocessing, with a certain time point as the starting time and a fixed number of frames (i.e., duration), three image segments to be processed (trained or recognized) are respectively extracted from three feature images to form a "window of image to be processed".

[0077] Specifically, a sliding window method is used to select an image window to be processed from the starting point of the multimodal CTG image, and then enter the subsequent training or recognition processing. After each window is processed, a number of frames are slid along the time axis, and then another image window is selected. Repeat the above operation until the multimodal CTG image is processed.

[0078] Specifically, each selected image window is 512 frames (adopting a frequency of 8 Hz, equivalent to 64 seconds), and each sliding duration is 25 frames.

[0079] Step 2: Generate large-scale visual window encoding data and local-scale visual window encoding data according to two-scale visual windows.

[0080] Reference Figure 1 For a "processed image window", "visual window encoding data" for training or recognition is generated according to the combination of two scale visual windows and three characteristic images of fetal heart rate, uterine contraction and fetal movement.

[0081] In one embodiment, the two scales of visual windows are respectively: a large-scale visual window and a local-scale visual window.

[0082] Specifically, the large-scale visual window is for a "processed image window". In the three characteristic images corresponding to the "multimodal CTG map", two times more images are collected on the left and right sides of the window to construct large-scale visual window encoding data. If the time and space on the left and right sides are sufficient, the duration of the large-scale visual window encoding data is 512x5=2,560 frames.

[0083] Specifically, the local scale visual window is for a "processed image window", in the three feature images corresponding to the "multimodal CTG map", on the left and right sides of the window, each collects 1 times the duration of the image, to construct the local scale visual window encoding data. If the time space on the left and right sides is sufficient, the duration of the local scale visual window encoding data is 512x3=1,536 frames.

[0084] Specifically, this is equivalent to that for every 512 frames of data, the large-size window takes into account the features of the 1024 frames of data before and after it for training and recognition, and the local-size window takes into account the features of the 512 frames of data before and after it for training and recognition.

[0085] Specifically, three sets of large-scale visual window encoding data (fetal heart rate, uterine contraction and fetal movement) and three sets of local-scale visual window encoding data (fetal heart rate, uterine contraction and fetal movement) are generated.

[0086] The second step is to use the convolutional layer to convert the one-dimensional sequence encoding data into a feature vector.

[0087] In one embodiment, the convolution layer of the 1D-CNN neural network of the present invention is composed of several convolution units, and the parameters of each convolution unit are optimized by the back propagation algorithm. The purpose of the convolution operation is to extract different features of the input. The first convolution layer may only extract some low-level features such as edges, lines, and corners. More layers of the network can iteratively extract more complex features from low-level features.

[0088] Specifically, the number of convolution kernels of the 1D-CNN in this embodiment is 5, and the stride is 2.

[0089] For all convolutional layers of the 1D-CNN neural network, the output eigenvalue Z ij The calculation of is as follows:

[0090]

[0091] Among them, Z ij is the output feature value at the time axis position i in the jth feature map, X i is the input feature value at position i on the time axis, W j is the shared weight coefficient of the jth filter, b j is the bias weight coefficient of the jth filter.

[0092] In this embodiment, the 1D-CNN neural network needs to be classified and labeled during the training process; this is accomplished in the following two steps:

[0093] In step 1, a 30% threshold is applied to distinguish between background and non-background categories. That is, when the percentage of acceleration or deceleration frames in all frames exceeds 30%, they are classified as acceleration or deceleration even if the background category accounts for the highest proportion.

[0094] Step 2: Once the conditions in the previous step are met, the class with the highest occurrence ratio can be used as the label of the "image window to be processed".

[0095] After these two steps, the number of samples in the background class is much larger than that of the other two classes. In order to rebalance the data volume of background and A / D, the background class data needs to be downsampled and the deceleration class data needs to be oversampled to match the data volume of the acceleration class.

[0096] The third step is to use the residual network to optimize the 1D-CNN to improve the processing efficiency of the 1D-CNN.

[0097] In one embodiment, the residual network (ResNet) of the 1D-CNN neural network improves the flow of gradients problem in CNN training through shortcut connections.

[0098] In one embodiment, the residual network of the 1D-CNN neural network is composed of one or more residual blocks (ResidualBlock).

[0099] Reference Figure 5 , which shows a schematic structural diagram of the residual block.

[0100] Specifically, the number of residual network layers in the CNN neural network has an important impact on the design of the CNN neural network structure. If the number of residual network layers is too small, the CNN's ability to learn complex features will be insufficient; conversely, it may cause overfitting of the CNN.

[0101] In an optional embodiment, the residual network of the 1D-CNN neural network has two or three layers.

[0102] The fourth step is to aggregate and fuse the feature vectors at different stages to form an integrated feature vector based on a fusion strategy.

[0103] In one embodiment, the fusion strategy of the 1D-CNN neural network of the present invention is used to extract feature vectors located in different layers of the CNN from the data encoding of the three signal channels, and then fuse the extracted features to form an integrated feature vector, thereby completing the relevant pattern feature learning.

[0104] Specifically, the three signal channels are: fetal heart rate, uterine contraction and fetal movement.

[0105] Specifically, the fusion strategy can be: input fusion strategy, early fusion strategy, mid-term fusion strategy and late fusion strategy, and one of them is selected when it is implemented in the CNN network.

[0106] Specifically, refer to Figure 6-9 , respectively showing a flow chart of the input fusion strategy processing of the fetal heart monitoring image provided by an embodiment of the present invention, a flow chart of the early fusion strategy processing of the fetal heart monitoring image provided by an embodiment of the present invention, a flow chart of the mid-term fusion strategy processing of the fetal heart monitoring image provided by an embodiment of the present invention, and a flow chart of the late fusion strategy processing of the fetal heart monitoring image provided by an embodiment of the present invention.

[0107] like Figure 6 As shown in the figure, the input fusion strategy directly fuses the encoded data of the three signal channels of fetal heart rate, uterine contraction and fetal movement in the input layer as feature vectors, and the resulting integrated feature vector is used as the input of the subsequent convolutional layer of the CNN network. In this method, the CNN network is not used to process the signal channel before fusion.

[0108] like Figure 7 As shown in the figure, the early fusion strategy fuses the feature vectors generated by the three signal channels after the first convolutional layer is processed, and the resulting integrated feature vector is used as the input of the first layer of the residual network. In this method, before fusion, each signal channel has been processed by its own separate convolutional layer to generate its own modal features.

[0109] like Figure 8 As shown in the figure, the mid-term fusion strategy, after a certain layer of residual network processing, fuses the feature vectors generated by the three signal channels, and the resulting integrated feature vector is used as the input of the next layer of residual network. In this method, before fusion, each signal channel has been processed by its own separate convolutional layer and several layers of residual network to generate its own modal features.

[0110] like Fig. 9 As shown in the figure, the late fusion strategy fuses the feature vectors generated by the three signal channels after the last layer of residual network processing, and the resulting integrated feature vector is used as the input of the fully connected layer. In this method, before fusion, each signal channel has been processed by its own separate convolutional layer and multi-layer residual network to generate its own modal features.

[0111] Specifically, visual windows of different scales encode data and use the same fusion strategy.

[0112] Specifically, according to the experimental test results, the 1D-CNN adopts the "two-layer residual network + mid-term fusion strategy" to obtain the best comprehensive performance. Figure 3 shown.

[0113] Step 5: Use the fully connected layer to convert the feature vector into a distribution category probability value, and determine the image recognition result according to the distribution category probability value.

[0114] In one embodiment, the fully connected layer of the 1D-CNN neural network of the present invention is used to convert the two-dimensional feature map (matrix) output by the convolutional layer processed by the residual network / fusion strategy into a one-dimensional feature vector, and convert the feature vector into a category probability distribution after processing.

[0115] Specifically, the 1D-CNN neural network in one embodiment has three fully connected layers, with 1024, 256 and 3 neurons respectively, and the number of neurons in the last fully connected layer is equal to the number of predicted categories.

[0116] Specifically, the classification results of the image recognition include three categories, including: accelerated image, decelerated image and background image.

[0117] Specifically, the distribution category probability value refers to the probability that a "processed image window" belongs to the background, acceleration, and deceleration categories respectively. The sum of these three probabilities is equal to 1. For example, it can be [0.7, 0.12, 0.18] (will be classified as background), [0.2, 0.6, 0.2] (will be classified as acceleration), [0.4, 0.1, 0.5] (will be classified as deceleration).

[0118] S11. Using the training image set to perform model training to obtain a 1D-CNN neural network.

[0119] In one embodiment, a training image set (divided into a training set and a validation set) that has undergone signal separation and image preprocessing is used and sent to the 1D-CNN neural network for model training;

[0120] Specifically, during the training phase, the "Focal loss function" is used to optimize the weights of the CNN model through back propagation. If the loss gap between the training set and the validation set increases, it means that overfitting has occurred. At this point, the training will stop and the weights will be saved, and then the scoring parameters of the model will be calculated on the test set.

[0121] Specifically, refer to Figure 3 , shows a schematic diagram of comparing scoring parameters of training effects using different residual network layers and different fusion strategy combinations during model training in one embodiment.

[0122] In one embodiment, the loss between the training set and the validation set can be used to determine the training epochs of the model. Specifically, the epochs of each model training are set to 65.

[0123] Specifically, the number of data sent to the model for training in each batch (batch size) is 64.

[0124] Specifically, the relevant parameters of the Adam optimizer are as follows: the learning rate is initialized to 0.1 and decreases by 0.001 every 15 training generations.

[0125] S12: Obtaining an intrapartum fetal heart rate monitoring image to be identified.

[0126] In one embodiment, the relevant format of the intrapartum fetal heart rate monitoring image to be identified should be completely consistent with the "training image set" used to perform the 1D-CNN network.

[0127] Specifically, the format should be a graphic-like file (such as .PDF) that can be directly visualized and read by doctors and complies with medical guidelines and / or expert consensus requirements. In addition, the location of text, graphics, data, waveforms and other characteristic information in the graphic file should be completely consistent with the "training image set".

[0128] After the intrapartum fetal heart rate monitoring image to be identified is acquired, signal separation and image preprocessing may be performed on the intrapartum fetal heart rate monitoring image to obtain a characteristic image.

[0129] In one embodiment, the characteristic images may include: fetal heart rate characteristic images, uterine contraction characteristic images and fetal movement characteristic images, and the image preprocessing includes: image filtering, curve smoothing and data standardization and regularization.

[0130] It should be noted that the operation procedures of the signal separation and the image preprocessing can be the same as the above analysis content. For details, please refer to the above analysis content. In order to avoid repetition, it will not be repeated here.

[0131] S13, calling 1D-CNN neural network for recognition processing.

[0132] Optionally, the 1D-CNN neural network can be called to perform image capture and encoding processing on the feature image according to two-scale visual windows to obtain one-dimensional sequence encoding data, and then recognition processing is performed on the one-dimensional sequence encoding data to obtain an image recognition result.

[0133] In one implementation, the two scales of visual windows are respectively: a large-scale visual window and a local-scale visual window.

[0134] Specifically, the three characteristic images may be respectively subjected to image capture and encoding processing according to the large-scale visual window and the local-scale visual window to obtain one-dimensional sequence encoding data, wherein the one-dimensional sequence encoding data includes: three groups of large-scale encoding data and three groups of local-scale encoding data.

[0135] In one embodiment, the signal separation, image preprocessing and 1D-CNN network structure and processing of the recognition process are the same as the signal separation, image preprocessing and 1D-CNN network structure and processing of the model training process. For details, please refer to the above analysis description. In order to avoid repetition, they will not be repeated here.

[0136] In one embodiment, in order to determine whether the training effect of the trained 1D-CNN neural network meets the requirements, the results of recognition processing of the "test set" in the training image set can be used to calculate the scoring parameters of the 1D-CNN neural network, and the scoring parameters include: accuracy, sensitivity, specificity and F1-score.

[0137] The accuracy, also called correctness, is the percentage of correctly predicted samples.

[0138] The sensitivity, also known as sensitivity or true positive rate, is the ratio of true positive samples that are correctly predicted to be positive. The higher the sensitivity, the lower the false positive rate.

[0139] The specificity, also called the true negative rate, is the ratio of true negative samples that are correctly predicted as negative by the model. The higher the specificity, the lower the false negative rate.

[0140] The F1-score is the harmonic mean of precision and recall.

[0141] Among the above parameters, true positive, true negative, false positive, and false negative are basic data, as follows:

[0142] True Positive: TP, true positive.

[0143] True Negative: TN, true negative.

[0144] False Positive: FP, false positive.

[0145] False Negative: FN, false negative.

[0146] Specifically, the accuracy rate (ACC, also known as correctness rate) is calculated as shown in the following formula:

[0147] ACC=(TP+TR) / (TP+FP+FN+TN).

[0148] The sensitivity (SEN, also known as the true positive rate) is calculated as follows:

[0149] SEN = TP / (TP+FN).

[0150] The specificity (SPE, also known as true negative rate) is calculated as follows:

[0151] SPE = TN / (TN + FP).

[0152] The calculation of the F1-score (F1-score, the harmonic mean of precision and recall) is as follows:

[0153]

[0154] In the above formula, the precision is Precision, which is calculated as follows:

[0155] Precision = TP / (TP+FP). For the prediction results, the denominator is all positive judgments.

[0156] The recall rate is Recall, which is calculated as follows:

[0157] Recall = TP / (TP+FN). For statistical samples, the denominator is all positives in the sample, which measures whether there are omissions.

[0158] Reference Figure 4, showing a comparison of the recognition of acceleration and deceleration by the present invention provided in one embodiment and the recognition of acceleration and deceleration by the currently optimal SOTA (State-Of-The-Art) signal feature processing algorithm.

[0159] Specifically, by comparing the method of the present invention with the SOTA algorithm, the specificity and F1-score indicators of the present invention are significantly better than those of the SOTA algorithm; among which, the F1-score is improved by 13.75% in accelerated recognition and 15.47% in decelerated recognition. Figure 4 shown.

[0160] In this embodiment, the embodiment of the present invention provides a method for identifying acceleration and deceleration of fetal heart rate monitoring images during delivery based on a neural network, and its beneficial effects are: the present invention can separate the image signal into three characteristic curves of fetal heart rate, uterine contraction and fetal movement, and use the trained 1D-CNN neural network to identify the characteristic images according to visual windows of different sizes, thereby improving the accuracy, sensitivity and specificity of identifying fetal heart rate monitoring images.

[0161] The embodiment of the present invention also provides a neural network-based fetal heart rate monitoring image acceleration and deceleration recognition system. Fig.10 , showing a schematic structural diagram of a neural network-based acceleration and deceleration recognition system for fetal heart rate monitoring during delivery provided by an embodiment of the present invention.

[0162] Wherein, as an example, the neural network-based intrapartum fetal heart rate monitoring chart acceleration and deceleration recognition system may include:

[0163] Signal acquisition interface module, used for collecting fetal heart monitoring images;

[0164] A data preprocessing module, used for preprocessing the fetal heart monitoring image, wherein the preprocessing includes: signal separation, image filtering, curve smoothing, and data standardization and regularization;

[0165] AI recognition and judgment module, used to identify the acceleration image, deceleration image and background image from the pre-processed fetal heart monitoring image;

[0166] The data output interface module is used to output the recognized accelerated image, decelerated image and background image to a display screen or output device.

[0167] Specifically, the signal acquisition interface module, the data preprocessing module, the AI ​​recognition and judgment module and the data output interface module can be connected in sequence to form a neural network-based intrapartum fetal heart rate monitoring acceleration and deceleration recognition system, which is installed in the EFM device to assist in image recognition. Fig.10 shown.

[0168] Those skilled in the art can clearly understand that, for the sake of convenience and brevity of description, the specific working process of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0169] Furthermore, an embodiment of the present application also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for identifying acceleration and deceleration of fetal heart rate monitoring images during delivery based on a neural network is implemented as described in the above embodiment.

[0170] Furthermore, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer-executable program, and the computer-executable program is used to enable a computer to execute the neural network-based acceleration and deceleration recognition method for the intrapartum fetal heart rate monitoring chart as described in the above embodiment.

[0171] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for identifying acceleration and deceleration of fetal heart rate monitoring images during delivery based on a neural network, characterized in that: include: After obtaining a training image set of a plurality of annotated fetal heart monitoring images, the training image set is used to perform model training to obtain a 1D-CNN neural network; After obtaining the intrapartum fetal heart rate monitoring image to be identified, performing signal separation and image preprocessing on the intrapartum fetal heart rate monitoring image to obtain a characteristic image, wherein the characteristic image includes: a fetal heart rate characteristic image, a uterine contraction characteristic image and a fetal movement characteristic image, and the image preprocessing includes: image filtering, curve smoothing and data standardization and regularization; Calling the 1D-CNN neural network, performing image interception and encoding processing on the feature image according to the visual windows of two scales, obtaining one-dimensional sequence encoding data, and then performing recognition processing on the one-dimensional sequence encoding data to obtain an image recognition result; The 1D-CNN neural network includes an input layer, a convolutional layer, a residual network and a fully connected layer connected in sequence; The identification process comprises: Using the one-dimensional sequence encoded data as input of the input layer; Converting the one-dimensional sequence encoding data into a feature vector using the convolutional layer; Utilizing the residual network to optimize the 1D-CNN; Based on the fusion strategy, the feature vectors at different stages are aggregated and fused to form an integrated feature vector; Using the fully connected layer to convert the feature vector into a distribution category probability value, and determining an image recognition result according to the distribution category probability value; The two scales of visual windows are: a large scale visual window and a local scale visual window; The calling of the 1D-CNN neural network to perform image interception and encoding processing on the feature image according to the visual windows of two scales to obtain one-dimensional sequence encoding data includes: According to the large-scale visual window and the local-scale visual window, the three characteristic images are respectively subjected to image interception and encoding processing to obtain one-dimensional sequence encoding data, wherein the one-dimensional sequence encoding data includes: three groups of large-scale encoding data and three groups of local-scale encoding data; The image recognition result includes: an accelerated image, a decelerated image and a background image.

2. The neural network-based acceleration and deceleration recognition method for intrapartum fetal heart rate monitoring according to claim 1, characterized in that: There are three kinds of characteristic images for signal separation, namely, fetal heart rate, uterine contraction and fetal movement.

3. The neural network-based acceleration and deceleration recognition method for intrapartum fetal heart rate monitoring according to claim 1, characterized in that: The residual network of the 1D-CNN neural network has two or three layers.

4. The neural network-based acceleration and deceleration recognition method for intrapartum fetal heart rate monitoring according to claim 1, characterized in that: The fusion strategy includes: input fusion strategy, early fusion strategy, mid-term fusion strategy and late fusion strategy.

5. The neural network-based method for identifying acceleration and deceleration of fetal heart rate monitoring images during delivery according to any one of claims 1 to 4, characterized in that: The step of obtaining a training image set of multiple annotated fetal heart monitoring images includes: A plurality of fetal heart rate monitoring images are collected and displayed to obstetric clinicians for annotation by the obstetric clinicians based on medical guidelines and / or expert consensus as interpretation criteria, and cross-checking and reviewing are performed with each other after annotation to obtain a plurality of fetal heart rate monitoring images with annotations, wherein the annotations include: acceleration images, deceleration images, start time and end time of acceleration images, and start time and end time of deceleration images.

6. The neural network-based acceleration and deceleration recognition method for intrapartum fetal heart rate monitoring according to claim 5, characterized in that: Scoring parameters for evaluating the completion of the 1D-CNN neural network training include: Accuracy, sensitivity, specificity and F1-score.

7. A neural network-based intrapartum fetal heart rate monitoring acceleration and deceleration recognition system, characterized in that: include: Signal acquisition interface module, used for collecting fetal heart monitoring images; A data preprocessing module, used for preprocessing the fetal heart monitoring image, wherein the preprocessing includes: signal separation, image filtering, curve smoothing, and data standardization and regularization; AI recognition and judgment module, used to identify the accelerated image, decelerated image and background image from the pre-processed fetal heart monitoring image; A data output interface module, used to output the recognized accelerated image, decelerated image and background image to a display screen or output device; The AI ​​recognition and judgment module recognizes the accelerated image, the decelerated image and the background image, and processes the image as follows: Performing image interception and encoding processing on the preprocessed fetal heart monitoring image to obtain one-dimensional sequence encoding data, and using the one-dimensional sequence encoding data as input of the input layer; The pre-processed fetal heart monitoring image is converted into a feature vector using a convolutional layer; Use residual network to optimize 1D-CNN; Based on the fusion strategy, the feature vectors at different stages are aggregated and fused to form an integrated feature vector; The feature vector is converted into a distribution category probability value by using a fully connected layer, and an accelerated image, a decelerated image and a background image are determined according to the distribution category probability value, wherein the accelerated image, the decelerated image and the background image are image recognition results; The method of performing image interception and encoding processing on the pre-processed fetal heart monitoring image to obtain one-dimensional sequence encoding data includes: According to the large-scale visual window and the local-scale visual window, the fetal heart monitoring image is respectively captured and encoded to obtain one-dimensional sequence encoding data, wherein the one-dimensional sequence encoding data includes: three groups of large-scale encoding data and three groups of local-scale encoding data, and the preprocessing includes: image filtering, curve smoothing, and data standardization and regularization.

Images