A method for extracting standard sections of fetal heart ultrasound based on human-in-the-loop intelligent assisted navigation
Through the intelligent assisted navigation method of human-in-loop mode, combined with multi-target area recommendation network and deep Q network, the problem of the dependence and labeling burden of fetal heart ultrasound standard section extraction method on expert knowledge and achieve efficient and accurate fetal Standard section extraction and navigation of cardiac ultrasound improves the efficiency of prenatal ultrasound diagnosis.
Patent Information
- Application Number
- CN202111176628.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-09
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2041-10-09
AI Technical Summary
The existing standard sectional extraction method for fetal heart ultrasound is highly dependent on expert knowledge and is difficult to promote. The high demand for later labeling leads to a heavy burden on doctor data labeling, affecting diagnostic efficiency.
Using an intelligent assisted navigation method based on human-in-loop mode, we use ultrasonic image data sets, multi-target area recommendation networks, Faster-RCNN networks and deep Q networks to carry out reinforcement learning in combination with doctor feedback to realize automatic extraction and navigation of ultrasonic standard sections.
It reduces the diagnostic load of doctors in ultrasound scans, improves the accuracy and efficiency of standard section extraction of fetal heartbeat ultrasound, reduces the workload of later labeling, and enhances navigation performance.
Smart Images

Figure CN113936775B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of computer-aided diagnosis, and in particular relates to a method for extracting standard sections of fetal heart ultrasound. Background Art
[0002] Congenital heart disease is one of the most common types of congenital defects in newborns, with an incidence rate of 0.8%-1.1% among newborns, ranking first among various birth defects. According to relevant statistics, congenital heart disease is not only the neonatal disease with the highest incidence rate in my country, but also the leading cause of infant death in my country. For congenital heart disease, early diagnosis of fetal heart rate through prenatal screening and relevant prenatal intervention measures can effectively reduce the incidence of congenital heart disease after the fetus is born. Prenatal screening of fetal congenital heart disease is mainly divided into two methods: puncture examination and imaging examination. Due to the non-invasive advantage of imaging examination, it has become the mainstream method of prenatal screening. Among the various existing imaging technologies, CT imaging has the disadvantage of radiation, while magnetic resonance imaging technology cannot be used for real-time examination and is easily affected by fetal movement and produces artifacts. In contrast, ultrasound imaging technology has the advantages of no radiation and low cost, and can effectively observe the fetal heart rate in real time during prenatal examinations. Existing studies have shown that in prenatal ultrasound testing of the fetus, it is necessary not only to search for the fetal heart area in the ultrasound image, but also to extract the standard ultrasound section of the fetal heart. The shape of the fetal heart shown by the standard ultrasound section plays an important guiding role in doctors' diagnosis of congenital heart disease.
[0003] In the existing research on extraction of standard sections of fetal heart ultrasound, the related research on extraction of standard sections of fetal heart ultrasound is still in its infancy. For the automatic extraction of standard sections of ultrasound, the existing methods mainly use the method of manually designing image features, and then use classifiers to discriminate images that meet the standard sections of ultrasound. Among them, the team of Professor Ni Dong of Shenzhen University automatically locates the standard section of the fetal abdomen in ultrasound images through a selective search method based on the radial component model. For the standard section of the fetal face, the team of Professor Wang Tianfu of Shenzhen University further extracts ultrasound image features through Fisher vector to realize automatic recognition of the standard section of the fetal face. However, the above methods all require manual design of ultrasound image features, and have too high a reliance on expert knowledge during the implementation process, which makes it difficult to promote them among many prenatal examination subjects with large individual differences.
[0004] Since the extraction of standard sections of fetal heart ultrasound is highly dependent on the clinical experience of ultrasound doctors, when facing large-scale prenatal screening, experienced ultrasound doctors are not only in short supply, but also face the problem of missed diagnoses due to fatigue and other reasons. Even for ultrasound doctors with a certain degree of experience, it is still difficult to effectively capture the standard sections of the fetal heart in the case of frequent fetal movements. Therefore, in prenatal ultrasound diagnosis of the fetus, the introduction of auxiliary navigation and image analysis systems for the extraction of standard sections of fetal heart ultrasound not only lowers the experience threshold of doctors in ultrasound scanning, but also effectively reduces the interference of doctors' subjective factors on the examination results. The introduction of intelligent auxiliary navigation extraction methods for ultrasound standard sections in fetal heart diagnosis can greatly alleviate the workload of medical technicians and effectively improve the screening efficiency of prenatal ultrasound diagnosis.
[0005] In order to overcome the high dependence of the automatic extraction model of ultrasound standard sections on expert knowledge, recent studies have gradually adopted a data-driven approach and a deep learning method to construct ultrasound standard section discrimination models. Among them, the automatic extraction method of standard sections based on deep convolutional neural networks has been initially tried on the data of fetal abdomen "Chen H, Ni D, Qin J, et al. Standard plane localization in fetal ultrasound via domain transferred deep neural networks [J]. IEEE journal of biomedical and health informatics, 2015, 19 (5): 1627-1636" and fetal face "Yu Z, Tan EL, Ni D, et al. A deep convolutional neural network-based framework for automatic fetal facial standard plane recognition [J]. IEEE journal of biomedical and health informatics, 2017, 22 (3): 874-885". On this basis, recent studies have further combined convolutional neural networks with recurrent neural networks to identify standard sections of ultrasound images, such as the paper "Chen H, Wu L, Dou Q, et al. Ultrasound standard plane detection using a composite neural network framework [J]. IEEE transactions on cybernetics, 2017, 47 (6): 1576-1586". Given that the current automatic recognition model for ultrasound standard sections is still some distance away from practical application, Baumgartner et al. in the paper "Baumgartner CF, Kamnitsas K, Matthew J, et al. SonoNet: real-time detection and localisation of fetal standard scan planes in freehand ultrasound [J]. IEEE transactions on medical imaging, 2017, 36 (11): 2204-2215" further considered the running rate when designing a deep learning model, and proposed a real-time standard section automatic recognition algorithm.However, the above methods have extremely high requirements for the post-annotation of ultrasound images. Annotating the accumulated massive ultrasound data will consume a lot of time and energy of ultrasound doctors outside of work. For example, in the existing research on ultrasound standard plane discrimination based on deep learning, the annotation requirements for ultrasound images are as high as more than 50,000, which imposes a serious data annotation burden on doctors in actual clinical practice, such as the literature "Chen H, Wu L, Dou Q, et al. Ultrasound standard plane detection using a composite neuralnetwork framework[J]. IEEE transactions on cybernetics, 2017, 47(6): 1576-1586".
[0006] In contrast, if human-computer interaction between doctors and intelligent systems is realized during the scanning process of ultrasound doctors, the workload of doctors' later annotation of images can be greatly saved by receiving doctors' evaluation of images in real time. In recent years, human-computer hybrid intelligent technology in the human-in-the-loop mode has gradually emerged. If the human-in-the-loop mode is introduced into ultrasound scanning, and the doctor's evaluation of ultrasound images is fed back to the system in real time, reinforcement learning of the automatic extraction model of ultrasound standard sections can be realized in the system. At the same time, the navigation system after reinforcement learning can also intelligently navigate the doctor's ultrasound scanning in the human-in-the-loop mode, thereby assisting doctors in searching and extracting standard sections. Therefore, for the large-scale prenatal screening needs of fetuses, the intelligent assisted navigation extraction method for extracting ultrasound standard sections in the human-in-the-loop mode can not only effectively reduce the diagnostic burden of doctors in ultrasound scanning, but also help promote the development of related industries of medical auxiliary system technology. Summary of the invention
[0007] In order to overcome the shortcomings of the prior art, the present invention provides a method for extracting standard sections of fetal heart ultrasound based on human-in-the-loop intelligent assisted navigation. First, an ultrasound image data set for marking the fetal heart position is constructed; then a multi-target region recommendation network is used to determine the candidate region of the fetal heart; next, the fetal heart region is located based on the Faster-RCNN network; then a human-in-the-loop system is constructed, including a doctor feedback unit and a machine navigation unit; finally, a reinforcement learning method for human-in-the-loop intelligent assisted navigation is used to realize the extraction of standard sections of fetal heart ultrasound based on the human-in-the-loop mode. The present invention performs reinforcement learning on the standard section discrimination model of the system, continuously improves the system's ultrasonic standard section discrimination ability, and thus realizes the gradual enhancement of navigation performance.
[0008] The technical solution adopted by the present invention to solve the technical problem includes the following steps:
[0009] Step 1: Construct an ultrasound image dataset for marking the fetal heart position;
[0010] Step 1-1: Collect ultrasound image data in DICOM format, convert the data format, and convert the ultrasound image data into TIF format images;
[0011] Step 1-2: Mark the fetal heart and the positions of organs around the fetal heart in the TIF format image;
[0012] Step 1-3: Convert the TIF format images annotated in step 1-2 into the PASCAL VOC annotation format to form an ultrasound image dataset;
[0013] Step 2: Use a multi-target region recommendation network to determine the candidate fetal heart region;
[0014] Step 2-1: Use the RPN link under the Faster-RCNN framework as a multi-target region recommendation network;
[0015] Objective function L MRPN As shown in formula (1):
[0016]
[0017] Where, L class L is the average of the attention loss of the fetal heart and organs around the fetal heart. loc is the sum of the average spatial distance loss between the fetal heart and the organs around the fetal heart; λ is the adjustment factor used to adjust L class and L loc The order of magnitude is used to adjust the loss weight ratio; i represents the index of the image, N class Indicates the number of images for which fetal heart rate is known, p i Indicates whether the i-th image is a known 0 / 1 label of the fetal heart. represents the predicted probability of whether the i-th picture is a fetal heart, N loc Indicates the number of coordinate values of the known fetal heart in the image space position calibration frame, t i Represents the known coordinate value of the fetal heart position calibration frame of the i-th image, Represents the predicted coordinate value of the fetal heart position calibration frame of the i-th image by PRN;
[0018] Step 2-2: The ultrasound image dataset constructed in step 1 is used to train the multi-target region recommendation network, and the output of the multi-target region recommendation network is the candidate regions of the fetal heart and the candidate regions of the organs around the fetal heart and the corresponding attention weights;
[0019] Step 2-3: For the overlapping parts of the candidate regions of the organs surrounding the fetal heart and the candidate regions of the fetal heart output by the multi-target region recommendation network, the output attention weights of the candidate regions of the organs surrounding the fetal heart and the output attention weights of the candidate regions of the fetal heart are compared, and the regions whose attention weights of the candidate regions of the fetal heart are smaller than those of the candidate regions of the organs surrounding the fetal heart are excluded, and the regions that are not excluded are used as the final candidate regions of the fetal heart after the anatomical structure is corrected;
[0020] Step 3: Fetal heart region positioning based on Faster-RCNN network;
[0021] Step 3-1: Use the multi-target region recommendation network as the region screening module in the Faster-RCNN network to obtain the candidate fetal heart region of the ultrasound image;
[0022] Step 3-2: The fetal heart candidate regions of different sizes are unified through the pooling layer and processed into a unified size format as the input of the Faster-RCNN network;
[0023] Step 3-3: Using the full convolution-pooling layer as a unit, a recurrent multi-layer network framework is constructed to extract features from the input ultrasound image, and then the extracted features are input into the fully connected layer for feature fusion;
[0024] Step 3-4: After feature fusion, the Softmax output layer is connected to determine whether the fetal heart candidate area contains the fetal heart; the fetal heart candidate area containing the fetal heart is regressed by bounding box, and multiple predicted fetal heart areas are output in the output layer;
[0025] Step 3-5: For multiple predicted fetal heart regions, the non-maximum suppression evaluation standard is used to select the fetal heart region with the highest discrimination score as the final result, and finally the fetal heart positioning in the ultrasound image is realized;
[0026] Step 4: constructing a human-in-the-loop system, which includes a doctor feedback unit and a machine navigation unit; the doctor feedback unit is used to record in real time the doctor's evaluation results on the degree of compliance of the standard section of the ultrasound image, and the machine navigation unit is used to give navigation instruction information according to the positioning information of the fetal heart;
[0027] Step 4-1: Construct a doctor feedback unit for human-computer interaction;
[0028] Step 4-1-1: Obtain a frame-by-frame sequence of ultrasound images through the linear array probe of the ultrasound image acquisition device as input to the doctor feedback unit and transmit it to the computer in real time as the current state s n ;
[0029] Step 4-1-2: a foot pressure sensor is set in the doctor feedback unit as an evaluation signal acquisition device in the doctor feedback unit, and the acquired signal is used as the output of the doctor feedback unit;
[0030] Step 4-1-3: The doctor observes the display screen of the ultrasound image acquisition device and feeds back the doctor's evaluation of the current ultrasound image to the computer through the foot pressure sensor. Pressing the foot pressure sensor indicates satisfaction, and not pressing the foot pressure sensor indicates dissatisfaction;
[0031] Step 4-1-4: The image collected by the ultrasound image acquisition device and the pedal signal collected by the evaluation signal acquisition device are synchronized through the clock signal, and the pedal signal after the clock signal synchronization is used as the doctor's feedback signal to the machine, serving as the reward function R of the Markov process n ;
[0032] Step 4-2: Construct a machine navigation unit based on the standard section discrimination model;
[0033] Step 4-2-1: Label the image data according to whether the ultrasound image meets the standard section, use the labeled image data as training data, and train the convolutional neural network to obtain a standard section discrimination model;
[0034] Step 4-2-2: For each frame of ultrasound image, the fetal heart positioning result obtained in step 3 is input into the standard section discrimination model trained in step 4-2-1 in real time, and the fetal heart positioning result and the discrimination result of the standard section discrimination model are used as navigation indication information and displayed on the display screen;
[0035] Step 5: Reinforcement learning method for human-in-the-loop intelligent assisted navigation;
[0036] Step 5-1: Use a deep Q network in the reinforcement learning framework of the machine navigation unit, where the output value Q(s n ,a n ) represents the corresponding n-th step state s n Next action a n The action value function is used; the standard section discrimination model based on convolutional neural network constructed in step 4-2-1 is used as the intelligent agent, and the doctor is used as the environment;
[0037] In the reinforcement learning process, let the current step be the nth step, and take the nth step current image scanned by the ultrasound probe as the state s n In this state, the system gives navigation instructions and executes action a n The doctor uses the navigation instructions displayed on the monitor and the real-time evaluation given by the pedal pressure sensor as the reward signal R of the environment. n ;
[0038] The update formula of the reinforcement learning deep Q network is as follows:
[0039] Q new (s n ,a n )←(1-α)·Q(s n ,a n )+α·{R n +γ·max a∈A Q(s n+1 ,a)-Q(s n ,a n )} (2) Where A is the action space of all possible actions of the agent, a is the action in the action space, and Q(s n ,a n ) is the system state s for a given n-th step n and action a n The output value of the deep Q network at this time, Q(s n+1 ,a) is the new state s of the system for the given n+1th step n+1 and the output value of the deep Q network at action a, Q new (s n ,a n ) is based on the state s n and action a n The updated Q output value, γ is the depreciation factor, which is a real number between 0 and 1, and the value is determined by the user. The larger the value, the greater the bias of the model towards long-term rewards. By updating formula (2), we can obtain the action value function Q(s n ,a n ) Strategy parameters that maximize overall return;
[0040] Step 5-2: The navigation unit of the machine provides the doctor with real-time feedback of the navigation information of the fetal heart position, thereby guiding the doctor in the scanning direction and technique;
[0041] Step 5-3: When the doctor successfully searches for the standard section image of fetal heart ultrasound according to the indication information of the machine navigation unit during scanning, the doctor uses the pedal pressure sensor to make a real-time evaluation of the image, and then feeds back the current standard section image and evaluation signal to the computer through the feedback unit;
[0042] Step 5-4: In order to improve the system's accuracy in distinguishing standard cross-section images of fetal heart ultrasound, the standard cross-section discrimination model of the machine navigation unit in step 4-2-1 is reinforced by learning through the real-time feedback signal of the doctor's evaluation: the feedback signal is the evaluation signal of the pedal pressure sensor, and the reinforcement learning strategy parameter set and the convolutional neural network parameter set of the standard cross-section discrimination model are repeatedly iterated, thereby continuously improving the navigation capability of the human-in-the-loop system and realizing the extraction of standard cross-sections of fetal heart ultrasound based on the human-in-the-loop mode.
[0043] Preferably, in step 1-1, data format conversion is performed using MicroDicom software.
[0044] Preferably, when marking the fetal heart and the positions of organs around the fetal heart in the TIF format image in step 1-2, labelImg software is used to create multiple rectangular frames to frame the positions of the fetal heart and the positions of organs around the fetal heart.
[0045] The beneficial effects of the present invention are as follows:
[0046] 1) In view of the problem that it is difficult to determine whether the current scan frame image contains the fetal heart during the navigation process, the present invention constructs a multi-target region recommendation network for the fetal heart and adjacent organs through the positional relationship between the fetal heart and adjacent organs in the anatomical structure, and uses it as a region screening module introduced into the Faster-RCNN framework to correct the fetal heart attention weight generated by the region recommendation network, thereby making the fetal heart positioning result more focused;
[0047] 2) In order to solve the problems of high intensity of fetal ultrasound standard section labeling and offline extraction, the present invention uses the navigation system and the doctor as the intelligent agent and the environment respectively, and uses the collected doctor's evaluation feedback, system navigation prompts and current scan images as reward signals, actions and states respectively, to perform reinforcement learning on the system's standard section discrimination model, continuously improve the system's ultrasound standard section discrimination capability, and thus achieve a gradual enhancement of navigation performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 The present invention is implemented in a specific flow chart.
[0049] Figure 2 It is a schematic diagram of the principle of the present invention.
[0050] Figure 3 This is the Faster-RCNN network framework diagram in the present invention.
[0051] Figure 4 Schematic diagram of the human-in-the-loop system of the present invention.
[0052] Figure 5 Schematic diagram of the deep Q network reinforcement learning framework of the present invention. DETAILED DESCRIPTION
[0053] The present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0054] The present invention discloses a method for extracting standard sections of fetal heart ultrasound based on human-in-the-loop intelligent assisted navigation, which mainly solves the problems of high labeling intensity and offline extraction of existing assisted navigation extraction methods of standard sections of fetal heart ultrasound, and realizes assisted navigation and online extraction of standard sections of fetal heart ultrasound during the scanning process.
[0055] like Figure 1 As shown, a method for extracting standard sections of fetal heart ultrasound based on human-in-the-loop intelligent assisted navigation includes the following steps:
[0056] Step 1: Construct an ultrasound image dataset for marking the fetal heart position;
[0057] Since the fetal heart has adjacent organs such as the fetal lungs in the anatomical structure, the present invention intends to utilize the positional relationship between the fetal heart and the adjacent organs of the anatomical structure to achieve fetal heart positioning and detection based on the ultrasound image guided by the anatomical structure;
[0058] Step 1-1: Collect ultrasound image data in DICOM format, convert the data format through MicroDicom software, and convert the ultrasound image data into TIF format images that are easier to process digital images;
[0059] Step 1-2: A doctor or other medical professional marks the fetal heart and the positions of the organs around the fetal heart in the TIF format image; marking method: using labelImg software to create multiple rectangular frames to frame the positions of the fetal heart and the organs around the fetal heart;
[0060] Step 1-3: Convert the TIF format images annotated in step 1-2 into the PASCAL VOC annotation format to form an ultrasound image dataset for subsequent target detection tasks;
[0061] Step 2: Use a multi-target region recommendation network to determine the candidate fetal heart region;
[0062] Aiming at the positional relationship between the fetal heart and adjacent organs in ultrasound images, the present invention constructs an anatomically guided ultrasound image region recommendation network in the RPN link under the Faster-RCNN framework, thereby achieving effective utilization of anatomical structure information and preliminary screening in the fetal heart positioning algorithm framework.
[0063] Step 2-1: Use the RPN link under the Faster-RCNN framework as a multi-target region recommendation network;
[0064] Objective function L MRPNAs shown in formula (1):
[0065]
[0066] Where, L class L is the average of the attention loss of the fetal heart and organs around the fetal heart. loc is the sum of the average spatial distance loss between the fetal heart and the organs around the fetal heart; λ is the adjustment factor used to adjust L class and L loc The order of magnitude is used to adjust the loss weight ratio; i represents the index of the image, N class Indicates the number of images for which fetal heart rate is known, p i Indicates whether the i-th image is a known 0 / 1 label of the fetal heart. represents the predicted probability of whether the i-th picture is a fetal heart, N loc Indicates the number of coordinate values of the known fetal heart in the image space position calibration frame, t i Represents the known coordinate value of the fetal heart position calibration frame of the i-th image, Represents the predicted coordinate value of the fetal heart position calibration frame of the i-th image by PRN;
[0067] Step 2-2: Use the ultrasound image dataset constructed in step 1 to train the multi-target region recommendation network, the output of which is the candidate regions of the fetal heart and organs around the fetal heart and the corresponding attention weights;
[0068] Step 2-3: For the overlapped part of the output surrounding organ region and the fetal heart candidate region, compare the output surrounding organ candidate region attention weight and the fetal heart candidate region attention weight, exclude the region whose fetal heart candidate region attention weight is less than the surrounding organ candidate region attention weight, and use the non-excluded region as the final fetal heart candidate region after anatomical structure correction; by using the adjacent organs to assist in the correction of the fetal heart attention weight, a more accurate and reliable fetal heart candidate region can be obtained, so that the subsequent positioning of the fetal heart region can save a lot of meaningless region comparison;
[0069] Step 3: If Figure 3 As shown, fetal heart region positioning based on Faster-RCNN network;
[0070] Step 3-1: Use the multi-target region recommendation network as the region screening module in the Faster-RCNN network to obtain the candidate fetal heart region in the ultrasound image;
[0071] Step 3-2: In order to facilitate the positioning of the fetal heart from the candidate areas of the ultrasound image, the fetal heart candidate areas of different sizes are unified through the pooling layer and processed into a unified size format as the input of the Faster-RCNN network;
[0072] Step 3-3: Using the full convolution-pooling layer as a unit, a recurrent multi-layer network framework is constructed to extract features from the input ultrasound image, and then the extracted features are input into the fully connected layer for feature fusion;
[0073] Step 3-4: After feature fusion, the Softmax output layer is connected to determine whether the fetal heart candidate area contains the fetal heart; the fetal heart candidate area containing the fetal heart is regressed by bounding box, and multiple predicted fetal heart areas are output in the output layer;
[0074] Step 3-5: For multiple predicted fetal heart regions, the non-maximum suppression evaluation standard is used to select the fetal heart region with the highest discrimination score as the final result, and finally the fetal heart positioning in the ultrasound image is realized;
[0075] Step 4: Figure 4 As shown, a human-in-the-loop system is constructed, and the human-in-the-loop system includes a doctor feedback unit and a machine navigation unit; the doctor feedback unit is used to record the doctor's evaluation result on the compliance degree of the standard section of the ultrasound image in real time, and the machine navigation unit is used to give navigation instruction information according to the positioning information of the fetal heart;
[0076] Step 4-1: Construct a doctor feedback unit for human-computer interaction;
[0077] like Figure 2 As shown, in prenatal ultrasound examinations, since the existing devices are unable to receive the doctor's evaluation of the image in real time, the present invention constructs a reinforcement learning model of human-computer interaction: a doctor feedback unit is introduced into the ultrasound detection device as an environmental element of the Markov process, and the system adjusts the decision in real time according to the data of the doctor feedback unit, iterates the parameter set of the model state transfer, and then continuously optimizes the performance of the entire system, thereby realizing the human-computer interaction function of the human-in-the-loop navigation extraction system.
[0078] Step 4-1-1: Obtain a frame-by-frame sequence of ultrasound images through the linear array probe of the ultrasound image acquisition device as input to the doctor feedback unit and transmit it to the computer in real time as the current state s n (Assume the current step is n);
[0079] Step 4-1-2: a foot pressure sensor is set in the doctor feedback unit as an evaluation signal acquisition device in the doctor feedback unit, and the acquired signal is used as the output of the doctor feedback unit;
[0080] Step 4-1-3: The doctor observes the display screen of the ultrasound image acquisition device and feeds back the doctor's evaluation of the current ultrasound image to the computer through the foot pressure sensor. Pressing the foot pressure sensor indicates satisfaction, and not pressing the foot pressure sensor indicates dissatisfaction;
[0081] Step 4-1-4: The image collected by the ultrasound image acquisition device and the pedal signal collected by the evaluation signal acquisition device are synchronized through the clock signal, and the pedal signal after the clock signal synchronization is used as the doctor's feedback signal to the machine, serving as the reward function R of the Markov process n ;
[0082] Step 4-2: Construct a machine navigation unit based on the standard section discrimination model;
[0083] Since prenatal fetal heart diagnosis requires that the fetal heart ultrasound image reaches a standard section, the present invention constructs a machine navigation unit based on a standard section discrimination model, i.e., the intelligent agent in the subsequent step 5-1 reinforcement learning model.
[0084] Step 4-2-1: Label the image data according to whether the ultrasound image meets the standard section, use the labeled image data as training data, and train the convolutional neural network to obtain a standard section discrimination model;
[0085] Step 4-2-2: For each frame of ultrasound image, the fetal heart positioning result obtained in step 3 is input into the standard section discrimination model trained in step 4-2-1 in real time, and the fetal heart positioning result and the discrimination result of the standard section discrimination model are used as navigation indication information and displayed on the display screen;
[0086] Step 5: Reinforcement learning method for human-in-the-loop intelligent assisted navigation;
[0087] In the human-in-the-loop system, not only can the navigation system provide auxiliary guidance and navigation functions for doctors' scans, but it can also optimize the model parameters through human-computer interaction with doctors, thereby improving the performance of the navigation system. In order to further improve the accuracy of fetal ultrasound standard section discrimination, the above-mentioned doctor feedback unit is combined with the machine navigation unit to perform reinforcement learning on the fetal ultrasound standard section navigation system.
[0088] Step 5-1: If Figure 5 As shown, a deep Q network is used in the reinforcement learning framework of the machine navigation unit, where the output value Q(s n ,a n ) represents the corresponding n-th step state s n Next action a n action value function; in the reinforcement learning here, the standard section discrimination model based on convolutional neural network constructed in step 4-2-1 is used as the agent, and the doctor is used as the environment;
[0089] In the reinforcement learning process, let the current step be the nth step, and take the nth step current image scanned by the ultrasound probe as the state sn In this state, the system gives navigation instructions (execute action a n ) and displayed on the display; the doctor uses the real-time evaluation given by the foot pressure sensor based on the navigation instructions displayed on the display as the reward signal R of the environment n ;
[0090] The update formula of the reinforcement learning deep Q network is as follows:
[0091] Q new (s n ,a n )←(1-α)·Q(s n ,a n )+α·{R n +γ·max a∈A Q(s n+1 ,a)-Q(s n ,a n )} (2) Where A is the action space of all possible actions of the agent, a is the action in the action space, and Q(s n ,a n ) is the system state s for a given n-th step n and action a n The output value of the deep Q network at this time, Q(s n+1 ,a) is the new state s of the system for the given n+1th step n+1 and the output value of the deep Q network at action a, Q new (s n ,a n ) is based on the state s n and action a n The updated Q output value, γ is the depreciation factor, which is a real number between 0 and 1, and the value is determined by the user. The larger the value, the greater the bias of the model towards long-term rewards. Through the above update formula (2), we can obtain the action value function Q(s n ,a n ) Strategy parameters that maximize overall return;
[0092] Step 5-2: To assist the doctor in effectively extracting the standard section of fetal heart ultrasound during the scanning process, the navigation unit of the machine provides the doctor with real-time feedback of the navigation information of the fetal heart position through the display of the machine, so as to guide the doctor in the scanning direction and technique;
[0093] Step 5-3: When the doctor successfully searches for the standard section image of fetal heart ultrasound according to the indication information of the machine navigation unit during scanning, the doctor uses the pedal pressure sensor to make a real-time evaluation of the image, and then feeds back the current standard section image and evaluation signal to the computer through the feedback unit;
[0094] Step 5-4: In order to improve the system's accuracy in distinguishing standard cross-section images of fetal heart ultrasound, the standard cross-section discrimination model of the machine navigation unit in step 4-2-1 is reinforced by learning through the real-time feedback signal of the doctor's evaluation: the feedback signal is the evaluation signal of the pedal pressure sensor, and the reinforcement learning strategy parameter set and the convolutional neural network parameter set of the standard cross-section discrimination model are repeatedly iterated, thereby continuously improving the navigation capability of the human-in-the-loop system and realizing the extraction of standard cross-sections of fetal heart ultrasound based on the human-in-the-loop mode.
Claims
1. A method for extracting standard sections of fetal heart ultrasound based on human-in-the-loop intelligent assisted navigation, It is characterized in that The following steps are involved: Step 1: Construct an ultrasound image dataset for marking the fetal heart position; Step 1-1: Collect ultrasound image data in DICOM format, convert the data format, and convert the ultrasound image data into TIF format images; Step 1-2: Mark the fetal heart and the positions of organs around the fetal heart in the TIF format image; Step 1-3: Convert the TIF format images annotated in step 1-2 into the PASCAL VOC annotation format to form an ultrasound image dataset; Step 2: Use a multi-target region recommendation network to determine the candidate fetal heart region; Step 2-1: Use the RPN link under the Faster-RCNN framework as a multi-target region recommendation network; Objective function L MRPN As shown in formula (1): Where, L class L is the average of the attention loss of the fetal heart and organs around the fetal heart. loc is the sum of the average spatial distance loss between the fetal heart and the organs around the fetal heart; λ is the adjustment factor used to adjust L class and L loc The order of magnitude is used to adjust the loss weight ratio; i represents the index of the image, N class Indicates the number of images for which fetal heart rate is known, p i Indicates whether the i-th image is a known 0 / 1 label of the fetal heart. Indicates the prediction probability of PRN for whether the i-th image is a fetal heart, N loc Indicates the number of coordinate values of the known fetal heart in the image space position calibration frame, t i Represents the known coordinate value of the fetal heart position calibration frame of the i-th image, Represents the predicted coordinate value of the fetal heart position calibration frame of the i-th image by PRN; Step 2-2: The ultrasound image dataset constructed in step 1 is used to train the multi-target region recommendation network, and the output of the multi-target region recommendation network is the candidate regions of the fetal heart and the candidate regions of the organs around the fetal heart and the corresponding attention weights; Step 2-3: For the overlapping parts of the candidate regions of the organs surrounding the fetal heart and the candidate regions of the fetal heart output by the multi-target region recommendation network, the output attention weights of the candidate regions of the organs surrounding the fetal heart and the output attention weights of the candidate regions of the fetal heart are compared, and the regions whose attention weights of the candidate regions of the fetal heart are smaller than those of the candidate regions of the organs surrounding the fetal heart are excluded, and the regions that are not excluded are used as the final candidate regions of the fetal heart after the anatomical structure is corrected; Step 3: Fetal heart region positioning based on Faster-RCNN network; Step 3-1: Use the multi-target region recommendation network as the region screening module in the Faster-RCNN network to obtain the candidate fetal heart region of the ultrasound image; Step 3-2: The fetal heart candidate regions of different sizes are unified through the pooling layer and processed into a unified size format as the input of the Faster-RCNN network; Step 3-3: Using the full convolution-pooling layer as a unit, a recurrent multi-layer network framework is constructed to extract features from the input ultrasound image, and then the extracted features are input into the fully connected layer for feature fusion; Step 3-4: After feature fusion, the Softmax output layer is connected to determine whether the fetal heart candidate area contains the fetal heart; the fetal heart candidate area containing the fetal heart is regressed by bounding box, and multiple predicted fetal heart areas are output in the output layer; Step 3-5: For multiple predicted fetal heart regions, the non-maximum suppression evaluation standard is used to select the fetal heart region with the highest discrimination score as the final result, and finally the fetal heart positioning in the ultrasound image is realized; Step 4: constructing a human-in-the-loop system, which includes a doctor feedback unit and a machine navigation unit; the doctor feedback unit is used to record in real time the doctor's evaluation results on the degree of compliance of the standard section of the ultrasound image, and the machine navigation unit is used to give navigation instruction information according to the positioning information of the fetal heart; Step 4-1: Construct a doctor feedback unit for human-computer interaction; Step 4-1-1: Obtain a frame-by-frame sequence of ultrasound images through the linear array probe of the ultrasound image acquisition device as input to the doctor feedback unit and transmit it to the computer in real time as the current state s n ; Step 4-1-2: a foot pressure sensor is set in the doctor feedback unit as an evaluation signal acquisition device in the doctor feedback unit, and the acquired signal is used as the output of the doctor feedback unit; Step 4-1-3: The doctor observes the display screen of the ultrasound image acquisition device and feeds back the doctor's evaluation of the current ultrasound image to the computer through the foot pressure sensor. Pressing the foot pressure sensor indicates satisfaction, and not pressing the foot pressure sensor indicates dissatisfaction; Step 4-1-4: The image collected by the ultrasound image acquisition device and the pedal signal collected by the evaluation signal acquisition device are synchronized through the clock signal, and the pedal signal after the clock signal synchronization is used as the doctor's feedback signal to the machine, serving as the reward function R of the Markov process n ; Step 4-2: Construct a machine navigation unit based on the standard section discrimination model; Step 4-2-1: Label the image data according to whether the ultrasound image meets the standard section, use the labeled image data as training data, and train the convolutional neural network to obtain a standard section discrimination model; Step 4-2-2: For each frame of ultrasound image, the fetal heart positioning result obtained in step 3 is input into the standard section discrimination model trained in step 4-2-1 in real time, and the fetal heart positioning result and the discrimination result of the standard section discrimination model are used as navigation indication information and displayed on the display screen; Step 5: Reinforcement learning method for human-in-the-loop intelligent assisted navigation; Step 5-1: Use a deep Q network in the reinforcement learning framework of the machine navigation unit, where the output value Q(s n , a n ) represents the corresponding n-th step state s n Next action a n The action value function is used; the standard section discrimination model based on convolutional neural network constructed in step 4-2-1 is used as the intelligent agent, and the doctor is used as the environment; In the reinforcement learning process, let the current step be the nth step, and take the nth step current image scanned by the ultrasound probe as the state s n In this state, the system gives navigation instructions and executes action a n The doctor uses the navigation instructions displayed on the monitor and the real-time evaluation given by the pedal pressure sensor as the reward signal R of the environment. n ; The update formula of the reinforcement learning deep Q network is as follows: Q new (s n ,a n )←(1-α)·Q(s n ,a n )+α·{R n +γ·max a∈A Q(s n+1 ,a)-Q(s n ,a n )} (2) Where A is the action space of all possible actions of the agent, a is the action in the action space, and Q(s n , a n ) is the system state s for a given n-th step n and action a n The output value of the deep Q network at this time, Q(s n+1 , a) is the system's new state s for the given n+1th step n+1 and the output value of the deep Q network at action a, Q new (s n , a n ) is based on the state s n and action a n The updated Q output value, γ is the depreciation factor, which is a real number between 0 and 1, and the value is determined by the user. The larger the value, the greater the bias of the model towards long-term rewards. By updating formula (2), we can obtain the action value function Q(s n , a n ) Strategy parameters that maximize overall return; Step 5-2: The navigation unit of the machine provides the doctor with real-time feedback of the navigation information of the fetal heart position, thereby guiding the doctor in the scanning direction and technique; Step 5-3: When the doctor successfully searches for the standard section image of fetal heart ultrasound according to the indication information of the machine navigation unit during scanning, the doctor uses the pedal pressure sensor to make a real-time evaluation of the image, and then feeds back the current standard section image and evaluation signal to the computer through the feedback unit; Step 5-4: In order to improve the system's accuracy in distinguishing standard cross-section images of fetal heart ultrasound, the standard cross-section discrimination model of the machine navigation unit in step 4-2-1 is reinforced by learning through the real-time feedback signal of the doctor's evaluation: the feedback signal is the evaluation signal of the pedal pressure sensor, and the reinforcement learning strategy parameter set and the convolutional neural network parameter set of the standard cross-section discrimination model are repeatedly iterated, thereby continuously improving the navigation capability of the human-in-the-loop system and realizing the extraction of standard cross-sections of fetal heart ultrasound based on the human-in-the-loop mode.
2. According to claim 1, a method for extracting standard sections of fetal heart ultrasound based on human-in-the-loop intelligent assisted navigation, It is characterized in that In step 1-1, data format conversion is performed using MicroDicom software.
3. According to claim 1, a method for extracting standard sections of fetal heart ultrasound based on human-in-the-loop intelligent assisted navigation, It is characterized in that When marking the fetal heart and the positions of organs around the fetal heart in the TIF format image in step 1-2, labelImg software is used to create multiple rectangular frames to frame the positions of the fetal heart and the positions of organs around the fetal heart.
