A fetal heart rate detection method, an ultrasonic imaging device and a storage medium

Abstract

The embodiment of the application discloses a kind of fetal heart rate detection method and ultrasonic imaging device, storage medium, can improve the intelligence of fetal heart rate measurement to M super and the efficiency of checking the development of fetus in mother, the method can include: obtaining the multiple frames of ultrasound images of target fetus;Determine target sampling line based on the pixel value of multiple frames of ultrasound images;Using target sampling line, generate target M-mode ultrasonic image corresponding to multiple frames of ultrasound images;Determine the heart rate of target fetus using target M-mode ultrasonic image.

Description

[0001] This application is a divisional application of application number 201911356447.1, entitled "A Fetal Heart Rate Detection Method, Ultrasonic Imaging Device, and Storage Medium". Technical Field

[0002] This application relates to the field of ultrasound imaging, and more particularly to a method for detecting fetal heart rate, an ultrasound imaging device, and a storage medium. Background Technology

[0003] Ultrasound instruments are a primary auxiliary tool in obstetrics and gynecology for prenatal examinations and disease diagnosis. Fetal heart rate monitoring is a mandatory component of all prenatal ultrasound examinations (levels I-IV). Echocardiography is characterized by being radiation-free and non-invasive, and M-mode ultrasound can effectively measure fetal heart rate. Currently, fetal heart rate measurement during M-mode ultrasound is primarily manual, which has several drawbacks. First, sampling lines are manually selected when acquiring the M-mode image. Second, the fetal heart rate needs to be manually measured on the M-mode image. Third, the fetal heart rate may deviate significantly or even disappear during acquisition, making it impossible to obtain a meaningful M-mode image. This results in low intelligence in fetal heart rate measurement using M-mode ultrasound and low efficiency in monitoring fetal development in the womb. Summary of the Invention

[0004] To address the aforementioned technical problems, this application aims to provide a fetal heart rate detection method, an ultrasound imaging device, and a storage medium, which can improve the intelligence of fetal heart rate measurement using M-mode ultrasound and the efficiency of examining fetal development in the womb.

[0005] The technical solution of this application embodiment can be implemented as follows:

[0006] This application provides a method for detecting fetal heart rate, the method comprising:

[0007] Acquire multiple frames of ultrasound images of the target fetus;

[0008] Based on the pixel values ​​of the multiple ultrasound images, the target sampling line is determined;

[0009] Using the target sampling line, a target M-mode ultrasound image corresponding to the multiple ultrasound images is generated;

[0010] The heart rate of the target fetus is determined using the target M-mode ultrasound image.

[0011] In the above method, the method further includes:

[0012] Displays the heart rate of the target fetus and the target M-mode ultrasound image.

[0013] In the above method, determining the target sampling line based on the pixel values ​​of the multiple ultrasound images includes:

[0014] The target sampling line is determined based on the pixel values ​​of all regions of the multi-frame ultrasound images.

[0015] In the above method, before determining the target sampling line based on the pixel values ​​of the multiple frames of ultrasound images, the method further includes:

[0016] Using a preset positioning method, ultrasound images containing a region of interest are determined from the multiple ultrasound images, wherein the region of interest includes the entire fetal heart region or a portion of the fetal heart region of the target fetus;

[0017] Accordingly, determining the target sampling line based on the pixel values ​​of the multiple ultrasound images includes:

[0018] Determine the pixel value of the region of interest from the ultrasound image containing the region of interest;

[0019] The target sampling line is determined based on the pixel values ​​of the region of interest.

[0020] In the above method, determining the ultrasound image containing the region of interest from the multiple ultrasound images using a preset positioning method includes:

[0021] The neural network is trained using preset ultrasound images, wherein the preset ultrasound images include regions of interest; the trained neural network is then used to perform feature matching on the multiple frames of ultrasound images to determine the ultrasound images containing the regions of interest from the multiple frames of ultrasound images.

[0022] Alternatively, feature extraction can be performed on preset ultrasound images; the extracted features can be learned, and the multi-frame ultrasound images can be classified according to the learning results to determine the ultrasound images containing regions of interest from the multi-frame ultrasound images.

[0023] In the above method, determining the target sampling line based on the pixel values ​​of the multiple ultrasound images includes:

[0024] Using a preset image cropping algorithm, the multiple frames of ultrasound images are divided into multiple image blocks;

[0025] The pixel values ​​of the plurality of image blocks are determined respectively;

[0026] The target sampling line is determined based on the variation range of pixel values ​​in the plurality of image blocks.

[0027] In the above method, determining the target sampling line based on the pixel values ​​of the multiple ultrasound images includes:

[0028] Based on the pixel values ​​of the multiple ultrasound images, multiple sampling lines are determined in the multiple ultrasound images.

[0029] The periodicity of the heartbeat frames in the multiple M-mode ultrasound images corresponding to the multiple sampling lines is determined respectively;

[0030] From the multiple M-mode ultrasound images, determine the target M-mode ultrasound image whose heartbeat frame periodicity meets the preset conditions;

[0031] The sampling line corresponding to the target M-mode ultrasound image is determined as the target sampling line.

[0032] In the above method, determining the target sampling line based on the pixel values ​​of the multiple ultrasound images includes:

[0033] The multi-frame ultrasound images are input into a preset neural network;

[0034] The preset neural network is used to analyze the variation range of pixel values ​​in the multi-frame ultrasound images;

[0035] The target sampling line is determined based on the sampling line position information output by the preset neural network.

[0036] In the above method, determining the target sampling line based on the pixel values ​​of the multiple ultrasound images includes:

[0037] Based on the pixel values ​​of the multiple ultrasound images, multiple sampling lines are determined in the multiple ultrasound images. The multiple sampling lines are composed of multiple sets of pixel values, and the multiple sampling lines and the multiple sets of pixel values ​​correspond one-to-one.

[0038] Obtain the pixel value variation amplitude of the multiple sets of pixel values ​​in the multiple frames of ultrasound images;

[0039] From the multiple sampling lines, determine the sampling line with the largest pixel value change.

[0040] The sampling line with the largest change in pixel value is determined as the target sampling line.

[0041] In the above method, determining the heart rate of the target fetus using the target M-mode ultrasound image includes:

[0042] Obtain the oscillation curve of the target M-mode ultrasound image;

[0043] The heart rate of the target fetus is determined based on the oscillation curve.

[0044] In the above method, obtaining the oscillation curve of the M-mode ultrasound image includes:

[0045] The target M-mode ultrasound image is input into a preset neural network, and the oscillation curve is obtained by the output of the preset neural network.

[0046] Alternatively, the oscillation curve can be obtained based on the location information of the maximum image gradient in the target M-mode ultrasound image.

[0047] In the above method, determining the heart rate of the target fetus based on the oscillation curve includes:

[0048] Find the peaks and troughs from the oscillation curve;

[0049] The heartbeat frame period is determined using the peaks and troughs.

[0050] The heart rate of the target fetus is determined based on the heartbeat frame period.

[0051] In the above method, determining the heart rate of the target fetus using the target M-mode ultrasound image includes:

[0052] A first image block is determined from the target M-mode ultrasound image, wherein the first image block is any image block in the target M-mode ultrasound image;

[0053] Find the second target image block with the highest similarity to the first target image block near the location of the first target image block, wherein the first target image block is any image block starting from the first image block, and the second target image block is the next image block after the first target image block;

[0054] Determine the cumulative motion displacement between the first target image block and the second target image block;

[0055] The heartbeat frame period is determined based on the cumulative motion displacement.

[0056] The heart rate of the target fetus is determined based on the heartbeat frame period.

[0057] In the above method, determining the heart rate of the target fetus using the target M-mode ultrasound image includes:

[0058] A first image block is determined from the target M-mode ultrasound image, wherein the first image block is any image block in the M-mode ultrasound image;

[0059] Near at least one preset heartbeat frame cycle position of the target M-mode ultrasound image, find a third image block that has the highest similarity to the first image block and is closest to it in the same horizontal direction as the first image block;

[0060] Determine the motion displacement between the first image block and the third image block;

[0061] The heartbeat frame period is determined based on the motion displacement;

[0062] The heart rate of the target fetus is determined based on the heartbeat frame period.

[0063] In the above method, the method further includes:

[0064] The start and end positions of one or more of the heartbeat frame cycles are marked in the target M-mode ultrasound image.

[0065] This application provides an ultrasound imaging device, the ultrasound imaging device comprising:

[0066] probe;

[0067] A transmitting circuit that excites the probe to emit ultrasound waves toward the target fetus;

[0068] A receiving circuit that receives ultrasound echoes returned from the target fetus via the probe to obtain an ultrasound echo signal;

[0069] A processor that processes the ultrasound echo signal to obtain an ultrasound image of the target fetus;

[0070] A display showing the ultrasound image;

[0071] The processor further performs the following steps:

[0072] Acquire multiple ultrasound images of the target fetus; determine a target sampling line based on the pixel values ​​of the multiple ultrasound images; generate a target M-mode ultrasound image corresponding to the multiple ultrasound images using the target sampling line; determine the heart rate of the target fetus using the target M-mode ultrasound image.

[0073] In the aforementioned ultrasound imaging device, the display is also used to display the heart rate of the target fetus and the target M-mode ultrasound image.

[0074] In the aforementioned ultrasound imaging device, the processor is further configured to determine the target sampling line based on the pixel values ​​of all regions of the multi-frame ultrasound images.

[0075] In the aforementioned ultrasound imaging device, the processor is further configured to use a preset positioning method to determine an ultrasound image containing a region of interest from the multiple ultrasound images, wherein the region of interest includes the entire fetal heart region or a portion of the fetal heart region of the target fetus; determine the pixel value of the region of interest from the ultrasound image containing the region of interest; and determine the target sampling line based on the pixel value of the region of interest.

[0076] In the aforementioned ultrasound imaging device, the processor is further used for

[0077] The neural network is trained using preset ultrasound images, wherein the preset ultrasound images include regions of interest; the trained neural network is then used to perform feature matching on the multiple frames of ultrasound images to determine the ultrasound images containing the regions of interest from the multiple frames of ultrasound images.

[0078] Alternatively, feature extraction can be performed on preset ultrasound images; the extracted features can be learned, and the multi-frame ultrasound images can be classified according to the learning results to determine the ultrasound images containing regions of interest from the multi-frame ultrasound images.

[0079] In the aforementioned ultrasound imaging device, the processor is further configured to use a preset image cropping algorithm to divide the multiple frames of ultrasound images into multiple image blocks; determine the pixel values ​​of the multiple image blocks respectively; and determine the target sampling line based on the variation range of the pixel values ​​of the multiple image blocks.

[0080] In the aforementioned ultrasound imaging device, the processor is further configured to: determine multiple sampling lines in the multiple ultrasound images based on the pixel values ​​of the multiple ultrasound images; determine the periodicity of the heartbeat frames of the multiple M-mode ultrasound images corresponding to the multiple sampling lines; determine a target M-mode ultrasound image whose heartbeat frame periodicity meets a preset condition from the multiple M-mode ultrasound images; and determine the sampling line corresponding to the target M-mode ultrasound image as the target sampling line.

[0081] In the aforementioned ultrasound imaging device, the processor is further configured to input the multi-frame ultrasound images into a preset neural network; use the preset neural network to analyze the variation range of pixel values ​​in the multi-frame ultrasound images; and determine the target sampling line based on the sampling line position information output by the preset neural network.

[0082] In the aforementioned ultrasound imaging device, multiple sampling lines are determined based on the pixel values ​​of the multiple ultrasound images. Each sampling line consists of multiple sets of pixel values, and the multiple sampling lines correspond one-to-one with the multiple sets of pixel values. The variation range of the pixel values ​​of the multiple sets of pixel values ​​in the multiple ultrasound images is obtained. The sampling line with the largest variation range of pixel values ​​is determined from the multiple sampling lines. The sampling line with the largest variation range of pixel values ​​is determined as the target sampling line.

[0083] In the aforementioned ultrasound imaging device, the processor is further configured to acquire an oscillation curve of the target M-mode ultrasound image and determine the heart rate of the target fetus based on the oscillation curve.

[0084] In the aforementioned ultrasound imaging device, the processor is further configured to input the target M-mode ultrasound image into a preset neural network and output the oscillation curve through the preset neural network; or, to obtain the oscillation curve based on the location information of the maximum image gradient of the target M-mode ultrasound image.

[0085] In the aforementioned ultrasound imaging device, the processor is further configured to find peaks and troughs from the oscillation curve; calculate the heartbeat frame period using the peaks and troughs; and determine the heart rate of the target fetus based on the heartbeat frame period.

[0086] In the aforementioned ultrasound imaging device, the processor is further configured to: determine a first image block from the target M-mode ultrasound image, wherein the first image block is any image block in the target M-mode ultrasound image; search for a second image block near the location of the first target image block that has the highest similarity to the first target image block, wherein the first target image block is any image block starting from the first image block, and the second target image block is the next image block after the first target image block; determine the cumulative motion displacement between the first target image block and the second target image block; determine the heartbeat frame period based on the cumulative motion displacement; and determine the heart rate of the target fetus based on the heartbeat frame period.

[0087] In the aforementioned ultrasound imaging device, the processor is further configured to: determine a first image block from the target M-mode ultrasound image, wherein the first image block is any image block in the M-mode ultrasound image; search for a third image block that has the highest similarity to the first image block and is closest to it in the same horizontal direction as the first image block near at least one preset heartbeat frame cycle position of the target M-mode ultrasound image; determine the motion displacement between the first image block and the third image block; determine the heartbeat frame cycle based on the motion displacement; and determine the heart rate of the target fetus based on the heartbeat frame cycle.

[0088] In the aforementioned ultrasound imaging device, the display is also used to mark the start and end positions of one or more heartbeat frame cycles in the target M-mode ultrasound image.

[0089] This application provides a computer-readable storage medium storing a computer program applied to an ultrasound imaging device. When executed by a processor, the computer program implements the fetal heart rate detection method as described in any of the preceding claims.

[0090] This application provides a method for detecting fetal heart rate, the method comprising:

[0091] Acquire ultrasound images of the target fetus;

[0092] The target sampling line is determined based on the pixel values ​​of the ultrasound image;

[0093] Using the target sampling line, a target M-mode ultrasound image is generated;

[0094] The heart rate of the target fetus is determined using the target M-mode ultrasound image.

[0095] In the above method, determining the target sampling line based on the pixel values ​​of the ultrasound image includes:

[0096] Determine the fetal heart region from the ultrasound images;

[0097] Determine the target location from the fetal heart region;

[0098] Determine the target sampling line based on the target location.

[0099] In the above method, determining the target location from the fetal heart region includes:

[0100] The target location is determined from the fetal heart region by the geometric relationship between the target location and the fetal heart region.

[0101] In the above method, determining the target sampling line based on the pixel values ​​of the ultrasound image includes:

[0102] The ultrasound image is input into a preset neural network; the target location is determined based on the output of the preset neural network; or, features of the target location in the ultrasound image are extracted, and the target location is determined from the ultrasound image using a classifier.

[0103] Determine the target sampling line based on the target location.

[0104] In the above method, determining the target sampling line based on the target location includes:

[0105] The sampling line passing through the target location is determined as the target sampling line.

[0106] This application provides a method for detecting fetal heart rate, the method comprising:

[0107] Acquire ultrasound images of the target fetus;

[0108] Determine the target sampling line based on the ultrasound image;

[0109] Using the target sampling line, a target M-mode ultrasound image is generated;

[0110] The heart rate of the target fetus is determined using the target M-mode ultrasound image.

[0111] This application provides a method for detecting fetal M-mode ultrasound images, the method comprising:

[0112] Acquire ultrasound images of the target fetus;

[0113] Determine the target sampling line based on the ultrasound image;

[0114] Using the target sampling line, a target M-mode ultrasound image is generated.

[0115] This application provides a heart rate detection method, an ultrasound imaging device, and a storage medium. The method may include: acquiring multiple frames of ultrasound images of a target fetus; determining a target sampling line based on the pixel values ​​of the multiple frames of ultrasound images; generating a target M-mode ultrasound image corresponding to the multiple frames of ultrasound images using the target sampling line; and determining the heart rate of the target fetus using the target M-mode ultrasound image. By employing the above method, the ultrasound imaging device automatically acquires the target sampling line based on the pixel values ​​of the multiple frames of ultrasound images, can generate a meaningful target M-mode ultrasound image based on the target sampling line, and automatically measures the heart rate of the target fetus based on the target M-mode ultrasound image. This improves the intelligence of fetal heart rate measurement using M-mode ultrasound and the efficiency of examining fetal development in the womb. Attached Figure Description

[0116] Figure 1 This is a schematic diagram of the structure of an ultrasound imaging device provided in an embodiment of this application;

[0117] Figure 2 A flowchart of a fetal heart rate detection method provided in this application embodiment Figure 1 ;

[0118] Figure 3 A schematic diagram of an exemplary ultrasound imaging device provided in this application embodiment;

[0119] Figure 4 A flowchart illustrating an exemplary fetal heart rate detection method provided in this application embodiment;

[0120] Figure 5 A flowchart of a fetal heart rate detection method provided in this application embodiment Figure 2 ;

[0121] Figure 6 A flowchart of a fetal heart rate detection method provided in this application embodiment Figure 3 ;

[0122] Figure 7 A schematic diagram illustrating an exemplary automatic fetal heart rate region positioning effect provided in this application embodiment;

[0123] Figure 8 A flowchart of a fetal heart rate detection method provided in this application embodiment Figure 2 . Detailed Implementation

[0124] In order to gain a more detailed understanding of the features and technical content of the embodiments of this application, the implementation of the embodiments of this application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference and illustration only and are not intended to limit the embodiments of this application.

[0125] Figure 1 This is a schematic diagram of the ultrasound imaging device 10 in an embodiment of this application. The ultrasound imaging device 10 may include a probe 100, a transmitting circuit 101, a transmitting / receiving selection switch 102, a receiving circuit 103, a beamforming circuit 104, a processor 105, and a display 106. The transmitting circuit 101 excites the probe to emit ultrasound waves toward the target fetus. The receiving circuit 103 receives the ultrasound echoes returned from the target fetus through the probe 100 to obtain an ultrasound echo signal. This ultrasound echo signal is processed by the beamforming circuit 104 and then sent to the processor 105. The processor 105 processes the ultrasound echo signal to obtain ultrasound image data of the target fetus, which is M-mode ultrasound image data. The M-mode ultrasound image data obtained by the processor 105 can be stored in a memory 107, and this M-mode ultrasound image data can be displayed on the display 106.

[0126] In this embodiment, the display 106 of the aforementioned ultrasonic imaging device 10 can be a touch screen, a liquid crystal display, or an independent display device such as a liquid crystal display or a television set, separate from the ultrasonic imaging device 10. It can also be a display screen on an electronic device such as a mobile phone or a tablet computer.

[0127] In this embodiment, the memory 107 of the aforementioned ultrasound imaging device 10 can be a flash memory card, solid-state memory, hard disk, etc.

[0128] This application also provides a computer-readable storage medium storing a plurality of program instructions. After being called and executed by the processor 105, the plurality of program instructions can execute some or all of the steps or any combination of the steps in the fetal heart rate detection method in M-mode ultrasound images in various embodiments of this application.

[0129] In one embodiment, the computer-readable storage medium may be a memory 107, which may be a non-volatile storage medium such as a flash memory card, a solid-state memory, or a hard disk.

[0130] In this embodiment, the processor 105 of the aforementioned ultrasound imaging device 10 can be implemented by software, hardware, firmware, or a combination thereof. It can use circuits, one or more application-specific integrated circuits (ASICs), one or more general-purpose integrated circuits, one or more microprocessors, one or more programmable logic devices, or a combination of the aforementioned circuits or devices, or other suitable circuits or devices, so that the processor 105 can execute the corresponding steps of the fetal heart rate detection method in the aforementioned embodiments.

[0131] The method for detecting fetal heart rate in this application is described in detail below. Please refer to [link / reference]. Figure 2 The embodiments of the fetal heart rate detection method in this application include:

[0132] S101. Acquire multiple ultrasound images of the target fetus.

[0133] This application provides a fetal heart rate detection method applicable to scenarios where fetal heart rate is automatically measured based on M-mode ultrasound.

[0134] Optionally, the ultrasound imaging device can acquire multiple ultrasound images of the target fetus in two ways: local acquisition and real-time acquisition. The specific method can be selected according to the actual situation, and this application embodiment does not impose specific limitations.

[0135] Specifically, the local acquisition method is as follows: the heart of the target fetus is scanned by an ultrasound imaging device to obtain multiple ultrasound images, which are then saved to the hard drive in real time. They are then read from the local drive during subsequent processing steps.

[0136] Specifically, the real-time acquisition method is as follows: when scanning the heart of the target fetus with an ultrasound imaging device, multiple frames of ultrasound images are loaded into the running memory in real time for calculation.

[0137] In this embodiment of the application, the process of obtaining multiple ultrasound images by scanning the heart of the target fetus using an ultrasound imaging device is as follows: Figure 3 As shown, the ultrasound imaging device consists of a transmitting circuit, a probe, a receiving circuit, a beamsynthesizer, a signal processing unit, an image processing unit, and a display. The ultrasound imaging device sends a set of pulses that have been delayed and focused to the probe through the transmitting circuit to emit ultrasound waves. After a delay, the probe receives the ultrasound echo reflected back from the target fetus. The ultrasound echo signal enters the beamsynthesizer to complete the focusing delay, weighting, and channel summation. After passing through the signal processing unit, the pre-reconstruction data is obtained. Then, through the image processing unit, the data is converted into a two-dimensional image and displayed on the display.

[0138] S102. Determine the target sampling line based on the pixel values ​​of multiple ultrasound images.

[0139] After the ultrasound imaging device acquires multiple frames of ultrasound images of the target fetus, the ultrasound imaging device determines the target sampling line based on the pixel values ​​of the multiple frames of ultrasound images. The target sampling line can be the optimal sampling line or the near-optimal sampling line among all sampling lines. Generally speaking, the optimal sampling line refers to the sampling line with the largest variation in pixel values, that is, a better M-mode ultrasound image can be obtained through the optimal sampling line.

[0140] In this embodiment, the target sampling line can be determined based on the pixel values ​​of the entire region of the multi-frame ultrasound images, or it can be determined based on the pixel values ​​of the region of interest (ROI) of the multi-frame ultrasound images. The ROI includes the entire fetal heart rate region or a portion of the fetal heart rate region of the target fetus. That is, the target sampling line can be determined using all pixels of the ultrasound images, or it can be determined using local pixels of the ultrasound images; the specific choice depends on the actual situation and is not specifically limited here.

[0141] In this embodiment, the ultrasound imaging device uses a preset image cropping algorithm to divide multiple frames of ultrasound images into multiple image blocks; then the ultrasound imaging device determines the pixel values ​​of the multiple image blocks respectively; and determines the target sampling line based on the variation range of the pixel values ​​of the multiple image blocks.

[0142] Optionally, the preset image cropping algorithms include: image pyramid, image division, etc. The specific selection is made according to the actual situation, and this application embodiment does not impose specific limitations.

[0143] In this embodiment, the ultrasound imaging device determines multiple sampling lines in the multiple ultrasound images based on the pixel values ​​of multiple ultrasound images; then, the ultrasound imaging device determines the periodicity of the heartbeat frames of multiple M-mode ultrasound images corresponding to the multiple sampling lines; the ultrasound imaging device determines a target M-mode ultrasound image whose periodicity of the heartbeat frames meets preset conditions from the multiple M-mode ultrasound images; the ultrasound imaging device determines the sampling line corresponding to the target M-mode ultrasound image as the target sampling line.

[0144] The target M-mode ultrasound image whose heartbeat frame periodicity meets the preset conditions, which can be the best or near-best periodicity. The periodicity can be analyzed based on the fluctuation waveform in the M-mode ultrasound image.

[0145] Optionally, the ultrasound imaging device may randomly determine multiple sampling lines from multiple frames of ultrasound images, or the ultrasound imaging device may determine multiple sampling lines from multiple frames of ultrasound images according to preset positions. The specific selection is made according to the actual situation, and the embodiments of this application do not impose specific limitations.

[0146] In this embodiment, the ultrasound imaging device inputs multiple frames of ultrasound images into a preset neural network; the ultrasound imaging device uses the preset neural network to analyze the variation range of pixel values ​​in the multiple frames of ultrasound images; the ultrasound imaging device determines the target sampling line based on the sampling line position information output by the preset neural network.

[0147] In this embodiment of the application, the neural network is preset to be a Long Short-Term Memory (LSMT) network.

[0148] In this embodiment, the ultrasound imaging device determines multiple sampling lines in the multiple ultrasound images based on the pixel values ​​of multiple frames of ultrasound images. The multiple sampling lines are composed of multiple sets of pixel values, and the multiple sampling lines correspond one-to-one with the multiple sets of pixel values. The ultrasound imaging device obtains the pixel value change amplitude of the multiple sets of pixel values ​​in the multiple frames of ultrasound images. Then, the ultrasound imaging device determines the sampling line with the largest pixel value change amplitude from the multiple sampling lines. The ultrasound imaging device determines the sampling line with the largest pixel value change amplitude as the target sampling line.

[0149] Furthermore, before the ultrasound imaging device determines the target sampling line based on the pixel values ​​of multiple ultrasound images, the ultrasound imaging device uses a preset positioning method to determine the ultrasound image containing the region of interest from the multiple ultrasound images. The region of interest includes the entire fetal heart area or part of the fetal heart area of ​​the target fetus. Then, the ultrasound imaging device determines the pixel value of the region of interest from the ultrasound image containing the region of interest, and determines the target sampling line based on the pixel value of the region of interest.

[0150] Specifically, the ultrasound imaging device uses a preset localization method to determine the ultrasound image containing the region of interest (ROI) from multiple ultrasound images. This can include determining the ultrasound image containing the ROI from multiple ultrasound images using a neural network or traditional methods. Specifically, the ultrasound imaging device can train the neural network using preset ultrasound images, where the preset ultrasound images include the ROI. The ultrasound imaging device uses the trained neural network to perform feature matching on multiple ultrasound images to determine the ultrasound image containing the ROI. Furthermore, it can also determine the ultrasound image containing the ROI and the location and size of the ROI from multiple ultrasound images. Alternatively, the ultrasound imaging device can extract features from the preset ultrasound images; learn the extracted features; and classify the multiple ultrasound images based on the learning results to determine the ultrasound image containing the ROI from multiple ultrasound images. Furthermore, it can also determine the ultrasound image containing the ROI and the location and size of the ROI from multiple ultrasound images.

[0151] Specifically, the identification of regions of interest (ROIs) by ultrasound imaging devices involves two steps: 1. Building a database containing multiple ultrasound images and corresponding ROI calibration results. The ROI calibration results can be set according to the actual task requirements, and can be either a bounding box containing the fetal heart or a mask that precisely segments the fetal heart; 2. Localization and recognition, which involves using machine learning algorithms to learn the features or patterns in the database that can distinguish between ROIs and non-ROIs to achieve the identification and localization of ROIs in ultrasound images.

[0152] Optionally, the preset machine learning algorithms include: sliding window-based methods, deep learning-based bounding-box methods, deep learning-based end-to-end semantic segmentation network methods, and methods that use the above methods to label regions of interest and design classifiers to classify and judge regions of interest based on the labeling results. The specific selection is made according to the actual situation, and this application embodiment does not make specific limitations.

[0153] Specifically, the sliding window-based method is as follows: First, feature extraction is performed on the region within the sliding window. Feature extraction methods can be traditional PCA, LDA, Haar features, texture features, etc., or deep neural networks can be used for feature extraction. Then, the extracted features are matched with the database, and discriminators such as KNN, SVM, random forest, and neural networks are used for classification to determine whether the current sliding window is a region of interest and to obtain its corresponding category.

[0154] Specifically, the deep learning-based Bounding-Box method learns features and regresses parameters on the constructed database by stacking basic convolutional layers and fully connected layers. For the input ultrasound image, the network can directly regress the corresponding Bounding-Box of the region of interest and obtain the category of the tissue structure within the region of interest. Common networks include R-CNN, Fast R-CNN, Faster R-CNN, SSD, YOLO, etc.

[0155] Specifically, end-to-end semantic segmentation network methods based on deep learning involve learning features and regressing parameters on a constructed database by stacking convolutional layers, upsampling layers, or deconvolutional layers. For an input image, the network can directly regress the bounding box of the corresponding region of interest. By adding upsampling or deconvolutional layers, the input and output sizes are made the same, thus directly obtaining the region of interest and its corresponding category of the input image. Common networks include FCN, U-Net, and Mask R-CNN.

[0156] Specifically, in the process of labeling the region of interest using the above method and designing a classifier based on the labeling results to classify and judge the region of interest, the method for classifying and judging the target is to use discriminators such as KNN, SVM, random forest, and neural networks for classification.

[0157] It should be noted that the process by which the ultrasound imaging device determines the target sampling line based on the pixel values ​​of the region of interest is the same as the process by which the ultrasound imaging device determines the target sampling line based on the pixel values ​​of multiple frames of ultrasound images, and will not be described again here.

[0158] S103. Using the target sampling line, generate target M-mode ultrasound images corresponding to multiple frames of ultrasound images.

[0159] After the ultrasound imaging device determines the target sampling line based on the pixels of multiple ultrasound images, the ultrasound imaging device uses the target sampling line to generate the target M-mode ultrasound image corresponding to the multiple ultrasound images.

[0160] In this embodiment of the application, the ultrasound imaging device uses a target sampling line to sample multiple frames of ultrasound images to obtain a target M-mode ultrasound image.

[0161] S104. Determine the heart rate of the target fetus using the target M-mode ultrasound image.

[0162] After the ultrasound imaging device generates target M-mode ultrasound images corresponding to multiple frames of ultrasound images using the target sampling line, the ultrasound imaging device uses the target M-mode ultrasound images to determine the heart rate of the target fetus.

[0163] In this embodiment of the application, the ultrasound imaging device acquires an oscillation curve of the target M-mode ultrasound image; the ultrasound imaging device determines the heart rate of the target fetus based on the oscillation curve.

[0164] Specifically, the ultrasound imaging device acquires the oscillation curve of the M-mode ultrasound image by: the ultrasound imaging device inputting the target M-mode ultrasound image into a preset neural network, and the ultrasound imaging device outputting the oscillation curve through the preset neural network; or, the ultrasound imaging device obtaining the oscillation curve based on the location information of the maximum image gradient of the target M-mode ultrasound image.

[0165] In this embodiment of the application, the preset neural network is a convolutional neural network (CNN).

[0166] Specifically, the ultrasound imaging device determines the target fetal heart rate based on the oscillation curve, including: the ultrasound imaging device finding peaks and troughs in the oscillation curve; the ultrasound imaging device using the peaks and troughs to determine the heartbeat frame period; and then, the ultrasound imaging device determining the target fetal heart rate based on the heartbeat frame period.

[0167] In one possible implementation, the ultrasound imaging device determines a first image block from a target M-mode ultrasound image, where the first image block is any image block in the target M-mode ultrasound image; the ultrasound imaging device searches for a second target image block near the location of the first target image block that has the highest similarity to the first target image block, wherein the first target image block is any image block that starts from the first image block, and the second target image block is the next image block after the first target image block; the ultrasound imaging device determines the cumulative motion displacement between the first target image block and the second target image block; then, the ultrasound imaging device determines the heart rate of the target fetus based on the cumulative motion displacement; and determines the heart rate of the target fetus based on the heart rate frame period.

[0168] Specifically, the ultrasound imaging device crops any image block or pixel value from the target M-mode ultrasound image, then finds the second image block with the highest similarity to the image block or pixel column on the time axis (horizontally and vertically), then finds the third image block with the highest similarity to the second image block on the time axis (horizontally and vertically), then finds the fourth image block with the highest similarity to the third image block on the time axis (horizontally and vertically), and so on, until a displacement corresponding to at least one heartbeat frame cycle is reached. The ultrasound imaging device acquires the cumulative motion displacement of all the identified image blocks with the highest similarity, i.e., the cumulative motion displacement of the first image block, the second image block, the third image block, the fourth image block, up to the last image block. Then, it finds the highest and lowest point positions from the cumulative motion displacement, calculates the heartbeat frame cycle of the target M-mode ultrasound image based on the highest and lowest point positions, and determines the heart rate of the target fetus based on the heartbeat frame cycle.

[0169] It should be noted that the ultrasound imaging device finds the highest and lowest points from the motion displacement as follows: The ultrasound imaging device moves the first image block horizontally, performing similarity calculations by moving one step at a time (where the step size is a preset value, for example, moving 1 pixel horizontally each time). Then, it finds the pixel block with the highest correlation in the vertical direction. This process is repeated, using this found pixel block or pixel column to find the next pixel block with the highest similarity, moving sequentially to these positions until the highest and lowest points of the motion displacement are found.

[0170] In this embodiment, the ultrasound imaging device determines a first image block from a target M-mode ultrasound image, where the first image block is any image block in the M-mode ultrasound image; the ultrasound imaging device searches for a third image block that has the highest similarity to the first image block and is closest to it in the same horizontal direction near at least one preset heartbeat frame cycle position of the target M-mode ultrasound image; the ultrasound imaging device determines the motion displacement between the first image block and the third image block; the ultrasound imaging device determines the heartbeat frame cycle based on the motion displacement; and the ultrasound imaging device determines the heart rate of the target fetus based on the heartbeat frame cycle.

[0171] Specifically, the ultrasound imaging device utilizes image correlation to initially detect the periodicity of the target M-mode ultrasound image. It then selects any image block within the target M-mode ultrasound image and, within N preset heartbeat frame cycles (N being one or more), finds a third image block that has the highest similarity to the first image block and is closest to it in the same horizontal direction. The heartbeat frame cycle is then obtained based on the distance between the first and third image blocks, and the fetal heart rate is further calculated.

[0172] In this embodiment, the ultrasound imaging device calculates the fetal heart rate using formula (1).

[0173] h = 60 x fr / N (1)

[0174] Where N is the heartbeat frame period, fr is the frame rate of the current image frame data, and h is the fetal heart rate value.

[0175] Furthermore, once the ultrasound imaging device determines the heart rate of the target fetus, it displays the heart rate of the target fetus and the target M-mode ultrasound image.

[0176] For example, such as Figure 4 As shown, the procedure for detecting and displaying fetal heart rate using an ultrasound imaging device is as follows:

[0177] 1. The ultrasound imaging device acquires multiple frames of ultrasound images of the target fetus;

[0178] 2. The ultrasound imaging device automatically acquires the optimal sampling lines corresponding to multiple frames of ultrasound images;

[0179] 3. The ultrasound imaging device uses the optimal sampling line to generate target M-mode ultrasound images corresponding to multiple frames of ultrasound images;

[0180] 4. The ultrasound imaging device automatically measures the heart rate of the target fetus based on the target M-mode ultrasound image;

[0181] 5. The ultrasound imaging device displays the target M-mode ultrasound image and the target fetal heart rate.

[0182] Understandably, the ultrasound imaging device automatically acquires the target sampling line based on the pixel values ​​of multiple ultrasound images, generates a meaningful target M-mode ultrasound image based on the target sampling line, and automatically measures the target fetal heart rate based on the target M-mode ultrasound image. This can improve the intelligence of fetal heart rate measurement for M-mode ultrasound and the efficiency of examining fetal development in the womb.

[0183] It should be noted that the target M-mode ultrasound image can be displayed on the interface before the heart rate is determined, and can also be displayed on the interface after the heart rate is determined. The target M-mode ultrasound image and the heart rate can be displayed on the same interface or on different interfaces. The heart rate can be displayed directly on the target M-mode ultrasound image area or on other areas outside the target M-mode ultrasound image area. No specific limitation is made here.

[0184] This application provides a method for detecting fetal heart rate, such as... Figure 5 As shown, the method may include:

[0185] S201. The ultrasound imaging device acquires multiple frames of ultrasound images of the target fetus.

[0186] The fetal heart rate detection method provided in this application embodiment is applicable to scenarios where fetal heart rate is automatically measured based on global images of M-mode ultrasound images.

[0187] Here, the description of S201 in this embodiment is the same as that of S101, and will not be repeated here.

[0188] S202. The ultrasonic imaging device determines the target sampling line based on the pixel values ​​of multiple frames of ultrasonic images.

[0189] After the ultrasound imaging device acquires multiple ultrasound images of the target fetus, it determines the target sampling line based on the pixel values ​​of all regions of the multiple ultrasound images.

[0190] In this embodiment, the ultrasound imaging device determines the target sampling line based on the pixel values ​​of all regions of multiple frames of ultrasound images using four methods, among which...

[0191] The first method is as follows: The ultrasound imaging device uses a preset image cropping algorithm to divide multiple frames of ultrasound images into multiple image blocks; then the ultrasound imaging device determines the pixel values ​​of multiple image blocks respectively; and determines the target sampling line based on the variation range of the pixel values ​​of multiple image blocks.

[0192] Optionally, the preset image cropping algorithms include: image pyramid, image division, etc. The specific selection is made according to the actual situation, and this application embodiment does not impose specific limitations.

[0193] Specifically, the ultrasound imaging device uses an image pyramid to divide multiple ultrasound images into multiple image blocks of different sizes according to a certain size ratio. Then, the ultrasound imaging device calculates the similarity of multiple image blocks of different sizes. The smaller the similarity, the greater the pixel change. The ultrasound imaging device determines the image block with the smallest similarity from multiple image blocks of different sizes, and determines the target sampling line based on the image block with the smallest similarity.

[0194] The second method is as follows: the ultrasound imaging device inputs multiple frames of ultrasound images into a preset neural network; the ultrasound imaging device uses the preset neural network to analyze the variation range of pixel values ​​in the multiple frames of ultrasound images; the ultrasound imaging device determines the target sampling line based on the sampling line position information output by the preset neural network.

[0195] In this embodiment, the preset neural network is LSTM. The ultrasound imaging device inputs multiple frames of ultrasound images into LSTM. LSTM analyzes the variation range of pixel values ​​in multiple frames of ultrasound images and outputs the position information of the target sampling line through LSTM. LSTM uses "gate" to control the discarding or addition of information, thereby realizing the functions of forgetting and remembering. By using the gate function, LSTM can effectively learn the position information of the target sampling line in previous ultrasound images, which is used for the process of determining the target sampling line in the current frame image.

[0196] The third method is as follows: The ultrasound imaging device determines multiple sampling lines in the multiple ultrasound images based on the pixel values ​​of multiple frames of ultrasound images. The multiple sampling lines are composed of multiple sets of pixel values, and the multiple sampling lines and multiple sets of pixel values ​​correspond one-to-one. Then, the ultrasound imaging device obtains the target sampling line by analyzing the multiple sampling lines.

[0197] Optionally, the ultrasound imaging device may randomly determine multiple sampling lines from multiple frames of ultrasound images, or the ultrasound imaging device may determine multiple sampling lines from multiple frames of ultrasound images according to preset positions. The specific selection is made according to the actual situation, and the embodiments of this application do not impose specific limitations.

[0198] Specifically, the ultrasound imaging device obtains the target sampling line by analyzing multiple sampling lines in the following ways: 1. The ultrasound imaging device determines the periodicity of the heartbeat frames of multiple M-mode ultrasound images corresponding to the multiple sampling lines; the ultrasound imaging device determines the target M-mode ultrasound image with the best periodicity of the heartbeat frames from the multiple M-mode ultrasound images; the ultrasound imaging device determines the sampling line corresponding to the target M-mode ultrasound image as the target sampling line; 2. The ultrasound imaging device acquires the pixel value variation amplitude of multiple sets of pixel values ​​in multiple frames of ultrasound images; then, the ultrasound imaging device determines the sampling line with the largest pixel value variation amplitude from the multiple sampling lines; the ultrasound imaging device determines the sampling line with the largest pixel value variation amplitude as the target sampling line.

[0199] In this embodiment of the application, the ultrasound imaging device determines the quality of the heartbeat frame period of multiple M-mode ultrasound images based on the correlation between them. The higher the correlation, the better the periodicity. The correlation is calculated by comparing any pixel block in the multiple M-mode ultrasound images with pixel blocks in other regions of the same image.

[0200] In this embodiment, since the pixel values ​​that the pixels on the sampling line pass through in different ultrasound images are different, the ultrasound imaging device determines the range of pixel value variation by comparing the pixel values ​​at the sampling line positions in multiple ultrasound images.

[0201] S203. The ultrasound imaging device uses the target sampling line to generate target M-mode ultrasound images corresponding to multiple frames of ultrasound images.

[0202] Once the ultrasound imaging device determines the target sampling line, it uses the target sampling line to generate multiple frames of ultrasound images corresponding to the target M-mode ultrasound image.

[0203] In this embodiment of the application, the ultrasound imaging device uses a target sampling line to sample multiple frames of ultrasound images to obtain a target M-mode ultrasound image.

[0204] S204. The ultrasound imaging device uses the target M-mode ultrasound image to determine the heart rate of the target fetus.

[0205] After the ultrasound imaging device generates multiple frames of ultrasound images corresponding to the target M-mode ultrasound image, the ultrasound imaging device uses the target M-mode ultrasound image to determine the heart rate of the target fetus.

[0206] In this embodiment of the application, the ultrasound imaging device determines the heart rate of the target fetus using the target M-mode ultrasound image in three ways, specifically...

[0207] The first method involves using an ultrasound imaging device to acquire an oscillation curve of the target M-mode ultrasound image; the ultrasound imaging device then determines the heart rate of the target fetus based on the oscillation curve.

[0208] The methods by which the ultrasound imaging device acquires the oscillation curve of the target M-mode ultrasound image include: 1. The ultrasound imaging device inputs the target M-mode ultrasound image into a preset neural network, and the ultrasound imaging device outputs the oscillation curve through the preset neural network; 2. The ultrasound imaging device obtains the oscillation curve based on the location information of the maximum image gradient of the target M-mode ultrasound image.

[0209] In this embodiment, the preset neural network is a CNN network. The ultrasound imaging device constructs a database for training the CNN network. The database stores the correspondence between M-mode ultrasound images and corresponding oscillation curves. Then, the ultrasound imaging device inputs the target M-mode ultrasound image into the trained CNN network to obtain the oscillation curve corresponding to the target M-mode ultrasound image.

[0210] In this embodiment, the ultrasound imaging device finds the peak and trough positions from the oscillation curve, where the peak position is the maximum value of the oscillation curve and the trough position is the minimum value of the oscillation curve. The ultrasound imaging device calculates the heartbeat frame period based on the peak and trough. Then, the ultrasound imaging device determines the heart rate of the target fetus based on the heartbeat frame period.

[0211] The second method involves the ultrasound imaging device determining a first image block from the target M-mode ultrasound image, where the first image block is any image block within the target M-mode ultrasound image. The ultrasound imaging device then searches for a second target image block near the location of the first target image block, where the first target image block is any image block originating from the first target image block, and the second target image block is the next image block after the first target image block. The ultrasound imaging device determines the cumulative motion displacement between the first and second target image blocks. Subsequently, the ultrasound imaging device determines the heart rate frame period based on the cumulative motion displacement and the target fetal heart rate based on the heart rate frame period.

[0212] Specifically, the ultrasound imaging device crops any image block or pixel value from the target M-mode ultrasound image. Then, on the time axis (in the horizontal and vertical directions of the target M-mode ultrasound image), it finds the second image block with the highest similarity to the first image block or pixel column. Then, on the time axis (in the horizontal and vertical directions of the target M-mode ultrasound image), it finds the third image block with the highest similarity to the second image block or pixel column. Then, on the time axis (in the horizontal and vertical directions of the target M-mode ultrasound image), it finds the fourth image block with the highest similarity to the third image block or pixel column, and so on, until a displacement corresponding to at least one heartbeat frame cycle is reached. After that, the ultrasound imaging device acquires the cumulative motion displacement between all the determined image blocks, including the first, second, third, and fourth image blocks. Then, it finds the highest and lowest point positions from the cumulative motion displacement. Based on the highest and lowest point positions, the ultrasound imaging device calculates the heartbeat frame cycle of the target M-mode ultrasound image and determines the heart rate of the target fetus based on the heartbeat frame cycle.

[0213] It should be noted that the ultrasound imaging device finds the highest and lowest points from the motion displacement as follows: The ultrasound imaging device moves the first image block horizontally, performing similarity calculations by moving one step at a time (where the step size is a preset value, for example, moving 1 pixel horizontally each time). Then, it finds the pixel block with the highest correlation in the vertical direction. This process is repeated, using this found pixel block or pixel column to find the next pixel block with the highest similarity, moving sequentially to these positions until the highest and lowest points of the motion displacement are found.

[0214] The third method: The ultrasound imaging device determines a first image block from the target M-mode ultrasound image, where the first image block is any image block in the M-mode ultrasound image; the ultrasound imaging device searches for a third image block with the highest similarity and closest distance to the first image block in the same horizontal direction near at least one preset heartbeat frame cycle position in the target M-mode ultrasound image; the ultrasound imaging device determines the motion displacement between the first image block and the third image block; the ultrasound imaging device determines the heartbeat frame cycle based on the motion displacement; and the ultrasound imaging device determines the heart rate of the target fetus based on the heartbeat frame cycle.

[0215] Specifically, the ultrasound imaging device uses image correlation to initially detect the heartbeat frame period of the target M-mode ultrasound image. Then, the device selects any image block within the target M-mode ultrasound image and, within a range of N periods (N being one or more), finds a third image block that has the highest similarity to the first image block and is closest to it in the same horizontal direction. The heartbeat frame period is then obtained based on the distance between the first and third image blocks, and the fetal heart rate is further calculated.

[0216] In this embodiment, the ultrasound imaging device calculates the fetal heart rate using formula (1).

[0217] h = 60 x fr / N (1)

[0218] Where N is the heartbeat frame period, fr is the frame rate of the current image frame data, and h is the fetal heart rate value.

[0219] S205, The ultrasound imaging device displays the heart rate of the target fetus and the target M-mode ultrasound image.

[0220] Once the ultrasound imaging device determines the target M-mode ultrasound image and the target fetal heart rate, the ultrasound imaging device displays the target fetal heart rate and the target M-mode ultrasound image.

[0221] In this embodiment, after generating a target M-mode ultrasound image, the ultrasound imaging device displays the target M-mode ultrasound image on the ultrasound device interface. After the ultrasound imaging device calculates the heart rate of the target fetus, it displays the heart rate of the target fetus on the ultrasound device interface. The heart rate and the target M-mode ultrasound image can be displayed on the same interface or on different interfaces. The heart rate can be displayed on the target M-mode ultrasound image or in other areas outside the target M-mode ultrasound image. No specific limitation is made here.

[0222] In some possible implementations, the heart rate can be displayed directly on the target M-mode ultrasound image. For example, it can be displayed in the upper left or upper right corner of the target M-mode ultrasound image where there is no region of interest, and displayed using a color such as white that is different from the target M-mode ultrasound image.

[0223] S206. The ultrasound imaging device marks the start and end positions of one or more heartbeat frame cycles in the target M-mode ultrasound image.

[0224] After the ultrasound imaging device displays the heart rate of the target fetus and the target M-mode ultrasound image, the ultrasound imaging device marks the start and end positions of one or more heartbeat frame cycles in the target M-mode ultrasound image.

[0225] In this embodiment of the application, the ultrasound imaging device determines the start and end positions of one or more heartbeat frame cycles in the M-mode ultrasound image, and then displays vertical lines at the start and end positions to mark the start and end positions of one or more heartbeat frame cycles.

[0226] It should be noted that the embodiments of this application do not limit the display form of the vertical line, including color, thickness, dotted line, etc.

[0227] Understandably, the ultrasound imaging device automatically acquires the target sampling line based on the pixel values ​​of multiple ultrasound images, generates a meaningful target M-mode ultrasound image based on the target sampling line, and automatically measures the target fetal heart rate based on the target M-mode ultrasound image. This can improve the intelligence of fetal heart rate measurement for M-mode ultrasound and the efficiency of examining fetal development in the womb.

[0228] This application provides a method for detecting fetal heart rate, such as... Figure 6 As shown, the method may include:

[0229] S301, The ultrasound imaging device acquires multiple frames of ultrasound images of the target fetus.

[0230] The fetal heart rate detection method provided in this application embodiment is applicable to scenarios where fetal heart rate is automatically measured based on the region of interest in an M-mode ultrasound image, wherein the region of interest includes the entire fetal heart area or a portion of the fetal heart area.

[0231] Here, the description of S301 in this embodiment is consistent with the description of S201, and will not be repeated here.

[0232] S302. The ultrasound imaging device uses a preset positioning method to determine the ultrasound image containing the region of interest from multiple ultrasound images, wherein the region of interest includes the entire fetal heart area or part of the fetal heart area of ​​the target fetus.

[0233] After the ultrasound imaging device acquires multiple ultrasound images of the target fetus, it uses a preset positioning method to identify the ultrasound image containing the region of interest from the multiple ultrasound images.

[0234] In this embodiment, an ultrasound image containing a region of interest (ROI) is determined from multiple ultrasound images using a neural network or conventional methods. Specifically, the ultrasound imaging device can train the neural network using preset ultrasound images, where the preset ultrasound images include the ROI. The ultrasound imaging device uses the trained neural network to perform feature matching on multiple ultrasound images to determine the ultrasound image containing the ROI. Furthermore, it can also determine the ultrasound image containing the ROI, as well as the location and size of the ROI, from the multiple ultrasound images. Alternatively, the ultrasound imaging device can extract features from the preset ultrasound images; the extracted features are then learned, and based on the learning results, the multiple ultrasound images are classified to determine the ultrasound image containing the ROI. Furthermore, it can also determine the ultrasound image containing the ROI, as well as the location and size of the ROI, from the multiple ultrasound images.

[0235] Specifically, the identification of regions of interest (ROIs) by ultrasound imaging devices involves two steps: 1. Building a database containing multiple ultrasound images and corresponding ROI calibration results. The ROI calibration results can be set according to the actual task requirements, and can be either a bounding box containing the fetal heart or a mask that precisely segments the fetal heart; 2. Localization and recognition, which involves using machine learning algorithms to learn the features or patterns in the database that can distinguish between ROIs and non-ROIs to achieve the identification and localization of ROIs in ultrasound images.

[0236] Optionally, the preset machine learning algorithms include: sliding window-based methods, deep learning-based bounding-box methods, deep learning-based end-to-end semantic segmentation network methods, and methods that use the above methods to label regions of interest and design classifiers to classify and judge regions of interest based on the labeling results. The specific selection is made according to the actual situation, and this application embodiment does not make specific limitations.

[0237] Specifically, the sliding window-based method is as follows: First, feature extraction is performed on the region within the sliding window. Feature extraction methods can be traditional PCA, LDA, Haar features, texture features, etc., or deep neural networks can be used for feature extraction. Then, the extracted features are matched with the database, and discriminators such as KNN, SVM, random forest, and neural networks are used for classification to determine whether the current sliding window is a region of interest and to obtain its corresponding category.

[0238] Specifically, deep learning-based bounding-box methods involve learning features and regressing parameters on a constructed database by stacking convolutional and fully connected layers. For an input ultrasound image, the network can directly regress the bounding box of the corresponding region of interest, while simultaneously obtaining the category of tissue structures within that region. Common networks used in this approach include R-CNN, Fast R-CNN, Faster R-CNN, SSD, and YOLO.

[0239] Specifically, end-to-end semantic segmentation network methods based on deep learning involve learning features and regressing parameters on a constructed database by stacking convolutional layers, upsampling layers, or deconvolutional layers. For an input image, the network can directly regress the bounding box of the corresponding region of interest. By adding upsampling or deconvolutional layers, the input and output sizes are made the same, thus directly obtaining the region of interest and its corresponding category of the input image. Common networks include FCN, U-Net, and Mask R-CNN.

[0240] Specifically, in the process of labeling the region of interest using the above method and designing a classifier based on the labeling results to classify and judge the region of interest, the method for classifying and judging the target is to use discriminators such as KNN, SVM, random forest, and neural networks for classification.

[0241] It should be noted that the region of interest includes the entire fetal heart rate region or a portion of the fetal heart rate region of the target fetus. The specific region to be selected depends on the actual situation, and this application does not impose any specific limitations on the embodiments.

[0242] For example, such as Figure 7 The image shown is a schematic diagram of the automatic positioning effect of the fetal heart region. The ultrasound imaging device positions the fetal heart region as XR*YR in multiple ultrasound images.

[0243] S303, The ultrasound imaging device determines the pixel value of the region of interest from an ultrasound image in which the region of interest exists.

[0244] After the ultrasound imaging device identifies an ultrasound image containing a region of interest from multiple ultrasound images, it determines the pixel value of the region of interest from the ultrasound image containing the region of interest.

[0245] In this embodiment of the application, the ultrasound imaging device determines the pixel value corresponding to the region of interest from an ultrasound image containing the region of interest.

[0246] S304. The ultrasonic imaging device determines the target sampling line based on the pixel values ​​of the region of interest.

[0247] After the ultrasound imaging device determines the pixel values ​​of the region of interest, it determines the target sampling line based on the pixel values ​​of the region of interest.

[0248] In this embodiment, the ultrasound imaging device determines the target sampling line based on the pixel values ​​of the region of interest using four methods, among which...

[0249] The first method is as follows: The ultrasound imaging device uses a preset image cropping algorithm to divide the region of interest into multiple image blocks; then the ultrasound imaging device determines the pixel values ​​of the multiple image blocks; and determines the target sampling line based on the variation range of the pixel values ​​of the multiple image blocks.

[0250] Optionally, the preset image cropping algorithms include: image pyramid, image division, etc. The specific selection is made according to the actual situation, and this application embodiment does not impose specific limitations.

[0251] Specifically, the ultrasound imaging device uses an image pyramid to divide the region of interest into multiple image blocks of different sizes according to a certain ratio. Then, the ultrasound imaging device calculates the similarity of multiple image blocks of different sizes. The smaller the similarity, the greater the pixel change. The ultrasound imaging device determines the image block with the smallest similarity from multiple image blocks of different sizes, and determines the target sampling line based on the image block with the smallest similarity.

[0252] The second method is as follows: the ultrasound imaging device inputs the region of interest into a preset neural network; the ultrasound imaging device uses the preset neural network to analyze the variation range of pixel values ​​in the region of interest; the ultrasound imaging device determines the target sampling line based on the sampling line position information output by the preset neural network.

[0253] In this embodiment, the preset neural network is LSTM. The ultrasound imaging device inputs the region of interest into LSTM. LSTM analyzes the variation range of pixel values ​​in the region of interest and outputs the position information of the target sampling line. LSTM uses "gates" to control the discarding or addition of information, thereby realizing the functions of forgetting and remembering. By using the gate function, LSTM can effectively learn the position information of the target sampling line in the region of interest in the past, and use it in the process of determining the target sampling line in the current region of interest.

[0254] The third method is as follows: The ultrasound imaging device determines multiple sampling lines in multiple frames of ultrasound images based on the pixel values ​​of the region of interest. Each sampling line consists of multiple sets of pixel values, and there is a one-to-one correspondence between the multiple sampling lines and the multiple sets of pixel values. Then, the ultrasound imaging device obtains the target sampling line by analyzing the multiple sampling lines.

[0255] Optionally, the ultrasound imaging device may randomly determine multiple sampling lines from the region of interest, or the ultrasound imaging device may determine multiple sampling lines from the region of interest according to preset positions. The specific selection is made according to the actual situation, and the embodiments of this application do not impose specific limitations.

[0256] Specifically, the ultrasound imaging device obtains the target sampling line by analyzing multiple sampling lines in the following ways: 1. The ultrasound imaging device determines the periodicity of the heartbeat frames of multiple M-mode ultrasound images corresponding to the multiple sampling lines; the ultrasound imaging device selects the target M-mode ultrasound image with the best periodicity of the heartbeat frames from the multiple M-mode ultrasound images; the ultrasound imaging device determines the sampling line corresponding to the target M-mode ultrasound image as the target sampling line; 2. The ultrasound imaging device acquires the pixel value variation amplitude of multiple sets of pixel values ​​in the region of interest; then, the ultrasound imaging device selects the sampling line with the largest pixel value variation amplitude from the multiple sampling lines; the ultrasound imaging device determines the sampling line with the largest pixel value variation amplitude as the target sampling line.

[0257] In this embodiment of the application, the ultrasound imaging device determines the quality of the heartbeat frame period of multiple M-mode ultrasound images based on the correlation between them. The higher the correlation, the better the periodicity. The correlation is calculated by comparing any pixel block in the multiple M-mode ultrasound images with pixel blocks in other regions of the same image.

[0258] In this embodiment, since the pixel values ​​of the pixels on the sampling line are different when they pass through different regions of interest, the ultrasound imaging device determines the range of pixel value variation by comparing the pixel values ​​at the sampling line positions in multiple frames of regions of interest.

[0259] S305. The ultrasound imaging device uses the target sampling line to generate target M-mode ultrasound images corresponding to multiple frames of ultrasound images.

[0260] Once the ultrasound imaging device determines the target sampling line, it uses the target sampling line to generate multiple frames of ultrasound images corresponding to the target M-mode ultrasound image.

[0261] Here, the description of S305 in this embodiment is the same as that of S203, and will not be repeated here.

[0262] S306. The ultrasound imaging device uses the target M-mode ultrasound image to determine the heart rate of the target fetus.

[0263] After the ultrasound imaging device generates a target M-mode ultrasound image, it uses the target M-mode ultrasound image to determine the heart rate of the target fetus.

[0264] Here, the description of S306 in this embodiment is consistent with the description of S204, and will not be repeated here.

[0265] S307, The ultrasound imaging device displays the heart rate of the target fetus and the target M-mode ultrasound image.

[0266] Once the ultrasound imaging device determines the target M-mode ultrasound image and the target fetal heart rate, the ultrasound imaging device displays the target fetal heart rate and the target M-mode ultrasound image.

[0267] Here, the description of S307 in this embodiment is the same as that of S205, and will not be repeated here.

[0268] S308. The ultrasound imaging device marks the start and end positions of one or more heartbeat frame cycles in the target M-mode ultrasound image.

[0269] After the ultrasound imaging device displays the heart rate of the target fetus and the target M-mode ultrasound image, the ultrasound imaging device marks the start and end positions of one or more heartbeat frame cycles in the target M-mode ultrasound image.

[0270] Here, the description of S308 in this embodiment is the same as that of S206, and will not be repeated here.

[0271] Understandably, the ultrasound imaging device automatically acquires the target sampling line based on the pixel values ​​of multiple ultrasound images, generates a meaningful target M-mode ultrasound image based on the target sampling line, and automatically measures the target fetal heart rate based on the target M-mode ultrasound image. This can improve the intelligence of fetal heart rate measurement for M-mode ultrasound and the efficiency of examining fetal development in the womb.

[0272] like Figure 8 As shown in the embodiments of this application, a method for detecting fetal heart rate is also provided, characterized in that the method includes:

[0273] S401. Obtain ultrasound images of the target fetus;

[0274] S402. Determine the target sampling line based on the pixel values ​​of the ultrasound image;

[0275] S403. Using the target sampling line, generate the target M-mode ultrasound image;

[0276] S404. Determine the heart rate of the target fetus using the target M-mode ultrasound image.

[0277] The descriptions of technical features similar to those in the above embodiments are also applicable to this embodiment, and will not be repeated here.

[0278] S401. Obtain ultrasound images of the target fetus.

[0279] Ultrasound imaging equipment acquires ultrasound images of the target fetus, including acquiring a single frame or multiple frames. Specifically, ultrasound images of the target fetus can be acquired locally or in real-time.

[0280] S402. Determine the target sampling line based on the pixel values ​​of the ultrasound image.

[0281] Once the ultrasound imaging device acquires an ultrasound image of the target fetus, it can determine the target sampling line based on the pixel values ​​of one or more frames of ultrasound images.

[0282] In one embodiment, determining a target sampling line based on pixel values ​​of an ultrasound image may include: determining a fetal heart region from the ultrasound image; determining a target location from the fetal heart region; and determining a target sampling line based on the target location.

[0283] The method for determining the fetal heart region from an ultrasound image is similar to the method for determining the region of interest (ROI) from an ultrasound image. The ultrasound imaging device can be trained using preset ultrasound images, which include the fetal heart region. Based on the training results, the ultrasound imaging device performs feature matching on the ultrasound images to determine the fetal heart region. Alternatively, the ultrasound imaging device can perform feature learning on preset ultrasound images and predict the ultrasound images based on the learning results to determine the fetal heart region. In one embodiment, the ultrasound imaging device's identification of the fetal heart region can be divided into two steps: 1. Constructing a database containing multiple ultrasound images and corresponding fetal heart region calibration results. The fetal heart region calibration results can be set according to actual task requirements; they can be a bounding box (ROI) containing the fetal heart or a mask that precisely segments the fetal heart; 2. Localization and recognition, i.e., using machine learning algorithms to learn features or patterns in the database that can distinguish between the fetal heart region and non-fetal heart regions to achieve the identification and localization of the fetal heart region in the ultrasound image. Optionally, the preset machine learning algorithms include: a sliding window-based method, a deep learning-based bounding-box method, a deep learning-based end-to-end semantic segmentation network method, and a method that uses the above methods to calibrate the fetal heart region and designs a classifier to classify and judge the fetal heart region based on the calibration results. The specific selection is made according to the actual situation, and the embodiments of this application do not impose specific limitations.

[0284] The target location within the fetal heart region can be the geometric center of the fetal heart, the part with the largest fetal heart movement amplitude, or the location of a fetal heart valve—any location within the fetal heart region that can represent fetal heart movement. In one embodiment, determining the target location from the fetal heart region can include: determining the target location from the fetal heart region based on the geometric relationship between the target location and the fetal heart region; for example, determining the geometric center of the determined fetal heart region as the target location; or, through clinical research, determining the location of the fetal heart valve at a specific geometric position within the fetal heart region, determining the valve's position by identifying that specific geometric position within the determined fetal heart region, and then determining the identified valve position as the target location. Determining the target location from the fetal heart region can also include: inputting the ultrasound image into a preset neural network; determining the target location within the fetal heart region based on the output of the preset neural network; or, extracting features of the fetal heart region from the ultrasound image and determining the target location from the fetal heart region using a classifier.

[0285] In one embodiment, determining the target sampling line based on the pixel values ​​of the ultrasound image includes: directly detecting the target location from the ultrasound image, and determining the target sampling line based on the target location. Detecting the target location from the ultrasound image may include: inputting the ultrasound image into a preset neural network; determining the target location based on the output of the preset neural network, specifically, through methods such as segmentation, target detection, and point regression; or extracting features of the target location in the ultrasound image and determining the target location from the ultrasound image using a classifier, specifically, through algorithms such as LBP and PCA to extract features from the ultrasound image.

[0286] In one embodiment, determining the target sampling line based on the target location includes: identifying a sampling line passing through the target location as the target sampling line. The target sampling line can be a sampling line passing through the target location at any angle. In one embodiment, the emission delay of each element of the probe can be adjusted to control the deflection of the ultrasonic scanning line so that the ultrasonic scanning line passes through the target location, and the ultrasonic scanning line at this time is identified as the target sampling line.

[0287] In one embodiment, determining the target sampling line based on the pixel value of the ultrasound image can be as follows: determine the fetal heart region in the ultrasound image, divide the fetal heart region into equal parts in the width direction according to a certain ratio, which can be preset by the machine or set by the user, obtain multiple sampling lines sequentially at equal intervals, and then analyze the positions passed by the multiple sampling lines to determine whether they pass through the target position. If they pass through the target position, the sampling line is determined as the target sampling line.

[0288] S403. Using the target sampling line, generate the target M-mode ultrasound image.

[0289] Ultrasound imaging equipment utilizes a target sampling line to generate an M-mode ultrasound image of the target. This M-mode ultrasound image can be generated from multiple corresponding ultrasound images; that is, by sampling from multiple ultrasound images using the target sampling line, a target M-mode ultrasound image is obtained. Alternatively, the target sampling line can be used to acquire M-mode ultrasound images of the target tissue individually. For example, when the ultrasound image is a B-mode ultrasound image, a single line of data can be extracted from the B-mode ultrasound image using the target sampling line, and the target M-mode ultrasound image can be obtained by sampling from multiple B-mode ultrasound images. For example, an M-mode ultrasound image can be obtained by emitting M-mode ultrasound waves individually towards the target tissue using the target sampling line. The M-mode ultrasound waves can be emitted simultaneously with B-mode ultrasound waves, or alternately with B-mode ultrasound waves.

[0290] S404. Determine the heart rate of the target fetus using the target M-mode ultrasound image.

[0291] The description of how ultrasound imaging equipment uses target M-mode ultrasound images to determine the heart rate of the target fetus can be found in the above embodiments and will not be repeated here. Determining the heart rate of the target fetus using target M-mode ultrasound images can be done automatically by the ultrasound imaging equipment, or it can be done by receiving user input from the ultrasound imaging equipment.

[0292] This application also provides a method for detecting fetal heart rate, including:

[0293] Acquire ultrasound images of the target fetus;

[0294] Determining the target sampling line based on ultrasound images;

[0295] Using the target sampling line, generate a target M-mode ultrasound image;

[0296] The heart rate of the target fetus was determined using the target M-mode ultrasound image.

[0297] The descriptions of technical features similar to those in the above embodiments are also applicable to this embodiment and will not be repeated here. Specifically, determining the target sampling line based on the ultrasound image can be done automatically based on the ultrasound image or by receiving user input. Automatically determining the target sampling line can be based on pixel values ​​of the ultrasound image or other image features of the ultrasound image. Receiving user input to determine the target sampling line can be based on the user selecting a target sampling line on the ultrasound image.

[0298] This embodiment also provides a method for detecting fetal M-mode ultrasound images, including:

[0299] Acquire ultrasound images of the target fetus;

[0300] Determining the target sampling line based on ultrasound images;

[0301] Target sampling lines are used to generate target M-mode ultrasound images.

[0302] The descriptions of technical features similar to those in the above embodiments are also applicable to this embodiment, and will not be repeated here.

[0303] The acquisition of ultrasound images of the target fetus can be done through local acquisition and real-time acquisition.

[0304] Among them, determining the target sampling line based on ultrasound images can be done automatically or manually, and can be based on multiple frames of ultrasound images or a single frame of ultrasound images.

[0305] Specifically, generating a target M-mode ultrasound image using a target sampling line can be achieved by sampling from multiple ultrasound images using the target sampling line; alternatively, the target sampling line can be used to acquire M-mode ultrasound images of a single target tissue. Furthermore, doctors can use the M-mode ultrasound image for diagnosis, including but not limited to determining the fetal heart rate.

[0306] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0307] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0308] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0309] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims. All of these forms are within the protection scope of this application.

Claims

1. A method for detecting fetal heart rate, characterized in that, The method includes: Acquire ultrasound images of the target fetus; Determine the target sampling line based on the ultrasound image; Using the target sampling line, a target M-mode ultrasound image is generated; The heart rate of the target fetus was determined using the target M-mode ultrasound image; Determining the heart rate of the target fetus using the target M-mode ultrasound image includes: A first image block is determined from the target M-mode ultrasound image, wherein the first image block is any image block in the target M-mode ultrasound image; a second target image block with the highest similarity to the first target image block is found near the location of the first target image block, wherein the first target image block is any image block starting from the first image block, and the second target image block is the next image block after the first target image block; the cumulative motion displacement between the first target image block and the second target image block is determined; the heart rate of the target fetus is determined based on the cumulative motion displacement; the heart rate of the target fetus is determined based on the heart rate frame period. Alternatively, a first image block is determined from the target M-mode ultrasound image, wherein the first image block is any image block in the target M-mode ultrasound image; near at least one preset heartbeat frame cycle position of the target M-mode ultrasound image, a third image block with the highest similarity and closest distance to the first image block is found in the same horizontal direction as the first image block; the motion displacement between the first image block and the third image block is determined; the heartbeat frame cycle is determined based on the motion displacement; and the heart rate of the target fetus is determined based on the heartbeat frame cycle.

2. The method according to claim 1, characterized in that, The method further includes: Displays the heart rate of the target fetus and the target M-mode ultrasound image.

3. The method according to claim 1, characterized in that, The method further includes: The start and end positions of one or more of the heartbeat frame cycles are marked in the target M-mode ultrasound image.

4. The method according to claim 1, characterized in that, The determination of the target sampling line based on the ultrasound image includes: Determine the fetal heart region from the ultrasound images; Determine the target location from the fetal heart region; Determine the target sampling line based on the target location.

5. The method according to claim 4, characterized in that, Determining the target location from the fetal heart region includes: The target location is determined from the fetal heart region by the geometric relationship between the target location and the fetal heart region.

6. The method according to claim 4, characterized in that, The determination of the target sampling line based on the ultrasound image includes: The ultrasound image is input into a preset neural network; the target location is determined based on the output of the preset neural network; or, features of the target location in the ultrasound image are extracted, and the target location is determined from the ultrasound image using a classifier. Determine the target sampling line based on the target location.

7. The method according to any one of claims 4 to 6, characterized in that, The determination of the target sampling line based on the ultrasound image includes: The sampling line passing through the target location is determined as the target sampling line.

8. A method for detecting fetal M-mode ultrasound images, characterized in that, The method includes: Control the ultrasound probe to emit ultrasound waves toward the target fetus and acquire the echo signal of the ultrasound waves; Based on the echo signal of the ultrasound, a target M-mode ultrasound image of the target fetus is obtained; The heart rate of the target fetus is automatically determined using the acquired target M-mode ultrasound images. The automatic determination of the target fetal heart rate using the acquired target M-mode ultrasound image includes: A first image block is determined from the target M-mode ultrasound image, wherein the first image block is any image block in the target M-mode ultrasound image; a second target image block with the highest similarity to the first target image block is found near the location of the first target image block, wherein the first target image block is any image block starting from the first image block, and the second target image block is the next image block after the first target image block; the cumulative motion displacement between the first target image block and the second target image block is determined; the heart rate of the target fetus is determined based on the cumulative motion displacement; the heart rate of the target fetus is determined based on the heart rate frame period. Alternatively, a first image block is determined from the target M-mode ultrasound image, wherein the first image block is any image block in the target M-mode ultrasound image; near at least one preset heartbeat frame cycle position of the target M-mode ultrasound image, a third image block with the highest similarity and closest distance to the first image block is found in the same horizontal direction as the first image block; the motion displacement between the first image block and the third image block is determined; the heartbeat frame cycle is determined based on the motion displacement; and the heart rate of the target fetus is determined based on the heartbeat frame cycle.

9. The method according to claim 8, characterized in that, The method further includes: The start and end positions of one or more of the heartbeat frame cycles are marked in the target M-mode ultrasound image.

10. An ultrasonic imaging device, characterized in that, The ultrasound imaging device includes: probe; A transmitting circuit that excites the probe to emit ultrasound waves toward the target fetus; A receiving circuit, wherein the receiving circuit receives ultrasound echoes returned from the target fetus via the probe to obtain ultrasound echo signals; A processor that processes the ultrasound echo signal to obtain an ultrasound image of the target fetus; A display showing the ultrasound image; The processor also executes the fetal heart rate detection method as described in any one of claims 1-9.

Images