Doppler imaging method and color Doppler diasonograph

Abstract

The invention relates to the technical field of image processing, in particular to a Doppler imaging method and a color Doppler diasonograph, and aims to provide a scheme for automatically positioning a target detection area during Doppler imaging and optimizing digital gain parameters based on a target blood vessel branch corresponding to the target detection area. According to the embodiment of the invention, an original image is subjected to image segmentation, and at least one candidate region is determined; determining a target detection area from the candidate areas based on the pixel information of each candidate area; obtaining at least one frame of reference image comprising a target blood vessel branch corresponding to a target detection area in the original image; performing image feature extraction on each frame of reference image, and determining the current blood flow state of the target blood vessel branch based on the image features extracted from each frame of reference image; and determining a digital gain parameter based on the blood flow state, and adjusting each frame of reference image based on the digital gain parameter.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and in particular to a Doppler imaging method and a color Doppler ultrasonic diagnostic apparatus. Background Art

[0002] Color Doppler imaging is a commonly used technical means in clinical ultrasound detection. This technology obtains relevant information about blood flow through the information of changes in sound wave frequency, and generates intuitive blood flow images after processing to assist in determining the nature and state of blood flow.

[0003] When performing color Doppler imaging, color Doppler signal processing is usually performed on the image of a preset ROI (region of interest) in the original image, and the processed image is adjusted based on a preset gain to obtain a target image; in this process, ROI and gain have a greater impact on the imaging effect. Appropriate ROI and gain can effectively improve the imaging effect and avoid undersaturation or oversaturation of blood flow.

[0004] In the related art, when entering the color Doppler examination mode, the preset ROI and gain are used by default, and the user manually adjusts them based on personal experience during use. In this way, each time the color Doppler examination mode is entered, the user needs to manually adjust the ROI and gain, which increases the operation process, and because the adjustment depends entirely on the user's personal experience, the final imaging effect is not good. Summary of the invention

[0005] The purpose of the present application is to provide a Doppler imaging method and a color Doppler ultrasound diagnostic instrument, which are used to provide a solution for automatically locating a target detection area during Doppler imaging and optimizing digital gain parameters based on target blood vessel branches corresponding to the target detection area.

[0006] In a first aspect, the present application provides a color Doppler ultrasound diagnostic instrument, which includes a probe, a processor and a display screen: the probe is used to collect an original image containing a target detection part of an object and at least one frame of reference image corresponding to the original image; the processor is used to perform image segmentation on the original image to determine at least one candidate area in the original image, wherein each candidate area includes at least one vascular branch of the target detection part; based on pixel information of each candidate area corresponding to the original image, determine the target detection area from at least one candidate area; obtain at least one frame of reference image collected by the probe, the reference image includes the target vascular branch corresponding to the target detection area in the original image; perform image feature extraction on each frame of reference image respectively, and determine the current blood flow state of the target vascular branch based on the image features extracted from each frame of reference image; determine a digital gain parameter based on the current blood flow state of the target vascular branch, and adjust each frame of reference image based on the digital gain parameter; a display screen is used to display each adjusted frame of reference image in a display interface.

[0007] In a second aspect, the present application provides a Doppler imaging method, the method comprising:

[0008] An original image including a target detection part of an object is acquired, and image segmentation is performed on the original image to determine at least one candidate area in the original image, wherein each candidate area includes at least one blood vessel branch of the target detection part; based on pixel information of each candidate area corresponding to the original image, a target detection area is determined from at least one candidate area; at least one frame of reference image corresponding to the original image is acquired, and the reference image includes a target blood vessel branch corresponding to the target detection area in the original image; image feature extraction is performed on each frame of the reference image, and based on the image features extracted from each frame of the reference image, a current blood flow state of the target blood vessel branch is determined; a digital gain parameter is determined based on the current blood flow state of the target blood vessel branch, and each frame of the reference image is adjusted based on the digital gain parameter.

[0009] In a third aspect, the present application provides a Doppler imaging device, comprising:

[0010] A segmentation module, used to perform image segmentation on the original image to determine at least one candidate region in the original image, wherein each candidate region includes at least one blood vessel branch of a target detection site;

[0011] A determination module, configured to determine a target detection area from at least one candidate area based on pixel information of each candidate area corresponding to the original image;

[0012] An acquisition module, used to acquire at least one frame of reference image corresponding to the original image, wherein the reference image includes a target blood vessel branch corresponding to the target detection area in the original image;

[0013] A feature extraction module is used to extract image features from each frame of reference image, and determine the current blood flow state of the target blood vessel branch based on the image features extracted from each frame of reference image;

[0014] The adjustment module is used to determine the digital gain parameter based on the current blood flow state of the target blood vessel branch, and adjust each frame of the reference image based on the digital gain parameter.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium. When instructions in the computer-readable storage medium are executed by an electronic device, the electronic device is enabled to perform the Doppler imaging method as described in the second aspect above.

[0016] In a fifth aspect, the present application provides a computer program product, comprising a computer program: when the computer program is executed by a processor, the Doppler imaging method of the second aspect described above is implemented.

[0017] The technical solution provided by the embodiments of the present application brings at least the following beneficial effects:

[0018] By determining at least one candidate area in the original image, each candidate area includes at least one blood vessel branch of the target detection part, so that the influence of the area outside the blood vessel branch on the subsequent determination of the target detection area can be screened out; based on the pixel information of each candidate area, the target detection area is automatically determined from the candidate area, which simplifies the user's operation process; at least one frame reference image of the target blood vessel branch in the target detection area is collected, and based on the image features extracted from each frame reference image, the current blood flow state of the target blood vessel branch is determined, and the current blood flow state is determined by comprehensively analyzing the feature changes of multiple frames of images, so that the determination of the blood flow state is more accurate; based on the current blood flow state, a digital gain parameter is determined, so that the value of the digital gain parameter is more accurate, and then after adjusting each frame reference image based on the digital gain parameter, the imaging effect of each frame reference image can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. Obviously, the drawings introduced below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 This is an optional application scenario of the Doppler imaging method of the embodiment of the present application;

[0021] Figure 2 This is an optional application scenario of the Doppler imaging method of the embodiment of the present application;

[0022] Figure 3 This is a flow chart of Doppler imaging according to an embodiment of the present application;

[0023] Figure 4 A schematic diagram of a possible sample image of an embodiment of the present application;

[0024] Figure 5 A schematic diagram of a possible labeling result of a sample image according to an embodiment of the present application;

[0025] Figure 6 A flowchart of a possible digital gain parameter determination according to an embodiment of the present application;

[0026] Figure 7 A flowchart of a possible target detection area determination according to an embodiment of the present application;

[0027] Figure 8 A flowchart of a possible prediction parameter determination according to an embodiment of the present application;

[0028] Fig. 9 A schematic diagram of a possible original image according to an embodiment of the present application;

[0029] Fig.10 A schematic diagram of a possible target detection area according to an embodiment of the present application;

[0030] Fig.11 A schematic diagram of a possible center point of a target candidate area in an embodiment of the present application;

[0031] Fig.12 A schematic diagram of a possible blood flow state detection model according to an embodiment of the present application;

[0032] Fig.13 A flow chart of a possible blood flow state determination process according to an embodiment of the present application;

[0033] Fig.14 A flowchart of a possible updating process of the first feature vector in an embodiment of the present application;

[0034] Fig.15 A schematic diagram of a possible transformer network according to an embodiment of the present application;

[0035] Fig.16 A flowchart of a possible second feature vector extraction process of an embodiment of the present application;

[0036] Fig.17 The figure is a schematic diagram of the structure of a Doppler imaging device according to an embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described in conjunction with the drawings in the embodiment of the present application. Among them, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application. In addition, in the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiment of the present application, "multiple" refers to two or more than two. In the following, the terms "first" and "second" are used only for descriptive purposes and cannot be understood as implying or implying relative importance or implicitly indicating the number of technical features indicated.

[0038] In order to facilitate understanding of a Doppler imaging method and a color Doppler ultrasound diagnostic apparatus provided in the embodiments of the present application, some terms in the embodiments of the present application are explained below to facilitate understanding by those skilled in the art.

[0039] (1) B-ultrasound: When using ultrasound to diagnose diseases, the diagnosis of diseases in the form of grayscale or brightness mode is called "two-dimensional display". Because the first letter of brightness is B, it is called B-ultrasound, also known as two-dimensional ultrasound or grayscale ultrasound.

[0040] (2) Attention mechanism: A machine learning module that can sense the importance of each part of the data. It can automatically sense the contribution of the input data to the output, so that effective information can be emphasized and the model performance can be improved. The transformer network using the attention mechanism can more fully obtain and use global information. By mapping the input into an ordered feature vector, it can more fully use the sequence information. After serialization in space and time, it can more fully understand the spatial sequence relationship and the time series relationship, which has a strong advantage in video analysis and processing.

[0041] (3) Residual Network (ResNet): A type of convolutional neural network that is easy to optimize and can improve accuracy by increasing the depth. Its internal residual blocks use skip connections to alleviate the gradient vanishing problem caused by increasing the depth of deep neural networks.

[0042] (4) Multi-Layer Perceptron (MLP): A type of artificial neural network with a directional structure that maps a set of input vectors to a set of output vectors. As a generalization of the perceptron, it overcomes the shortcoming of the perceptron that it cannot recognize linearly inseparable data. It can be viewed as a directed graph consisting of multiple node layers, with each layer fully connected to the next layer. Except for the input node, each node is a neuron (or processing unit) with a nonlinear activation function.

[0043] Color Doppler imaging is a commonly used technical means in clinical ultrasound detection. This technology obtains relevant information about blood flow through the information of sound wave frequency changes, and generates intuitive blood flow images after processing to assist in judging the nature and state of blood flow. When performing color Doppler imaging, color Doppler signal processing is usually performed on the image of the preset ROI in the original B-ultrasound image, and the processed image is adjusted based on the digital gain to generate a blood flow-related image. In this process, the settings of ROI and digital gain have a great influence on the imaging effect. A suitable ROI that can include the target area can improve the temporal resolution and imaging effect. Digital gain can adjust the signal intensity to adjust the image effect; too small a gain will cause blood flow to be insufficient, and too large a gain will cause blood flow to overflow. Appropriate gain can improve image quality and make detailed information richer.

[0044] In the related art, when entering the color Doppler examination mode, the preset ROI and gain are used by default, and the user manually adjusts them based on personal experience during use. In this way, each time the color Doppler examination mode is entered, the user needs to manually adjust the ROI and gain, which increases the operation process, and because the adjustment depends entirely on the user's personal experience, the final imaging effect is not good.

[0045] At present, the optimization of color Doppler imaging mainly uses traditional image processing technology. The extraction of features in traditional methods depends on manually designed extractors, and the design of extractors is targeted to a certain extent. Different parameters need to be designed for different usage situations to cope with data changes, such as gradients, geometric features, grayscale features, etc., to analyze the blood flow data information in the scanned B-ultrasound images or ROI, and the accuracy and adaptability are limited. The method for optimizing parameters such as ROI and gain is mainly to obtain the image results under the current parameter value by continuously changing the parameter value, and then evaluate the image through the image quality evaluation module, and use the parameter with the highest image quality as the final output parameter. This type of method repeats the process of image generation and image evaluation after parameter adjustment many times, and the efficiency and accuracy cannot be effectively guaranteed.

[0046] Based on the above problems, the embodiments of the present application propose a Doppler imaging method and a color Doppler ultrasound diagnostic instrument to provide a solution for automatically locating the target detection area during Doppler imaging and optimizing the digital gain parameters based on the target blood vessel branches corresponding to the target detection area.

[0047] Optionally, the Doppler imaging method proposed in the embodiment of the present application can be used to generate ROI when initially entering the color Doppler mode, automatically generate the optimal ROI, and automatically adjust ROI parameters and digital gain when performing color Doppler mode examination to optimize blood flow imaging.

[0048] The following describes an optional application scenario of the Doppler imaging method provided by the embodiment of the present application in conjunction with the accompanying drawings. Figure 1 The color Doppler ultrasound diagnostic apparatus 10 shown includes a probe 101, a processor 102 and a display screen 103, wherein:

[0049] The probe 101 is used to collect an original image including a target detection part of the object and at least one frame of reference image corresponding to the original image; and transmit the collected original image and at least one frame of reference image to the processor 102;

[0050] The processor 102 is configured to perform image segmentation on the original image, determine at least one candidate area in the original image, wherein each candidate area includes at least one blood vessel branch of a target detection site; determine a target detection area from the at least one candidate area based on pixel information of each candidate area corresponding to the original image; obtain at least one frame of reference image, wherein the reference image includes a target blood vessel branch corresponding to the target detection area in the original image; perform image feature extraction on each frame of reference image, and determine a current blood flow state of the target blood vessel branch based on the image features extracted from each frame of reference image; determine a digital gain parameter based on the current blood flow state of the target blood vessel branch, and adjust each frame of reference image based on the digital gain parameter;

[0051] The display screen 103 is used to display the adjusted reference image frames in the display interface.

[0052] In addition, the present application embodiment also provides an optional application scenario of the Doppler imaging method, such as Figure 2 As shown, it includes an ultrasound device 20 and a server 21;

[0053] The ultrasound device 20 includes a display screen 201 and a probe 202;

[0054] The ultrasound device 20 collects an original image including a target detection part of the object and at least one frame of reference image corresponding to the original image through the probe 202; sends the collected original image and at least one frame of reference image to the server 21; and receives the adjusted reference images returned by the server 21, and displays them on the display interface of the display screen 201;

[0055] The server 21 is used to receive the original image sent by the ultrasound device 20, perform image segmentation on the original image, determine at least one candidate area in the original image, wherein each candidate area includes at least one blood vessel branch of a target detection site; determine a target detection area from at least one candidate area based on pixel information of each candidate area corresponding to the original image; receive at least one frame of reference image sent by the ultrasound device 20, wherein the reference image includes a target blood vessel branch corresponding to the target detection area in the original image; perform image feature extraction on each frame of reference image, and determine a current blood flow state of the target blood vessel branch based on the image features extracted from each frame of reference image; determine a digital gain parameter based on the current blood flow state of the target blood vessel branch, adjust each frame of reference image based on the digital gain parameter, and send each adjusted frame of reference image to the ultrasound device 20.

[0056] Of course, the method provided in the embodiment of the present application is not limited to Figure 1 or Figure 2 The application scenario shown can also be used in other possible application scenarios, which are not limited in the embodiments of the present application.

[0057] like Figure 3 As shown, a flowchart of Doppler imaging in an embodiment of the present application, the specific steps are as follows:

[0058] Step S301, obtaining an original image containing a target detection part of the object, performing image segmentation on the original image, and determining at least one candidate region in the original image;

[0059] Each candidate region includes at least one blood vessel branch of the target detection part; in a possible implementation, each candidate region includes a blood vessel branch of the target detection part, that is, a complete blood vessel region. Optionally, the target detection region may be a region including the neck, abdomen, etc. of the object (i.e., the person being detected), and its original image may be a B image.

[0060] Optionally, the embodiment of the present application can input the acquired original image into a trained semantic segmentation network model, use the trained semantic segmentation network model to perform image segmentation on the original image, and obtain at least one candidate region output by the model; the training process of the semantic segmentation network model is described below:

[0061] Collect images of different people, angles, positions, brightness, dynamic ranges and detection parts as sample images. Figure 4 This is a schematic diagram of a possible sample image. The candidate regions (i.e., the regions corresponding to the blood vessel branches) in each sample image are labeled to obtain a mask labeling result of the same size as the original sample image. Figure 5 is a possible sample image (i.e. Figure 4 Schematic diagram of the annotation results of the corresponding sample image).

[0062] The acquired sample images are divided into a training set and a test set, for example, in a ratio of 7:3, and image enhancement processing such as cropping, affine transformation, and random noise is performed on the data in the training set to further enrich the data set and increase the complexity of the data set.

[0063] The sample images in the training set are input into the semantic segmentation model, and the annotation results corresponding to the sample images in the training set are used as the target output to sequence the semantic segmentation model.

[0064] After the training is completed, the semantic segmentation model is tested using the sample images in the test set and their corresponding annotation results. If the test passes, the trained semantic segmentation network model is obtained. It should be noted that if the test fails, a new sample image is obtained to retrain the semantic segmentation network model.

[0065] Step S302, determining a target detection area from at least one candidate area based on pixel information of each candidate area corresponding to the original image;

[0066] Optionally, based on the size, grayscale distribution, edge gradient and other features of each candidate area corresponding to the original image, the target detection area is determined from the candidate areas.

[0067] Optionally, the target detection region is ROI. Determining the target detection region is to determine the center point position, size and preset angle of the target detection region.

[0068] Step S303, obtaining at least one frame of reference image corresponding to the original image;

[0069] Among them, the reference image includes the target blood vessel branch corresponding to the target detection area in the original image; the above-mentioned reference image is an image of continuous frames corresponding to the above-mentioned target detection area. In a specific implementation, it can be obtained by receiving at least one frame of reference image corresponding to the original image input by the user, or by the above-mentioned color Doppler ultrasound diagnostic instrument automatically identifying the target blood vessel branch corresponding to the target detection area and automatically acquiring at least one frame of reference image.

[0070] Optionally, the reference image is a four-channel image consisting of a B-ultrasound image, a blood flow velocity image, a blood flow energy image, and a blood flow signal variance image.

[0071] Step S304, extracting image features from each frame of reference image, and determining the current blood flow state of the target blood vessel branch based on the image features extracted from each frame of reference image;

[0072] Step S305, determining a digital gain parameter based on the current blood flow state of the target blood vessel branch, and adjusting each frame of the reference image based on the digital gain parameter;

[0073] Optionally, in an embodiment of the present application, a default value of a digital gain parameter can be pre-set, and the default value of the digital gain parameter can be adjusted based on the current blood flow state of the target blood vessel branch. Alternatively, the default value of the digital gain parameter is not pre-set, and the value of the digital gain parameter is determined directly based on the current blood flow state of the target blood vessel branch.

[0074] In a possible implementation, Figure 6 As shown, the above-mentioned determination of the digital gain parameter based on the current blood flow state of the target blood vessel branch specifically includes the following steps:

[0075] Step S601, matching the current blood flow state of the target blood vessel branch with a preset blood flow state type, and determining a target blood flow state type corresponding to the current blood flow state of the target blood vessel branch;

[0076] Optionally, the preset blood flow state type may be different levels set for the saturation state of the blood flow, for example, the blood flow state type may be set to severe supersaturation, moderate supersaturation, slight supersaturation, normal, slight undersaturation, moderate undersaturation and severe undersaturation.

[0077] Step S602: Determine the digital gain parameter corresponding to the target blood flow state type based on the corresponding relationship between the blood flow state type and the digital gain parameter.

[0078] Optionally, when a default value of a digital gain parameter is set in advance and the default value of the digital gain parameter is adjusted based on the current blood flow state of the target blood vessel branch, the correspondence between the blood flow state type and the numerical gain can be the adjustment direction and adjustment amplitude of the digital gain parameter value corresponding to the pre-defined different blood flow state types, for example, the value is increased by 5; when the default value of the digital gain parameter is not set in advance and the value of the digital gain parameter is determined directly based on the current blood flow state of the target blood vessel branch, the correspondence between the blood flow state type and the numerical gain can be the pre-defined values ​​of the digital gain parameter corresponding to the different blood flow state types.

[0079] The following is an explanation of the process of determining the target detection area in step S302:

[0080] like Figure 7 As shown, the above-mentioned determining the target detection area from at least one candidate area based on the pixel information of each candidate area corresponding to the original image specifically includes the following steps:

[0081] Step S701, for any candidate region in the original image, determine the prediction parameter corresponding to the candidate region according to the pixel information of the candidate region;

[0082] Among them, the prediction parameters are used to characterize the blood flow strength of the vascular branches contained in the candidate area;

[0083] Optionally, the above pixel information includes size information of the candidate area, a first grayscale value of a pixel in the candidate area, and a second grayscale value of a pixel in an associated area corresponding to the candidate area.

[0084] In some embodiments, the size information of the candidate area refers to the length and width of the candidate area (a×b, where a and b are the number of pixels in the length and width directions respectively), the first grayscale value of the pixel in the candidate area may be the average grayscale value of each pixel in the candidate area, and the second grayscale value of the pixel in the associated area corresponding to the candidate area may be the average grayscale value of each pixel in the associated area of ​​the candidate area. The associated area may be set based on user needs. In some embodiments, the associated area may be an area on both sides of a blood vessel branch included in the candidate area within a preset range of the blood vessel branch. For example, if the candidate area includes a blood vessel branch with a horizontal blood flow direction, the associated area of ​​the candidate area is an area within a preset range of the upper and lower sides of the blood vessel branch, wherein the preset distance may be set based on user needs.

[0085] Optional, such as Figure 8 As shown, the prediction parameters corresponding to each candidate region can be determined based on the following method:

[0086] Step S801, normalizing the size information, the first grayscale value, and the second grayscale value in the pixel information of each candidate area;

[0087] In a specific implementation, the size information corresponding to each candidate region is normalized to obtain the normalized size information corresponding to each candidate region, the first grayscale value corresponding to each candidate region is normalized to obtain the normalized first grayscale value corresponding to each candidate region, and the second grayscale value corresponding to each candidate region is normalized to obtain the normalized second grayscale value corresponding to each candidate region.

[0088] Step S802, summing the normalized size information, the first grayscale value, and the second grayscale value to obtain prediction parameters corresponding to the candidate region.

[0089] The faster the blood flow velocity in the vascular branch, the stronger the blood flow energy, and the larger the grayscale value in the corresponding image; the thinner the blood vessel of the vascular branch, the more likely the grayscale values ​​on both sides will be lower; the larger the size of the vascular branch, the larger the size of the corresponding candidate area. Therefore, the prediction parameters of the candidate area obtained by the above method can characterize the blood flow strength of the vascular branch contained in the candidate area.

[0090] Step S702, selecting a target candidate region from at least one candidate region according to prediction parameters corresponding to each candidate region;

[0091] In some embodiments, after the prediction parameters corresponding to each candidate region are calculated, the candidate region with the largest prediction parameter value is used as the target candidate region. Optionally, after determining the target candidate region, the prediction parameter value corresponding to the target candidate region can be compared with a preset threshold value. If the prediction parameter value is not less than the preset threshold value, the subsequent steps are executed; if the prediction parameter value is less than the preset threshold value, it is determined that the target candidate region does not meet the requirements, and the subsequent steps are no longer executed. The original image and at least one corresponding reference image are re-acquired, and a reminder is given to the user.

[0092] Step S703: determining a target detection region based on the blood flow direction and blood flow width of the blood vessel branches included in the target candidate region.

[0093] Optionally, the direction of the blood flow of the above-mentioned blood vessel branch can be the direction of the blood flow centerline of the blood vessel branch. In the embodiment of the present application, various known image segmentation methods can be used to achieve the acquisition of the blood flow centerline.

[0094] Fig. 9 is a schematic diagram of a possible original image. Fig.10 For Fig. 9 Schematic diagram of the corresponding target detection area, where the white diamond box is the target detection area. In some embodiments, the position of the center point of the target candidate area is determined, and the center point of the target candidate area is used as the center point of the target detection area, such as Fig.11 As shown in the figure, point A is the center point of the target candidate area; the blood flow width of the blood vessel branch included in the target candidate area is determined, and the blood flow width is used as the width of the target detection area, and the length of the target detection area is determined based on the preset ratio of the length to the width of the target detection area; the blood flow direction of the blood vessel branch included in the target candidate area is determined, and it is determined as the inclination angle of the target detection area, such as Fig.11As shown in the figure, the white straight line L is the blood flow direction of the blood vessel branch. The target detection area can be obtained based on the center point, length, width and tilt angle of the target detection area.

[0095] The following describes a process of determining the blood flow direction of a blood vessel branch in a possible candidate region:

[0096] The direction of the blood vessels along the blood vessel branch in the candidate region is taken as the x direction, and the direction perpendicular to the blood vessel direction in the blood vessel branch is taken as the y direction. i , get the coordinate x i The corresponding y direction of the upper edge pixel coordinate point y of the candidate area it , and the coordinates y of the pixel point at the lower edge of the candidate area il , calculate the average value of the upper edge pixel coordinates and the edge pixel coordinates below, as the coordinate x i The corresponding blood flow center position coordinates. According to the above process, the blood flow center position coordinates corresponding to each coordinate in the x direction in the candidate area are calculated, and after smoothing the blood flow center position coordinates corresponding to each coordinate in the x direction, fitting is performed (for example, fitting using the least squares method) to obtain the blood flow direction of the blood vessel branch included in the current candidate area.

[0097] The following describes a process for determining the blood flow width of a blood vessel branch in a possible candidate region:

[0098] The direction of the blood vessels along the blood vessel branch in the candidate region is taken as the x direction, and the direction perpendicular to the blood vessel direction in the blood vessel branch is taken as the y direction. i , get the coordinate x i The corresponding y direction of the upper edge pixel coordinate point y of the candidate area it , and the coordinates y of the pixel point at the lower edge of the candidate area il , calculate the difference between the upper edge pixel coordinates and the lower edge pixel coordinates as the blood flow width corresponding to the coordinate xi. According to the above process, the blood flow width corresponding to each coordinate in the x direction of the candidate area is calculated, and after averaging the blood flow width corresponding to each coordinate in the x direction, the average value is used as the blood flow width of the blood vessel branch included in the current candidate area.

[0099] Optionally, the embodiment of the present application can determine the current blood flow state of the target blood vessel branch based on a trained blood flow state detection model, and the blood flow state detection model includes a residual network, a transformer network, and a classification network, such as Fig.12 shown.

[0100] The following is an explanation of the process of determining the current blood flow state of the target blood vessel branch. Fig.13As shown, after inputting each frame of reference image into the above-mentioned trained blood flow state detection model, the following steps are specifically performed:

[0101] Step S1301, using a residual network to extract features from each frame reference image, to obtain a first feature vector corresponding to each frame reference image;

[0102] Optionally, before inputting each frame reference image into the above-mentioned residual network, each frame reference image is compressed, and the corresponding image data is uniformly scaled to a size of H×W×C, where H, W, and C are the height, width, and number of channels of the scaled image, respectively, and their values ​​can be set based on demand. Usually, C is the original number of channels of the reference image, that is, the number of channels of the reference image is usually not compressed.

[0103] In order to accelerate the gradient descent of the model, the pixels in the compressed image obtained after compression in each frame of the reference image are normalized. For the compressed image of any frame of the reference image, the specific processing method is as follows:

[0104]

[0105] in, is the normalized value of the kth pixel in the compressed image of any of the above reference frames, x k is the value of the kth pixel in the compressed image of any of the above reference frames, x min , x max are respectively the maximum and minimum values ​​of pixels in the compressed image of any of the above reference images.

[0106] Then, a convolutional neural network with a residual structure (i.e., a residual network) is used to perform feature extraction on the normalized compressed images corresponding to each frame of the reference image to obtain the first feature vector corresponding to each frame of the reference image. The result of the residual network can be expressed as:

[0107] H(x)=F(x)+x

[0108] Among them, F(x) represents the convolutional neural network, which includes convolutional layers, activation layers, etc., and x represents the input of the residual network.

[0109] The first eigenvector (feature map T) corresponding to each frame reference image is obtained. It is assumed that the feature map T is an n×n×d matrix, where n is the two-dimensional size of the feature map T (i.e., the latitude corresponding to the length / width), and d is the number of channels of the feature map T, d=H / n×W / n×C.

[0110] Optional, such as Fig.14As shown, before the first feature vector corresponding to each frame reference image is input into the transformer network, the embodiment of the present application further performs the following processing for any frame reference image:

[0111] Step S1401, determining a position vector corresponding to a reference image, and fusing a first feature vector corresponding to the reference image with the position vector to obtain a fused vector;

[0112] Wherein, the position vector is used to identify the position coordinates of each pixel point in the reference image;

[0113] Step S1402: Use the fused vector as the updated first feature vector.

[0114] Specifically, for any reference image, obtain its corresponding position vector PE (the position vector is used to identify the position coordinates of each element in its corresponding first eigenvector T in the first eigenvector), and flatten its corresponding first eigenvector T into an N×d eigenvector E, where N=n×n.

[0115] Concatenate the feature vector E with the position vector PE to obtain the feature vector Z, and use the feature vector Z as the first feature vector of the updated input transformer network:

[0116] Z=T+PE

[0117] Among them, PE∈R N , T∈R N×d , Z∈R N×(d+1) .

[0118] Step S1302, inputting the first feature vector corresponding to each frame reference image into the transformer network, and using the transformer network to extract the feature by fusing the spatial feature and the temporal feature to obtain the second feature vector;

[0119] Optional, such as Fig.15 As shown, the above transformer network includes a first transformer unit and a second transformer unit; the first transformer unit is a spatial encoder, that is, it processes the spatial information in each frame of reference image, outputs the spatial features of each frame of reference image, and combines the spatial features of multiple frames of reference images as the input of the second transformer unit; the second transformer unit is a temporal encoder, which is used to process the spatial features of multiple frames of images aggregated together and increase the data features in the time dimension.

[0120] Specifically, Fig.16As shown, the above-mentioned extraction of the second feature vector based on the transformer network specifically includes the following steps:

[0121] Step S1601, using the first transformer unit to extract spatial features from the first feature vectors corresponding to each frame of the reference image, and based on the time sequence corresponding to each frame of the reference image, fusing the extracted spatial features of multiple frames of reference images into a third feature vector;

[0122] Specifically, the first transformer unit uses a multi-head attention mechanism, where the number of heads h is the number of first feature vectors of the input (i.e., the number of reference images). m , based on the eigenvector Z m The corresponding spatial weight matrix W qm , W km , W vm , get Q m , K m , V m Three vectors, the calculation process is as follows:

[0123] Q m =W qm LN(Z m )

[0124] K m =W km LN(Z m )

[0125] V m =W vm LN(Z m )

[0126] Among them, LN represents layer normalization, and the spatial weight matrix W qm , W km , W vm Can be set based on needs.

[0127] Then, based on the attention mechanism, the spatial feature P corresponding to the reference image m is calculated m , the specific calculation process is as follows:

[0128]

[0129] in, K m The transposed matrix, d K is the vector K mThe dimension is used to prevent gradient explosion. Softmax() is a normalized exponential function, which is used to "compress" a C-dimensional vector containing arbitrary real numbers into another C-dimensional real vector so that each element ranges between (0, 1) and the sum of all elements is 1.

[0130] Then, based on the time sequence corresponding to each frame of the reference image, the spatial features of the extracted multiple frames of reference images are spliced ​​to obtain the third feature vector y. The specific calculation process is as follows:

[0131] y=MSA(Q,K,V)=concat(P1,P2,...,P h )W out

[0132] Among them, W out It is a preset weight matrix used to reduce the dimension of the data. The concat() function is used to connect two or more arrays.

[0133] Step S1602: input the third feature vector into the second transformer unit, and use the second transformer unit to extract the third feature vector by integrating the temporal features to obtain a second feature vector, wherein the second feature vector integrates the spatial features and temporal features of multiple frames of reference images.

[0134] Step S1303: input the second feature vector into the classification network, and use the classification network to determine the current blood flow state of the target blood vessel branch.

[0135] Optionally, the above classification network can be an MLP classification network or a regression network.

[0136] The following describes the training process of the above blood flow state detection model:

[0137] Collect multiple continuous reference images corresponding to the target detection area of ​​the original image as reference sample images, and comprehensively consider different factors such as scanning position, crowd, angle, gain, etc. during collection to obtain image data under more working conditions. Set multiple blood flow state categories based on the collected multiple reference sample images, and annotate the blood flow state of the blood flow branches included in each reference sample image to obtain the reference label corresponding to each reference sample image.

[0138] The collected reference sample images are divided into a training set and a test set, for example, in a ratio of 7:3. The reference sample images in the training set are subjected to image enhancement processing such as cropping, affine transformation, random noise, etc., to further enrich the data set and increase the complexity of the data set.

[0139] The reference sample images in the training set are input into the blood flow state detection model, and the reference labels corresponding to the reference sample images in the training set are used as the target output. The blood flow state detection model is trained using the stochastic gradient descent method.

[0140] During the training process, the following multi-classification cross entropy loss function loss is used:

[0141]

[0142] Among them, y j represents the reference label of the jth reference sample image, represents the output result of the blood flow state detection model corresponding to the jth reference sample image, and R represents the number of reference sample images in the training set.

[0143] After the training is completed, the blood flow state detection model is tested using the reference sample images in the test set and their corresponding reference labels. If the test passes, the trained blood flow state detection model is obtained. It should be noted that if the test fails, a new reference sample image can be obtained to retrain the blood flow state detection model.

[0144] The embodiment of the present application provides a Doppler imaging method, which realizes automatic positioning of the target detection area and automatic adjustment of the digital gain through deep learning, thereby providing better imaging effects.

[0145] The positioning of the target detection area supports automatic or manual calling. When entering the color Doppler mode, you can choose to manually determine the target detection area, or select the automatic positioning function to assist in determining the target detection area.

[0146] For the adjustment of digital gain, the embodiment of the present application uses a four-channel image. Since the four image data information has a strong correlation, channel fusion can more effectively describe the blood flow state characteristics, and make full use of the spatial and temporal characteristics of continuous multi-frame reference images, so as to more accurately obtain the current blood flow state, and then obtain the adjustment plan of the digital gain parameters according to the current blood flow state.

[0147] like Fig.17 As shown, the embodiment of the present application provides a Doppler imaging device 1700, which can be applied to the color Doppler ultrasound diagnostic instrument in the above embodiment, specifically comprising:

[0148] The segmentation module 1701 is used to perform image segmentation on the original image to determine at least one candidate region in the original image, wherein each candidate region includes at least one blood vessel branch of a target detection site;

[0149] A determination module 1702, configured to determine a target detection area from at least one candidate area based on pixel information of each candidate area corresponding to the original image;

[0150] An acquisition module 1703 is used to acquire at least one frame of reference image corresponding to the original image, wherein the reference image includes a target blood vessel branch corresponding to the target detection area in the original image;

[0151] The feature extraction module 1704 is used to extract image features from each frame of reference image, and determine the current blood flow state of the target blood vessel branch based on the image features extracted from each frame of reference image;

[0152] The adjustment module 1705 is used to determine a digital gain parameter based on the current blood flow state of the target blood vessel branch, and adjust each frame of the reference image based on the digital gain parameter.

[0153] Optionally, the above-mentioned determination module 1702 is specifically used to: for any candidate area in the original image, determine the prediction parameters corresponding to the candidate area according to the pixel information of the candidate area; the prediction parameters are used to characterize the blood flow strength of the vascular branches contained in the candidate area; according to the prediction parameters corresponding to each candidate area, select a target candidate area from at least one candidate area; based on the blood flow direction and blood flow width of the vascular branches included in the target candidate area, determine the target detection area.

[0154] Optionally, the pixel information includes size information of the candidate area, a first grayscale value of pixels in the candidate area, and a second grayscale value of pixels in an associated area corresponding to the candidate area; the above-mentioned determination module 1702 is specifically used to: normalize the size information, the first grayscale value and the second grayscale value in the pixel information of the candidate area, respectively; and sum the normalized size information, the first grayscale value and the second grayscale value to obtain the prediction parameters corresponding to the candidate area.

[0155] Optionally, the feature extraction module 1704 is specifically used to: input each frame reference image into a blood flow state detection model; use a residual network to extract features from each frame reference image to obtain a first feature vector corresponding to each frame reference image; input the first feature vector corresponding to each frame reference image into a transformer network, use the transformer network to extract features that fuse spatial features and temporal features, and obtain a second feature vector; input the second feature vector into a classification network, and use the classification network to determine the current blood flow state of the target blood vessel branch.

[0156] Optionally, the transformer network includes a first transformer unit and a second transformer unit, and the feature extraction module 1704 is specifically used to: use the first transformer unit to perform spatial feature extraction on the first feature vector corresponding to each frame reference image, and based on the time sequence corresponding to each frame reference image, fuse the extracted spatial features of multiple frame reference images into a third feature vector; input the third feature vector into the second transformer unit, and use the second transformer unit to perform feature extraction on the third feature vector by fusing the time feature to obtain a second feature vector.

[0157] Optionally, the feature extraction module 1704 is also used to: determine the position vector corresponding to any frame of the reference image, and fuse the first feature vector corresponding to the reference image with the position vector to obtain a fused vector; the position vector is used to identify the position coordinates of each pixel point in the reference image; and use the fused vector as the updated first feature vector.

[0158] Optionally, the adjustment module 1705 is specifically used to: match the current blood flow state of the target blood vessel branch with a preset blood flow state type to determine the target blood flow state type corresponding to the current blood flow state of the target blood vessel branch; and determine the digital gain parameter corresponding to the target blood flow state type based on the corresponding relationship between the blood flow state type and the digital gain parameter. Optionally, the reference image is a four-channel image consisting of a B-ultrasound image, a blood flow velocity image, a blood flow energy image, and a blood flow signal variance image.

[0159] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0160] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0161] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0162] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0163] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A color Doppler ultrasonic diagnostic instrument, characterized in that: The color Doppler ultrasonic diagnostic instrument comprises a probe, a processor and a display screen; The probe is used to collect an original image containing a target detection part of the object and at least one frame of reference image corresponding to the original image; The processor is configured to perform image segmentation on the original image to determine at least one candidate region in the original image, wherein each candidate region includes at least one blood vessel branch of the target detection site; and determine a target detection region from the at least one candidate region based on pixel information of each candidate region corresponding to the original image; Acquire the at least one frame of reference image acquired by the probe, wherein the reference image includes the target blood vessel branch corresponding to the target detection area in the original image; extract image features from each frame of reference image, and determine the current blood flow state of the target blood vessel branch based on the image features extracted from each frame of reference image; determine a digital gain parameter based on the current blood flow state of the target blood vessel branch, and adjust each frame of reference image based on the digital gain parameter; The display screen is used to display the adjusted reference images of each frame in the display interface.

2. The color Doppler ultrasonic diagnostic instrument according to claim 1, characterized in that: The processor is specifically used for: For any candidate region in the original image, a prediction parameter corresponding to the candidate region is determined according to pixel information of the candidate region; the prediction parameter is used to characterize the blood flow strength of the blood vessel branch contained in the candidate region; Selecting a target candidate region from the at least one candidate region according to prediction parameters corresponding to each candidate region; The target detection area is determined based on the blood flow direction and blood flow width of the blood vessel branches included in the target candidate area.

3. The color Doppler ultrasonic diagnostic apparatus according to claim 2, characterized in that: The pixel information includes size information of the candidate area, a first grayscale value of a pixel in the candidate area, and a second grayscale value of a pixel in an associated area corresponding to the candidate area; The processor is specifically used for: The size information, the first grayscale value, and the second grayscale value in the pixel information of the candidate area are normalized respectively; and the normalized size information, the first grayscale value, and the second grayscale value are summed to obtain the prediction parameter corresponding to the candidate area.

4. The color Doppler ultrasonic diagnostic apparatus according to claim 1, characterized in that: The processor is specifically used for: Inputting each frame of reference image into a blood flow state detection model, wherein the blood flow state detection model includes a residual network, a transformer network, and a classification network; Using the residual network to extract features from each frame reference image, respectively, to obtain a first feature vector corresponding to each frame reference image; Inputting the first feature vector corresponding to each frame reference image into the transformer network, and using the transformer network to extract features integrating spatial features and temporal features to obtain a second feature vector; The second feature vector is input into the classification network, and the classification network is used to determine the current blood flow state of the target blood vessel branch.

5. The color Doppler ultrasonic diagnostic apparatus according to claim 4, characterized in that: The transformer network includes a first transformer unit and a second transformer unit, and the processor is specifically used for: Using the first transformer unit to extract spatial features from the first feature vectors corresponding to each frame of the reference image, and based on the time sequence corresponding to each frame of the reference image, fusing the extracted spatial features of multiple frames of reference images into a third feature vector; The third feature vector is input into the second transformer unit, and the second transformer unit is used to perform feature extraction on the third feature vector by integrating the time feature to obtain a second feature vector.

6. The color Doppler ultrasonic diagnostic apparatus according to claim 4 or 5, characterized in that: Before inputting the first feature vector corresponding to each frame reference image into the transformer network, the processor is further used to: For any frame of reference image, determine the position vector corresponding to the reference image, and fuse the first feature vector corresponding to the reference image with the position vector to obtain a fusion vector; the position vector is used to identify the position coordinates of each pixel point in the reference image; The fused vector is used as the updated first feature vector.

7. The color Doppler ultrasonic diagnostic apparatus according to any one of claims 1 to 5, characterized in that: The processor is specifically used for: Matching the current blood flow state of the target blood vessel branch with a preset blood flow state type to determine a target blood flow state type corresponding to the current blood flow state of the target blood vessel branch; Based on the correspondence between the blood flow state type and the digital gain parameter, the digital gain parameter corresponding to the target blood flow state type is determined.

8. The color Doppler ultrasonic diagnostic apparatus according to any one of claims 1 to 5, characterized in that: The reference image is a four-channel image consisting of a B-ultrasound image, a blood flow velocity image, a blood flow energy image, and a blood flow signal variance image.

9. A Doppler imaging method, characterized in that: The method comprises: Acquire an original image containing a target detection part of the object, perform image segmentation on the original image, and determine at least one candidate region in the original image, wherein each candidate region includes at least one blood vessel branch of the target detection part; Determine a target detection area from the at least one candidate area based on pixel information of each candidate area corresponding to the original image; Acquire at least one frame of reference image corresponding to the original image, wherein the reference image includes a target blood vessel branch corresponding to the target detection area in the original image; Extracting image features from each frame of reference image respectively, and determining the current blood flow state of the target blood vessel branch based on the image features extracted from each frame of reference image; A digital gain parameter is determined based on the current blood flow state of the target blood vessel branch, and each frame of the reference image is adjusted based on the digital gain parameter.

10. The method according to claim 9, characterized in that The extracting image features from each frame of reference image respectively, and determining the current blood flow state of the target blood vessel branch based on the image features extracted from each frame of reference image, includes: Inputting each frame of reference image into a blood flow state detection model, wherein the blood flow state detection model includes a residual network, a transformer network, and a classification network; Using the residual network to extract features from each frame reference image, respectively, to obtain a first feature vector corresponding to each frame reference image; Inputting the first feature vector corresponding to each frame reference image into the transformer network, and using the transformer network to extract features integrating spatial features and temporal features to obtain a second feature vector; The second feature vector is input into the classification network, and the classification network is used to determine the current blood flow state of the target blood vessel branch.

Images