Method, device and storage medium for monitoring target tissue in ultrasonic image

Abstract

The present application relates to a method, device and storage medium for monitoring target tissue in an ultrasonic image, and belongs to the field of medical imaging technology. The method comprises: determining a target ultrasonic image to be monitored from continuous m-frame ultrasonic images, wherein the m-frame ultrasonic images are obtained by ultrasonic scanning of the target tissue; obtaining a tissue region of the target tissue in the target ultrasonic image; performing curve evolution using image data of the tissue region so that the evolved curve coincides with the tissue edge in the tissue region; displaying the tissue edge of the target tissue in the target ultrasonic image based on the evolved curve; and solving the problem of low efficiency in manually locating the target tissue from the ultrasonic image. The tissue edge of the target tissue is automatically determined by means of curve evolution and displayed in the target ultrasonic image, without the need for manual identification of the target tissue, thereby improving the efficiency of locating the target tissue.

Description

[Technical field]

[0001] The present application relates to a method, device and storage medium for monitoring target tissue in an ultrasonic image, and belongs to the field of medical imaging technology. [Background technology]

[0002] Ultrasound images refer to images obtained by scanning biological tissues using ultrasound equipment. Ultrasound images can reflect the differences in acoustic parameters of different tissues in the medium. After obtaining the ultrasound image, doctors can locate the target tissue based on the image content presented by the ultrasound image and determine the size of the target tissue based on experience. However, manual identification of target tissues is inefficient. [Summary of the invention]

[0003] The present application provides a method, device and storage medium for monitoring target tissue in an ultrasound image, which can solve the problem of low efficiency in manually locating target tissue from an ultrasound image. The present application provides the following technical solutions:

[0004] In a first aspect, a method for monitoring a target tissue in an ultrasound image is provided, the method comprising:

[0005] Determining a target ultrasound image to be monitored from m consecutive ultrasound image frames, wherein the m ultrasound image frames are obtained by performing ultrasound scanning on the target tissue, and m is an integer greater than 1;

[0006] Acquire a tissue region of a target tissue in the target ultrasound image;

[0007] Using the image data of the tissue region to perform curve evolution so that the evolved curve coincides with the tissue edge in the tissue region;

[0008] Based on the evolved curve, a tissue edge of the target tissue is displayed in the target ultrasound image.

[0009] Optionally, determining a target ultrasound image to be monitored from the continuous m frames of ultrasound images includes:

[0010] sequentially determining the amount of target tissue in the m frames of ultrasound images;

[0011] A frame of ultrasound image with the largest amount of tissue is determined as the target ultrasound image.

[0012] Optionally, sequentially determining the amount of target tissue in the m frames of ultrasound images includes:

[0013] For the i-th frame of ultrasound image in the m frames of ultrasound images, when the value of i is divisible by n, input the i-th frame of ultrasound image into a pre-trained target detection model to obtain a detection result of a target tissue in the i-th frame of ultrasound image; the detection result includes a tissue area and a tissue quantity of the target tissue;

[0014] In the case where the value of i cannot be divided by n, the detection result of the target tissue in the i-1th frame of ultrasound image is input into a pre-created tracking model to obtain the detection result of the target tissue in the i-th frame of ultrasound image; the tracking model is used to predict the position of the target tissue in the i-th frame of ultrasound image based on the position of the target tissue in the i-1th frame of ultrasound image, and determine a search range in the i-th frame of ultrasound image based on the predicted position, and search for the target tissue within the search range;

[0015] Wherein, i is an integer ranging from 0 to m-1 in sequence; and n is a positive integer with a fixed value.

[0016] Optionally, when the frame number of the target ultrasound image is not divisible by n, acquiring the tissue region of the target tissue in the target ultrasound image includes:

[0017] Inputting the target ultrasound image into the target detection model to obtain a detection result of the target ultrasound image;

[0018] A union of the detection result output by the target detection model and the detection result output by the tracking model is determined to obtain a tissue region of the target tissue in the target ultrasound image.

[0019] Optionally, the target detection model includes:

[0020] k layers of feature extraction layers connected in sequence, each feature extraction layer is used to extract image features of input data to obtain a feature map; k is an integer greater than 1;

[0021] Each of the first k-1 feature extraction layers is connected to an attention layer, and the attention layer is used to provide attention information in the feature map output by the feature extraction layer;

[0022] The k-1th feature fusion layer is connected to the k-th feature extraction layer and the k-1th attention layer respectively; the k-1th feature fusion layer is used to perform feature fusion on the feature map output by the k-th feature extraction layer and the attention information output by the k-1th attention layer to obtain the k-1th fusion feature;

[0023] The j-th feature fusion layer is connected to the j-th attention layer and the j+1-th feature fusion layer respectively; the j-th feature fusion layer is used to fuse the attention information output by the j-th attention layer with the j+1-th fusion feature output by the j+1-th feature fusion layer to obtain the j-th fusion feature; j is an integer from k-2 to 1 in sequence;

[0024] An output layer connected to the first-layer feature fusion layer, wherein the output layer is used to calculate the detection result of the target ultrasound image based on the first-layer fusion features output by the first-layer feature fusion layer.

[0025] Optionally, the step of performing curve evolution using the image data of the tissue region so that the evolved curve coincides with a tissue edge in the tissue region comprises:

[0026] Preprocessing the tissue region to obtain a preprocessed tissue region; the preprocessing includes histogram equalization processing and / or median filtering processing;

[0027] The preprocessed tissue region is input into a pre-created parametric active contour model to obtain a closed curve that coincides with the tissue edge, wherein the closed curve is the evolved curve.

[0028] Optionally, the step of inputting the preprocessed tissue region into a pre-created parametric active contour model to obtain a closed curve that coincides with the tissue edge includes:

[0029] generating an initialization envelope curve based on the center position of the preprocessed tissue region;

[0030] Based on the initialized envelope curve, using the energy function of the parametric active contour model to perform curve evolution, and determining the envelope curve corresponding to the minimum function value of the energy function as the closed curve;

[0031] Wherein, the energy function is used to integrate the capacity of each point on the envelope curve; the energy function includes a first integral term, a second integral term and a third integral term; the first integral term is used to control the smoothness of the envelope curve; the second integral term is used to control the continuity of the envelope curve; the third integral term is used to control the envelope curve to approach the tissue edge.

[0032] Optionally, displaying the tissue edge of the target tissue in the target ultrasound image based on the evolved curve includes:

[0033] For each target tissue in the target ultrasound image, the tissue edge of the target tissue is depicted in the target ultrasound image by an evolved curve corresponding to each target tissue.

[0034] In a second aspect, a device for monitoring target tissue in an ultrasound image is provided, the device comprising a processor and a memory; a program is stored in the memory, and the program is loaded and executed by the processor to implement the method for monitoring target tissue in an ultrasound image provided in the first aspect.

[0035] According to a third aspect, a computer-readable storage medium is provided, wherein a program is stored in the storage medium, and when the program is executed by a processor, the program is used to implement the method for monitoring target tissue in an ultrasound image provided in the first aspect.

[0036] The beneficial effects of the present application include at least: determining a target ultrasound image to be monitored from continuous m-frame ultrasound images, where the m-frame ultrasound images are obtained by ultrasonic scanning of the target tissue; obtaining the tissue region of the target tissue in the target ultrasound image; using the image data of the tissue region to perform curve evolution so that the evolved curve coincides with the tissue edge in the tissue region; displaying the tissue edge of the target tissue in the target ultrasound image based on the evolved curve; solving the problem of low efficiency in manually locating the target tissue from the ultrasound image; automatically determining the tissue edge of the target tissue by means of curve evolution and displaying it in the target ultrasound image, without the need for manual identification of the target tissue, thereby improving the efficiency of locating the target tissue.

[0037] In addition, when the traditional pixel-based neural network segments the target tissue, there will be problems such as incomplete segmentation results, overflow or difficulty in image segmentation. The curve evolution method obtains a smoother curve after evolution and is suitable for images with blurred edges. Therefore, in this embodiment, the tissue edge of the target tissue is determined by using the curve evolution method, and the determined tissue edge is more accurate than the tissue edge calculated by the pixel-based neural network.

[0038] In addition, by selecting a frame of target ultrasound image with the largest number from the m frames of ultrasound images for analysis, the target tissue can be analyzed more comprehensively and the accuracy of the analysis result can be improved.

[0039] In addition, when determining the target ultrasound image, if the frame number of the ultrasound image is divisible by n, the target detection model is used to detect the target of the ultrasound image; and the tracking model is used to track the target of the n-1 frame image after the frame ultrasound image; because the tracking model is based on the detection result of the previous frame ultrasound image, the speed of calculating the detection result using the tracking model is faster than that of calculating the detection result using the detection model. At the same time, due to the omission, false detection and other unstable detection results that often occur in the detection model, using the tracking model to calculate the results can avoid the problems of the detection model and improve the accuracy of the detection results.

[0040] In addition, if the target detection model is used to detect the target of the 0th frame ultrasound image, and the subsequent ultrasound images only use the tracking model to track the target tissue, if there is a new target tissue, the new target tissue will be missed by the tracking model. Based on this, in this embodiment, the tracking model is used to track only part of the ultrasound images. When i is divisible by n, the target detection model is used again to identify the follicle target that has been tracked before the i-th frame, and the new target tissue can also be identified, thereby ensuring the accuracy of identifying the target tissue.

[0041] In addition, by setting the target detection model on the basis of the convolutional neural network, adding an attention layer and a feature fusion layer, the image features that the user expects to pay attention to can be extracted from the feature map, and the accuracy of the target detection model calculation detection results can be improved. At the same time, by fusing the underlying attention information and the mid-to-high-level attention information into a feature fusion layer, it can not only strengthen the detail information that cannot be observed by the mid-to-high-level attention layer, but also overcome the problem that the underlying attention layer cannot observe the target global information. By extracting features layer by layer from low to high through the feature extraction layer, and then fusing features layer by layer from high to low through the feature fusion layer, the local features and global features of the ultrasound image are modeled, which can better identify target tissues of different sizes. In addition, due to the high target overlap of the target tissue in the ultrasound image, it is easy to cause the problem of incomplete target recognition. By fusing the shallow features with the deep features, more comprehensive features can be obtained, which is conducive to identifying all target tissues.

[0042] In addition, by determining the tissue region of the target tissue based on the union of the detection results output by the target detection model and the detection results output by the tracking model, the target tissue in the target ultrasound image can be detected more comprehensively.

[0043] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present application in conjunction with the accompanying drawings.

Brief Description of the Drawings

[0044] Figure 1 is a flow chart of a method for monitoring target tissue in an ultrasound image provided by an embodiment of the present application;

[0045] Figure 2 is a flow chart of determining a target ultrasound image provided by an embodiment of the present application;

[0046] Figure 3 is a schematic diagram of the structure of a target detection model provided by an embodiment of the present application;

[0047] Figure 4is a schematic diagram of a tissue region of a target tissue in a target ultrasound image provided by an embodiment of the present application;

[0048] Figure 5 is a schematic diagram of curve evolution of various tissue regions provided by an embodiment of the present application;

[0049] Figure 6 is a schematic diagram of displaying tissue edges in a target ultrasound image provided by an embodiment of the present application;

[0050] Figure 7 is a block diagram of a device for monitoring target tissue in an ultrasound image provided by an embodiment of the present application;

[0051] Figure 8 It is a block diagram of an ultrasound device provided by an embodiment of the present application. [Specific implementation method]

[0052] The specific implementation methods of the present application are further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application but are not intended to limit the scope of the present application.

[0053] In medical image processing, image segmentation is required in many scenarios. For example, if you need to extract follicle tissue and / or ovarian tissue from an ultrasound image, you need to segment the image in the process of extracting the target tissue. Another example is that when identifying carotid artery plaques, you need to segment the plaques in the carotid artery image.

[0054] Traditional image segmentation methods include edge-based Canny edge detection methods, region-based Markov segmentation methods, etc. However, these methods can only obtain local information of the image and cannot be combined with the overall information of the image for calculation, resulting in inaccurate image segmentation results. In addition, the image segmentation results obtained by traditional segmentation methods have discontinuous edges, while the edges of target tissues in medical images (such as ultrasound images in this application) are continuous, which leads to the problem that the image segmentation results cannot meet the requirements of regional integrity.

[0055] Deformable models can integrate the interpretation of image data and prior knowledge to obtain segmentation results that are more in line with human visual mechanisms. In general, a deformable model includes a priori assumptions about the shape and features of the target to be segmented (i.e., the target tissue in this application), i.e., a priori model; and uses an energy function to measure the degree of fit between the priori model and the image data. The purpose of curve or surface deformation is to improve the degree of fit between the priori model and the image data. The way to improve the degree of fit is to minimize the energy function.

[0056] Deformation models include active contour models (ACMs), which are used to define curves or surfaces in the image domain and move under the action of internal forces related to the curves or surfaces themselves and external forces defined by the image data. The role of internal forces is to maintain the smoothness of the model, and the role of external forces is to move the model closer to the edge of the target to be segmented. By maintaining the smoothness of the model and integrating other prior information such as the shape of the target, the active contour model is not only robust to image noise and edge gaps, but also can integrate image edge elements into a relevant and consistent mathematical expression for subsequent higher-level image information processing.

[0057] The segmentation methods based on active contour models include parametric active contour models and geometric active contour models. In the parametric active contour model, the contour is represented by parameters, such as the Snake model; in the geometric active contour model, the contour is represented by geometry, such as the level set method.

[0058] Among them, the Snake model is a deformable parameter curve and the corresponding energy function. The model aims to minimize the energy objective function and controls the parameter curve to evolve. The closed curve with the minimum energy is the target contour.

[0059] Curve evolution theory: refers to the theory of using the unit normal vector and curvature of a curve to study the deformation of a curve over time. When the geometric curve evolution model based on curve evolution theory is applied to image segmentation, the basic idea is to couple the image data and the evolution speed of the curve so that the curve evolution stops at the edge of the target tissue.

[0060] Object Detection: It is used to identify the object of interest in the image, determine the type and location of the object. For example, it is used to identify the target tissue from the ultrasound image, and determine the type and location of the target tissue.

[0061] Traditional target detection algorithms include two categories:

[0062] The first category: Two-stage algorithm, the main principles include: first determine the pre-selected box including the target in the image; then classify the samples through Convolutional Neural Networks (CNN).

[0063] The Two-stage algorithm includes, but is not limited to, Region-CNN (R-CNN), Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition (SPP-Net), etc. This embodiment does not limit the type of the Two-stage algorithm.

[0064] The second category: One-stage algorithm, the main principles include: there is no need to determine the pre-selected box including the target in the image, and the neural network model is directly used to extract image features to predict the target classification and position.

[0065] One-stage algorithms include, but are not limited to: Single Shot MultiBox Detector (SSD), You Only Look Once (YOLO) series algorithms, etc. This embodiment does not limit the type of One-stage algorithm.

[0066] Visual Object Tracking: refers to the technology of locating a target in continuous image frames. Since the position, direction and speed of the target in the previous frame are known, in the next frame, this information can be used to predict the position of the target in the next frame, and the predicted position of the target can be used to search in a small range to accurately locate the target. Therefore, the calculation speed of target tracking is faster than that of target detection.

[0067] Target tracking algorithms include generation methods, discriminative methods, and deep learning (Deep ConvNet based) methods. Among them, discriminative methods include correlation filter (CF) methods, such as discriminative correlation filter (DCF), kernel correlation filter (KCF) algorithm, etc.

[0068] Taking the KCF algorithm as an example, the principles of the KCF algorithm include: using regression methods to distinguish between targets and backgrounds, using the target area as a positive sample, and constructing multiple negative samples through a circulant matrix; using each sample to train the tracker using the ridge regression method, and when solving nonlinear regression, mapping low-dimensional data to high-dimensional space based on the kernel method to convert nonlinear calculations into linear calculations. After the tracker training is completed, the trained tracker is used to filter the image to obtain the output distribution map, and the maximum response position is used as the center position of the target. After that, the center position of the target is used to determine a new positive sample, and a new negative sample is generated based on the positive sample to update the tracker, so that the same target in consecutive image frames can be tracked iteratively.

[0069] Attention mechanism: It is a data processing method in machine learning and is widely used in various types of machine learning tasks such as natural language processing, image processing, and speech recognition. The attention mechanism enables the neural network to focus on a subset of inputs (or features), that is, to select specific inputs.

[0070] Optionally, the present application uses the monitoring method of target tissue in an ultrasound image provided in each embodiment as an example for use in a monitoring device for target tissue in an ultrasound image. The monitoring device may be an ultrasound imaging device, or other device connected to an ultrasound scanning device. This embodiment does not limit the type of the monitoring device.

[0071] Figure 1 : is a flow chart of a method for monitoring target tissue in an ultrasound image provided by an embodiment of the present application, the method comprising at least the following steps:

[0072] Step 101: determine a target ultrasound image to be monitored from m consecutive ultrasound image frames.

[0073] The m frames of ultrasound images are obtained by performing ultrasound scanning on the target tissue, and m is an integer greater than 1.

[0074] Optionally, the m frames of ultrasound images may be m consecutive frames of images in the same video stream; or, may be m consecutive images obtained by performing multiple image acquisitions on the target tissue.

[0075] In this embodiment, the target tissue includes but is not limited to: follicles, ovaries, hearts, lungs, brain tissues and other biological tissues. The tissue edge of the target tissue is a closed curve. This embodiment does not limit the tissue type of the target tissue.

[0076] Since the image quality of the target ultrasound image will affect the monitoring result of the target tissue, based on this, in this embodiment, the target ultrasound image is the ultrasound image with the most target tissues in the m-frame ultrasound image. Accordingly, determining the target ultrasound image to be monitored from the continuous m-frame ultrasound image includes: sequentially determining the number of target tissues in the m-frame ultrasound image; and determining the frame of ultrasound image with the largest number of tissues as the target ultrasound image.

[0077] In one example, sequentially determining the number of target tissues in m frames of ultrasound images includes: for the i-th frame of ultrasound images in the m frames of ultrasound images, when the value of i is divisible by n, inputting the i-th frame of ultrasound images into a pre-trained target detection model to obtain a detection result of the target tissue in the i-th frame of ultrasound images; when the value of i is not divisible by n, inputting the detection result of the target tissue in the i-1-th frame of ultrasound images into a pre-created tracking model to obtain a detection result of the target tissue in the i-th frame of ultrasound images. The detection result includes the tissue area and the number of tissues of the target tissue.

[0078] Wherein, i is an integer from 0 to m-1 in sequence; n is a fixed positive integer. The value of n is set based on the empirical value of the number of frames in which the number of tissues of the target tissue remains unchanged continuously, and the value of n can be 6 or 5. This embodiment does not limit the value of n.

[0079] The tracking model is used to predict the position of the target tissue in the i-th frame ultrasound image based on the position of the target tissue in the i-1-th frame ultrasound image, and determine the search range in the i-th frame ultrasound image based on the predicted position, and search for the target tissue within the search range.

[0080] In this embodiment, each frame of the ultrasound image is pre-set with a frame number, and the frame numbers of the ultrasound images are numbered sequentially starting from 0 in the order of acquisition time from the earliest to the latest.

[0081] In order to more clearly understand the method for determining the target ultrasound image provided by this embodiment, an example is given below to illustrate the method. Figure 2 , the process of determining the target ultrasound image in the m frames of ultrasound images at least includes steps 21-23:

[0082] Step 21, initializing i=0;

[0083] Step 22, determine whether i is equal to m; if so, execute step 26; if not, execute step 23;

[0084] Step 23, determine whether i can be divided by n integers; if so, execute step 24; if not, execute step 25;

[0085] Step 24, input the i-th frame of ultrasound image into the target detection model, obtain the detection result of the target tissue in the i-th frame of ultrasound image, set i=i+1 and execute step 22 again;

[0086] Step 25, input the detection result of the target tissue in the i-1th frame of ultrasound image into the tracking model to obtain the detection result of the target tissue in the i-th frame of ultrasound image; let i=i+1, and execute step 22 again;

[0087] Step 26, determining the frame of ultrasound image with the largest amount of tissue as the target ultrasound image, and the process ends.

[0088] according to Figure 2 As can be seen from the flowchart shown, for each frame of ultrasound image whose frame number is divisible by n, the detection result of this frame of ultrasound image is calculated by the detection model; the detection result of the n-1 frame of image after this frame of ultrasound image is calculated by the tracking model. Since the tracking model is based on the detection result of the previous frame of ultrasound image, the speed of calculating the detection result using the tracking model is faster than that of calculating the detection result using the detection model. At the same time, due to the omission, false detection and other unstable detection results that often occur in the detection model, using the tracking model to calculate the results can avoid the problems of the detection model and improve the accuracy of the detection results.

[0089] In addition, if the target detection model is used to detect the target of the 0th frame ultrasound image, and the subsequent ultrasound images only use the tracking model to track the target tissue, if there is a new target tissue, the new target tissue will be missed by the tracking model. Based on this, in this embodiment, the tracking model is used to track only part of the ultrasound images. When i is divisible by n, the target detection model is used again to identify the follicle target that has been tracked before the i-th frame, and the new target tissue can also be identified, thereby ensuring the accuracy of identifying the target tissue.

[0090] In this embodiment, the target detection model can classify each image content in the ultrasound image, thereby obtaining each target tissue in the ultrasound image; at the same time, the target detection model can calculate the tissue area of ​​each target tissue.

[0091] Optionally, the tissue region is represented by an outer rectangular frame of the target tissue (in other words, the image content in the outer rectangular frame includes the image of the target tissue), and the position of the tissue region on the ultrasound image is represented by the position of the upper left vertex of the outer rectangular frame, the width and height of the outer rectangular frame.

[0092] In one example, the object detection model is obtained by adding an attention layer and a feature fusion layer to the convolutional neural network. Figure 3The target detection model shown in the figure includes, in the data transmission direction, k-layer feature extraction layers 31 connected in sequence, an attention layer 32 connected to each of the first k-1 layers of feature extraction layers, a k-1th feature fusion layer 33 connected to the kth feature extraction layer and the k-1th attention layer, a jth feature fusion layer 34 connected to the jth attention layer and the j+1th feature fusion layer, and an output layer 35 connected to the 1st feature fusion layer. k is an integer greater than 1; j is an integer from k-2 to 1 in sequence.

[0093] Figure 3 In the description, k is taken as 4 as an example. In actual implementation, the value of k can be a larger or smaller value. This embodiment does not limit the value of k.

[0094] Each feature extraction layer 31 is used to extract image features of input data to obtain a feature map. Schematically, the feature extraction layer 31 can be a convolution layer. The number of convolution layers in each feature extraction layer 31 can be one or more layers, and this embodiment does not limit the implementation method of the feature extraction layer 31.

[0095] Each feature extraction layer in the first k-1 feature extraction layers is connected to an attention layer 32. That is, there are k-1 attention layers 32 in the target detection model. The attention layer 32 is used to provide attention information in the feature map output by the feature extraction layer. The attention information can be spatial attention information or channel attention information. In this way, the image features that the user wants to pay attention to can be extracted from the feature map, thereby improving the accuracy of the detection results calculated by the target detection model. In this embodiment, the attention layer 32 can not only extract more effective information from the feature map output by the feature extraction layer to improve the target detection effect, but also provide the feature fusion layer with the fusion of attention information at different levels.

[0096] The k-1th feature fusion layer 33 is connected to the k-th feature extraction layer and the k-1th attention layer respectively. The k-1th feature fusion layer 33 is used to fuse the feature map output by the k-th feature extraction layer with the attention information output by the k-1th attention layer to obtain the k-1th fusion feature.

[0097] Since the k-th feature extraction layer is used to extract deep features of the ultrasound image, the data of the deep features is usually less. In this embodiment, the attention layer is not introduced after the k-th feature extraction layer. This can prevent the problem of excessive extraction of deep features, resulting in fewer deep features and affecting the target detection results.

[0098] In addition, by fusing the feature map output by the kth feature extraction layer with the attention information output by the k-1th attention layer, it is possible to ensure that the deep feature data is fused with the attention information of the middle and high levels, thereby combining data at different levels for target detection and improving the accuracy of target detection.

[0099] The j-th feature fusion layer 31 is connected to the j-th attention layer and the j+1-th feature fusion layer respectively. The j-th feature fusion layer is used to fuse the attention information output by the j-th attention layer with the j+1-th fusion feature output by the j+1-th feature fusion layer to obtain the j-th fusion feature.

[0100] In this embodiment, by performing feature fusion layer on the bottom layer attention information and the middle and high layer attention information, it is possible to enhance the detail information that cannot be observed by the middle and high layer attention layers, and also to overcome the problem that the bottom layer attention layer cannot observe the target global information. By performing feature extraction layer by layer from low to high, and then performing feature fusion layer by layer from high to low by the feature fusion layer, the local features and global features of the ultrasound image are modeled, and target tissues of different sizes can be better identified.

[0101] In addition, due to the high degree of target overlap of target tissues in ultrasound images, it is easy to cause incomplete target recognition. By fusing shallow features with deep features, more comprehensive features can be obtained, which is conducive to identifying all target tissues.

[0102] The output layer 35 is used to calculate the detection result of the target ultrasound image based on the first-layer fusion features output by the first-layer feature fusion layer.

[0103] In this embodiment, the target detection model is obtained by training the neural network model using the training data. The model structure of the neural network model is the same as the model structure of the target detection model, for example: Figure 3 The model structure shown in Figure 1. Training the neural network model using training data includes the following steps:

[0104] Step 1: Get training data.

[0105] The training data includes a sample image and a label of the target tissue in the sample image. Optionally, the label is obtained by marking the target tissue in the sample image with a rectangular box, and the label includes the marking probability of the target tissue and the position information of the actual rectangular box. If the marking probability is equal to 1, it means that the current target is the target tissue. If the marking probability is less than 1, it means that the current target may not be the target tissue. The position information of the actual rectangular box includes the center point position, width and height of the actual rectangular box.

[0106] Step 2: Input the sample images in the training data into the neural network model to obtain the model results.

[0107] The model results include: the predicted probability of each target tissue and the position information of the predicted rectangular box of each target tissue. The position information of the predicted rectangular box includes the center point position, width and height of the predicted rectangular box.

[0108] Step 3: Input the model result and the label corresponding to the sample image into a preset loss function to obtain the loss function value.

[0109] Optionally, a loss function is used to represent the error between the model results and the labels.

[0110] Schematically, the loss function is expressed as follows:

[0111]

[0112] Where B represents the total number of target tissues in a sample image. q represents the qth target tissue. q represents the labeling probability of the qth target tissue. represents the model result (or predicted probability) of the qth target organization output by the neural network, The value range is [0,1]. Represents the square error between the labeled probability and the predicted probability. q Represents the horizontal coordinate of the center point of the actual rectangular box to which the qth target tissue belongs. Indicates the horizontal coordinate of the center point of the predicted rectangular box to which the qth target tissue belongs. q Represents the ordinate of the center point of the actual rectangular box to which the qth target tissue belongs. Indicates the ordinate of the center point of the predicted rectangular box to which the qth target tissue belongs. q Indicates the width of the actual rectangular box to which the qth target tissue belongs. Indicates the width of the predicted rectangular box to which the qth target tissue belongs. h q Indicates the height of the actual rectangular box to which the qth target tissue belongs. Indicates the height of the predicted rectangular box to which the qth target tissue belongs.

[0113] Step 4: Iteratively train the neural network model based on the loss function value until the neural network model converges or the number of iterations reaches a preset number, thereby obtaining a target detection model.

[0114] Among them, iterative training of the neural network model refers to calculating the partial derivatives of the parameters in the neural network with respect to the loss function, and updating the parameters in the neural network so that the loss function value gradually decreases.

[0115] Optionally, the tracking model can be a KCF algorithm. At this time, the detection result of the target tissue in the i-1th frame of the ultrasound image is input into the pre-created tracking model to obtain the detection result of the target tissue in the i-th frame of the ultrasound image, including: taking the tissue area indicated by the detection result of the i-1th frame of the ultrasound image as a positive sample, and constructing multiple negative samples through a circulant matrix; using each sample to train the tracker using the ridge regression method, and when solving the nonlinear regression, mapping the low-dimensional data to a high-dimensional space based on the kernel method to convert the nonlinear calculation into a linear calculation; after the tracker training is completed, the trained tracker is used to filter the i-th frame of the ultrasound image to obtain the output distribution map, and the maximum response position is used as the center position of the target tissue to obtain the detection result of the target tissue in the i-th frame of the ultrasound image.

[0116] Step 102: Acquire a tissue region of a target tissue in a target ultrasound image.

[0117] In this embodiment, the tissue region is obtained by performing target detection on the target ultrasound image. In other implementations, the tissue region may also be selected by the user, and this embodiment does not limit the method for obtaining the tissue region.

[0118] In one example, obtaining a tissue region of a target tissue in a target ultrasound image includes: inputting the target ultrasound image into a target detection model to obtain the tissue region of the target tissue.

[0119] The relevant description of the target detection model is detailed in step 101, which will not be repeated here in this embodiment.

[0120] In another example, obtaining a tissue region of a target tissue in a target ultrasound image includes: when a frame number of the target ultrasound image cannot be divided by n, inputting the target ultrasound image into a target detection model to obtain a detection result of the target ultrasound image; determining a union of the detection result output by the target detection model and the detection result output by the tracking model to obtain the tissue region of the target tissue in the target ultrasound image.

[0121] Since the frame number of the target ultrasound image cannot be divided by n, it means that the target ultrasound image has only been calculated by the tracking model but not by the target detection model. At this time, inputting the target ultrasound image into the target detection model can ensure the accuracy of target detection.

[0122] In addition, in this embodiment, by determining the tissue region of the target tissue based on the union of the detection results output by the target detection model and the detection results output by the tracking model, the target tissue in the target ultrasound image can be detected more comprehensively.

[0123] Optionally, when the frame number of the target ultrasound image is divisible by n, the detection result already calculated by the target detection model is directly read to obtain the tissue area of ​​the target tissue. In this case, there is no need to input the target ultrasound image into the target detection model again to obtain the detection result of the target ultrasound image, which can save computing resources of the monitoring device.

[0124] Step 103: Use the image data of the tissue region to perform curve evolution so that the evolved curve coincides with the tissue edge in the tissue region.

[0125] In traditional edge recognition methods, pixel-based neural networks such as U-net are usually used to segment the target tissue and obtain the edge of the target tissue. However, the edge of the target tissue in the ultrasound image may have unclear parts, and the pixel-based neural network may cause incomplete segmentation results or overflow problems when performing image segmentation. In addition, since the size of the target tissue may be very small, it will cause difficulties in image segmentation using pixel-based neural networks.

[0126] Based on the above technical problems, in this embodiment, the tissue edge of the target tissue is determined by using a curve evolution method. Since the evolved curve obtained by the curve evolution method is relatively smooth and suitable for images with blurred edges, the tissue edge determined by the curve evolution method is more accurate than the tissue edge calculated by the pixel-based neural network.

[0127] Optionally, the curve evolution method includes, but is not limited to: using a parametric active contour model to evolve the curve, or using a geometric active contour model to evolve the curve, etc. This embodiment does not limit the curve evolution method.

[0128] In one example, compared with other curve evolution methods, the parameter active contour model can determine a smoother closed curve and is more suitable for images with blurred edges. Therefore, this embodiment takes the curve evolution method as the parameter active contour model as an example for explanation. At this time, the image data of the tissue area is used to evolve the curve so that the evolved curve coincides with the tissue edge in the tissue area, including: preprocessing the tissue area to obtain a preprocessed tissue area; the preprocessing includes histogram equalization processing and / or median filtering processing; the preprocessed tissue area is input into a pre-created parameter active contour model to obtain a closed curve that coincides with the tissue edge, and the closed curve is the evolved curve.

[0129] The histogram equalization process is used to increase the difference between adjacent pixels, thereby increasing the contrast of the image of the tissue region.

[0130] The median filter process is used to protect the edge information in the image of the tissue area, smooth the image, and eliminate some noise in the image.

[0131] The preprocessed tissue region is input into a pre-created parametric active contour model to obtain a closed curve that coincides with the tissue edge, including: generating an initialization envelope curve based on the center position of the preprocessed tissue region; based on the initialization envelope curve, using the energy function of the parametric active contour model to evolve the curve, and determining the envelope curve corresponding to the minimum function value of the energy function as a closed curve.

[0132] Among them, the energy function is used to integrate the capacity of each point on the envelope curve; the energy function includes a first integral term, a second integral term and a third integral term; the first integral term is used to control the smoothness of the envelope curve; the second integral term is used to control the continuity of the envelope curve; the third integral term is used to control the envelope curve to approach the tissue edge.

[0133] Schematically, the energy function is represented by the following formula:

[0134]

[0135] Among them, s represents any point on the current envelope curve, and the total energy calculated by the energy function is the integration of the capacity of each point on the previous envelope curve. is the first integral term, which represents the elastic capacity generated when the evolution of point s changes, and is used to control the smoothness of the envelope curve. is the second integral term, which represents the bending energy generated when the evolution of point s changes, and is used to control the continuity of the envelope curve. is the third integral term, which represents the image gradient value at point s when the evolution of point s changes, and is used to control the envelope curve to approach the edge of the tissue. When the envelope curve approaches the edge of the tissue, the image gradient value at point s becomes larger, and the total energy of the energy function becomes smaller. When the total energy reaches the minimum value, it means that the envelope curve coincides with the edge of the tissue, that is, the envelope of the target tissue is obtained. Compared with pixel-based neural networks such as unet, the energy function of the parametric active contour model also has the first integral term and the second integral term, thereby limiting the shape of the envelope curve.

[0136] It should be noted that, in this embodiment, only the tissue area of ​​each target tissue is used for curve evolution. In actual implementation, the image data in a complete target ultrasound image can also be used for curve evolution. This embodiment does not limit the image data used for curve evolution.

[0137] Step 104 : displaying the tissue edge of the target tissue in the target ultrasound image based on the evolved curve.

[0138] Since in step 103, the image content corresponding to the tissue region of each target tissue is subjected to curve evolution, after the evolved curve is obtained, the evolved curve needs to be mapped to the original target ultrasound image.

[0139] Schematically, based on the evolved curve, the tissue edge of the target tissue is displayed in the target ultrasound image, including: for each target tissue in the target ultrasound image, the tissue edge of the target tissue is depicted in the target ultrasound image by the evolved curve corresponding to each target tissue. In actual implementation, the first tissue region of the target tissue can be used to replace the second tissue region of the target tissue.

[0140] Among them, the first tissue region is a tissue region with an evolved curve. In other words, the first tissue region is a tissue region obtained after curve evolution of the original tissue region; the second tissue region is the original region of the target tissue in the target ultrasound image, that is, the original tissue region.

[0141] Optionally, replacing the second tissue region of the target tissue with the first tissue region includes: covering the second tissue region with the first tissue region on the target ultrasound image.

[0142] For example: The detection result obtained after the target ultrasound image is detected by the target detection model is referenced Figure 4 As shown. Figure 4 It can be seen that the target ultrasound image includes 4 tissue regions, each of which is represented by a rectangular box. Figure 5 For each tissue region 51 , the tissue region is preprocessed to obtain a preprocessed tissue region 52 ; then, each preprocessed tissue region 52 is input into a parameter active contour model 53 to obtain a tissue region having an evolved curve, that is, a first tissue region 54 .

[0143] Afterwards, the first tissue region 54 is used to replace the original tissue region (ie, the first tissue region) at the corresponding position in the target ultrasound image to obtain a replaced target ultrasound image. Figure 6 As shown. Figure 6 It can be seen that the replaced target ultrasound image includes an evolved curve corresponding to each target tissue, and the evolved curve coincides with the tissue edge of the target tissue.

[0144] In summary, the monitoring method of target tissue in an ultrasound image provided in the present embodiment determines the target ultrasound image to be monitored from m consecutive ultrasound image frames, where the m ultrasound image frames are obtained by ultrasonic scanning of the target tissue; obtains the tissue region of the target tissue in the target ultrasound image; uses the image data of the tissue region to perform curve evolution so that the evolved curve coincides with the tissue edge in the tissue region; based on the evolved curve, displays the tissue edge of the target tissue in the target ultrasound image; can solve the problem of low efficiency in manually locating the target tissue from the ultrasound image; automatically determines the tissue edge of the target tissue by means of curve evolution and displays it in the target ultrasound image, without the need for manual identification of the target tissue, thereby improving the efficiency of locating the target tissue.

[0145] In addition, when the traditional pixel-based neural network segments the target tissue, there will be problems such as incomplete segmentation results, overflow or difficulty in image segmentation. The curve evolution method obtains a smoother curve after evolution and is suitable for images with blurred edges. Therefore, in this embodiment, the tissue edge of the target tissue is determined by using the curve evolution method, and the determined tissue edge is more accurate than the tissue edge calculated by the pixel-based neural network.

[0146] In addition, by selecting a frame of target ultrasound image with the largest number from the m frames of ultrasound images for analysis, the target tissue can be analyzed more comprehensively and the accuracy of the analysis result can be improved.

[0147] In addition, when determining the target ultrasound image, if the frame number of the ultrasound image is divisible by n, the target detection model is used to detect the target of the ultrasound image; and the tracking model is used to track the target of the n-1 frame image after the frame ultrasound image; because the tracking model is based on the detection result of the previous frame ultrasound image, the speed of calculating the detection result using the tracking model is faster than that of calculating the detection result using the detection model. At the same time, due to the omission, false detection and other unstable detection results that often occur in the detection model, using the tracking model to calculate the results can avoid the problems of the detection model and improve the accuracy of the detection results.

[0148] In addition, if the target detection model is used to detect the target of the 0th frame ultrasound image, and the subsequent ultrasound images only use the tracking model to track the target tissue, if there is a new target tissue, the new target tissue will be missed by the tracking model. Based on this, in this embodiment, the tracking model is used to track only part of the ultrasound images. When i is divisible by n, the target detection model is used again to identify the follicle target that has been tracked before the i-th frame, and the new target tissue can also be identified, thereby ensuring the accuracy of identifying the target tissue.

[0149] In addition, by setting the target detection model on the basis of the convolutional neural network, adding an attention layer and a feature fusion layer, the image features that the user expects to pay attention to can be extracted from the feature map, and the accuracy of the target detection model calculation detection results can be improved. At the same time, by fusing the underlying attention information and the mid-to-high-level attention information into a feature fusion layer, it can not only strengthen the detail information that cannot be observed by the mid-to-high-level attention layer, but also overcome the problem that the underlying attention layer cannot observe the target global information. By extracting features layer by layer from low to high through the feature extraction layer, and then fusing features layer by layer from high to low through the feature fusion layer, the local features and global features of the ultrasound image are modeled, which can better identify target tissues of different sizes. In addition, due to the high target overlap of the target tissue in the ultrasound image, it is easy to cause the problem of incomplete target recognition. By fusing the shallow features with the deep features, more comprehensive features can be obtained, which is conducive to identifying all target tissues.

[0150] In addition, by determining the tissue region of the target tissue based on the union of the detection results output by the target detection model and the detection results output by the tracking model, the target tissue in the target ultrasound image can be detected more comprehensively.

[0151] Optionally, based on the above embodiment, after obtaining the tissue edge of the target tissue, the monitoring device may also automatically measure the size of the target tissue to obtain the size data of the target tissue.

[0152] At this time, after step 103, the method for monitoring target tissues in ultrasound images further includes: calculating the size data of each target tissue based on the evolved curve of the target tissue; and displaying the size data of each target tissue.

[0153] Optionally, the size data of the target tissue includes but is not limited to: at least one of the area and diameter of the target tissue. Taking the target tissue as a follicle as an example, the diameter includes the long axis and the short axis of the follicle; or the diameter is the average value of the long axis and the short axis of the follicle. This embodiment does not limit the calculation method of the diameter of the follicle.

[0154] In one example, the size data of the target tissue includes the diameter of the target tissue. At this time, based on the evolved curve of each target tissue, the size data of the target tissue is calculated, including: using the evolved curve to perform ellipse fitting to obtain the major axis and minor axis of the fitted ellipse.

[0155] In one example, the size data of the target tissue includes the diameter and area of ​​the target tissue. At this time, based on the evolved curve of each target tissue, the size data of the target tissue is calculated, including: using the evolved curve to perform ellipse fitting to obtain the major axis and minor axis of the fitted ellipse; using the major axis and minor axis to calculate the area of ​​the ellipse to obtain the area of ​​the target tissue.

[0156] Optionally, when an ellipse cannot be fitted using the evolved curve, it indicates that the size of the target tissue is too small. In this case, the target tissue is ignored.

[0157] In other embodiments, the maximum value between any two points on the evolved curve can be determined as the major axis; the minimum value between any two points on the edge of the tissue can be determined as the minor axis; the area of ​​the target tissue can be calculated based on the major axis and the minor axis. This embodiment does not limit the method of calculating the diameter and area.

[0158] In addition, the size data of the target tissue may also include the volume of the target tissue. At this time, the size measurement of the target tissue corresponding to the image position includes: using multiple frames of ultrasound images including the target tissue to perform three-dimensional reconstruction of the target tissue; counting voxels of the three-dimensionally reconstructed target tissue to obtain the volume of the target tissue. Alternatively, using multiple frames of ultrasound images including the target tissue to perform three-dimensional reconstruction of the target tissue to obtain the three-dimensionally reconstructed target tissue; using a tetrahedron partitioning algorithm to calculate the volume of the three-dimensionally reconstructed target tissue to obtain the volume of the target tissue. This embodiment does not limit the data content of the size data and the method of calculating the volume.

[0159] In this embodiment, after obtaining the tissue edge of the target tissue, the size data of the target tissue is calculated based on the tissue edge, so that the automatic measurement of the target tissue can be realized, thereby improving the measurement efficiency of the target tissue.

[0160] Figure 7 7 is a block diagram of a monitoring device for target tissue in an ultrasound image provided by an embodiment of the present application. The device comprises at least the following modules: an image determination module 710 , a region determination module 720 , a curve evolution module 730 and an edge display module 740 .

[0161] An image determination module 710 is used to determine a target ultrasound image to be monitored from m consecutive ultrasound image frames, where the m ultrasound image frames are obtained by performing ultrasound scanning on the target tissue, and m is an integer greater than 1;

[0162] A region determination module 720 is used to obtain a tissue region of a target tissue in the target ultrasound image, where the tissue region is obtained by performing target detection on the target ultrasound image;

[0163] A curve evolution module 730, configured to perform curve evolution using the image data of the tissue region, so that the evolved curve coincides with the tissue edge in the tissue region;

[0164] The edge display module 740 is used to display the tissue edge of the target tissue in the target ultrasound image based on the evolved curve.

[0165] For relevant details, refer to the above method embodiment.

[0166] It should be noted that: the monitoring device for target tissue in an ultrasonic image provided in the above embodiment only uses the division of the above functional modules as an example when monitoring the target tissue in an ultrasonic image. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the monitoring device for target tissue in an ultrasonic image is divided into different functional modules to complete all or part of the functions described above. In addition, the monitoring device for target tissue in an ultrasonic image provided in the above embodiment and the monitoring method for target tissue in an ultrasonic image belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0167] See also Figure 8 , Figure 8 FIG. 1 is a schematic diagram of the structure of an ultrasonic device provided by an optional embodiment of the present invention, and the ultrasonic device can be a monitoring device for the target tissue in the above embodiment. Figure 8 As shown, the ultrasound device may include: an ultrasound probe, at least one processor 81, such as a CPU (Central Processing Unit), at least one communication interface 83, a memory 84, and at least one communication bus 82. The communication bus 82 is used to realize the connection and communication between these components. The communication interface 83 may include a display screen (Display) and a keyboard (Keyboard), and the optional communication interface 83 may also include a standard wired interface and a wireless interface. The memory 84 may be a high-speed RAM memory (Random Access Memory) or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 84 may optionally be at least one storage device located away from the aforementioned processor 81. The processor 81 may be combined with Figure 7 In the described device, the memory 84 stores an application program, and the processor 81 calls the program code stored in the memory 84 to execute any of the above method steps.

[0168] Ultrasonic probe: It consists of a piezoelectric element, a connector, and a supporting structure. In the transmitting mode, the ultrasonic probe converts electrical energy into mechanical energy, and the generated mechanical waves propagate toward the medium. In the receiving mode, the reflected mechanical waveform is received and converted into an electrical signal by the ultrasonic probe.

[0169] The communication bus 82 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The communication bus 82 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0170] Among them, the memory 84 may include a volatile memory (English: volatile memory), such as a random access memory (English: random-access memory, abbreviated: RAM); the memory may also include a non-volatile memory (English: non-volatile memory), such as a flash memory (English: flash memory), a hard disk drive (English: hard disk drive, abbreviated: HDD) or a solid-state drive (English: solid-state drive, abbreviated: SSD); the memory 84 may also include a combination of the above types of memory.

[0171] The processor 81 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and a NP.

[0172] The processor 81 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0173] Optionally, the memory 84 is also used to store program instructions. The processor 81 can call the program instructions to implement the present application Figure 1 The method for monitoring target tissue in ultrasound images is shown in the embodiments.

[0174] Optionally, the ultrasonic device may further include the following components:

[0175] Waveform generator: used to generate a data signal so that the pulse generator can generate a transmit pulse according to the data signal.

[0176] Transmit beam synthesizer: used to achieve electronic focusing and control of the sound beam of the multi-element ultrasound probe. And appropriately delay the transmitted signal for each element so that the transmitter signal reaches the target at the same time and generates the highest sound intensity at the target, that is, to obtain the strongest echo signal.

[0177] Pulse generator: used to generate transmit pulses.

[0178] T / R switch: Transmit / receive switch, used to control whether the ultrasound probe is currently in transmit mode or receive mode.

[0179] TGC gain: controls the amplifier gain to increase with the increase of detection depth to compensate for the attenuation of ultrasonic signals with propagation distance.

[0180] Analog-to-digital converter: used to convert analog signals into digital signals.

[0181] Receive beamformer: Similar to the transmit beamformer, it is used to electronically focus and control the acoustic beam of a multi-element ultrasound probe. It also applies appropriate delays to the received echoes to linearly superimpose the echo signals from multiple elements to achieve the highest sensitivity.

[0182] Matched filter: A filter that matches the transmitted code to achieve code compression.

[0183] Transverse filter: used to suppress the range sidelobe of the received signal.

[0184] Bandpass filter: used to select and filter out the desired frequency band from the received signal.

[0185] Amplitude detector: used to detect the amplitude of the received signal.

[0186] Scan conversion / display: used to perform data conversion and then display ultrasound images.

[0187] Of course, in actual implementation, the ultrasonic device may include more or fewer components, and this embodiment does not limit this.

[0188] Optionally, the present application also provides a computer-readable storage medium, in which a program is stored, and the program is loaded and executed by a processor to implement the method for monitoring target tissue in an ultrasound image of the above method embodiment.

[0189] Optionally, the present application also provides a computer product, which includes a computer-readable storage medium, in which a program is stored, and the program is loaded and executed by a processor to implement the method for monitoring target tissue in an ultrasound image of the above method embodiment.

[0190] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0191] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for monitoring target tissue in an ultrasound image, characterized in that: The method comprises: Determining a target ultrasound image to be monitored from m consecutive ultrasound image frames, wherein the m ultrasound image frames are obtained by performing ultrasound scanning on the target tissue, and m is an integer greater than 1; determining the target ultrasound image to be monitored from the m consecutive ultrasound image frames comprises: sequentially determining the number of target tissues in the m ultrasound image frames; and determining a frame of ultrasound image with the largest number of tissues as the target ultrasound image; The sequentially determining the number of target tissues in the m-frame ultrasound image comprises: for the i-th frame ultrasound image in the m-frame ultrasound image, when the value of i is divisible by n, inputting the i-th frame ultrasound image into a pre-trained target detection model to obtain a detection result of the target tissue in the i-th frame ultrasound image; the detection result includes the tissue area and the number of tissues of the target tissue; when the value of i is not divisible by n, inputting the detection result of the target tissue in the i-1-th frame ultrasound image into a pre-created tracking model to obtain a detection result of the target tissue in the i-th frame ultrasound image; the tracking model is used to predict the position of the target tissue in the i-th frame ultrasound image based on the position of the target tissue in the i-1-th frame ultrasound image, and determining a search range in the i-th frame ultrasound image based on the predicted position, and searching for the target tissue within the search range; wherein i is an integer from 0 to m-1 in sequence; and n is a positive integer with a fixed value; Acquire a tissue region of a target tissue in the target ultrasound image; Using the image data of the tissue region to perform curve evolution so that the evolved curve coincides with the tissue edge in the tissue region; Based on the evolved curve, a tissue edge of the target tissue is displayed in the target ultrasound image.

2. The method according to claim 1, characterized in that In a case where the frame number of the target ultrasound image cannot be divided by n, acquiring the tissue region of the target tissue in the target ultrasound image includes: Inputting the target ultrasound image into the target detection model to obtain a detection result of the target ultrasound image; A union of the detection result output by the target detection model and the detection result output by the tracking model is determined to obtain a tissue region of the target tissue in the target ultrasound image.

3. The method according to claim 2, characterized in that The target detection model is obtained by training a neural network model using training data, including: Acquire training data, the training data including a sample image and a label of a target tissue in the sample image; the label is obtained by marking the target tissue in the sample image with a rectangular frame, and the label includes a marking probability of the target tissue and position information of the actual rectangular frame; the position information of the actual rectangular frame includes a center point position, width, and height of the actual rectangular frame; Input the sample images in the training data into the neural network model to obtain the model results; the model results include: the predicted probability of each target tissue and the position information of the predicted rectangular box of each target tissue, and the position information of the predicted rectangular box includes the center point position, width and height of the predicted rectangular box; The model result and the label corresponding to the sample image are input into the preset loss function to obtain the loss function value. The loss function is expressed by the following formula: Where B represents the total number of target tissues in a sample image; q represents the qth target tissue; p q represents the labeling probability of the qth target tissue; represents the model result of the qth target organization output by the neural network, The value range of is [0,1]; represents the square error between the labeled probability and the predicted probability; x q The horizontal coordinate of the center point of the actual rectangular box to which the qth target tissue belongs; The horizontal coordinate of the center point of the predicted rectangular box to which the qth target tissue belongs; y q The ordinate represents the center point of the actual rectangular box to which the qth target tissue belongs; represents the vertical coordinate of the center point of the predicted rectangular box to which the qth target tissue belongs; w q Indicates the width of the actual rectangular box to which the qth target tissue belongs; represents the width of the predicted rectangular box to which the qth target tissue belongs; h q Indicates the height of the actual rectangular box to which the qth target organization belongs; Indicates the height of the predicted rectangular box to which the qth target tissue belongs; The neural network model is iteratively trained based on the loss function value until the neural network model converges or the number of iterations reaches a preset number, and the target detection model is obtained.

4. The method according to claim 1 or 2, characterized in that: The target detection model includes: k layers of feature extraction layers connected in sequence, each feature extraction layer is used to extract image features of input data to obtain a feature map; k is an integer greater than 1; Each of the first k-1 feature extraction layers is connected to an attention layer, and the attention layer is used to provide attention information in the feature map output by the feature extraction layer; The k-1th feature fusion layer is connected to the k-th feature extraction layer and the k-1th attention layer respectively; the k-1th feature fusion layer is used to perform feature fusion on the feature map output by the k-th feature extraction layer and the attention information output by the k-1th attention layer to obtain the k-1th fusion feature; The j-th feature fusion layer is connected to the j-th attention layer and the j+1-th feature fusion layer respectively; the j-th feature fusion layer is used to fuse the attention information output by the j-th attention layer with the j+1-th fusion feature output by the j+1-th feature fusion layer to obtain the j-th fusion feature; j is an integer from k-2 to 1 in sequence; An output layer connected to the first-layer feature fusion layer, wherein the output layer is used to calculate the detection result of the target ultrasound image based on the first-layer fusion features output by the first-layer feature fusion layer.

5. The method according to claim 1, characterized in that The step of using the image data of the tissue region to perform curve evolution so that the evolved curve coincides with the tissue edge in the tissue region includes: Preprocessing the tissue region to obtain a preprocessed tissue region; the preprocessing includes histogram equalization processing and / or median filtering processing; The preprocessed tissue region is input into a pre-created parametric active contour model to obtain a closed curve that coincides with the tissue edge, wherein the closed curve is the evolved curve.

6. The method according to claim 5, characterized in that The step of inputting the pre-processed tissue region into a pre-created parameter active contour model to obtain a closed curve that coincides with the tissue edge comprises: generating an initialization envelope curve based on the center position of the preprocessed tissue region; Based on the initialized envelope curve, using the energy function of the parametric active contour model to perform curve evolution, and determining the envelope curve corresponding to the minimum function value of the energy function as the closed curve; The energy function is used to integrate the capacity of each point on the envelope curve; the energy function includes a first integral term, a second integral term and a third integral term; the first integral term is used to control the smoothness of the envelope curve; the second integral term is used to control the continuity of the envelope curve; the third integral term is used to control the envelope curve to approach the tissue edge; The energy function is expressed as follows: Among them, s represents any point on the current envelope curve, and the total energy calculated by the energy function is the integration of the energy of each point on the previous envelope curve; is the first integral term, which represents the elastic capacity generated when the evolution of point s changes, and is used to control the smoothness of the envelope curve; is the second integral term, which represents the bending energy generated when the evolution of point s changes, and is used to control the continuity of the envelope curve; is the third integral term, which represents the image gradient value at point s when the evolution of point s changes, and is used to control the envelope curve to approach the tissue edge.

7. The method according to claim 1, characterized in that The step of displaying the tissue edge of the target tissue in the target ultrasound image based on the evolved curve comprises: For each target tissue in the target ultrasound image, the tissue edge of the target tissue is depicted in the target ultrasound image by an evolved curve corresponding to each target tissue.

8. A device for monitoring target tissue in an ultrasonic image, characterized in that: The device includes a processor and a memory; a program is stored in the memory, and the program is loaded and executed by the processor to implement the method for monitoring target tissue in an ultrasound image as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The storage medium stores a program, and when the program is executed by the processor, it is used to implement the method for monitoring target tissue in an ultrasound image as described in any one of claims 1 to 7.

Images